Compare commits
144 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9589967808 | ||
|
|
311846dc26 | ||
|
|
533f8c97fd | ||
|
|
75c289f526 | ||
|
|
e97865166d | ||
|
|
53144bfcf1 | ||
|
|
ee97a1b62e | ||
|
|
01a3d31734 | ||
|
|
a546ff0819 | ||
|
|
67bc032d26 | ||
|
|
8f816cf1af | ||
|
|
e2b028abdc | ||
|
|
00fb4bdf07 | ||
|
|
7067c75525 | ||
|
|
ef82328d74 | ||
|
|
be59e3a958 | ||
|
|
99427d888a | ||
|
|
8ec2da952f | ||
|
|
a5d007e258 | ||
|
|
78b82e3a96 | ||
|
|
1303a0eaae | ||
|
|
ab41d2d1ed | ||
|
|
d78f2b8650 | ||
|
|
5e5df483e2 | ||
|
|
009ec9e1d4 | ||
|
|
a08dc374c8 | ||
|
|
5b0ec742c4 | ||
|
|
04e28bc2d8 | ||
|
|
966b9e8c61 | ||
|
|
5f75619063 | ||
|
|
03ff9357d7 | ||
|
|
d05032963d | ||
|
|
7d9f9a627c | ||
|
|
2d19908119 | ||
|
|
d487e1c466 | ||
|
|
159b848234 | ||
|
|
53e6890f20 | ||
|
|
83c124f7fc | ||
|
|
05348f3981 | ||
|
|
e40c8fbca0 | ||
|
|
27684eef60 | ||
|
|
cc740c92c9 | ||
|
|
e673304221 | ||
|
|
974fcec21e | ||
|
|
36e3a1f72e | ||
|
|
5993e79e00 | ||
|
|
484e8d4298 | ||
|
|
fa2034d30b | ||
|
|
a84d681782 | ||
|
|
ab844a77b8 | ||
|
|
6c7a174424 | ||
|
|
474e4f07f8 | ||
|
|
7c254ec5dc | ||
|
|
09a741f540 | ||
|
|
a540d2dd3f | ||
|
|
3f8c406898 | ||
|
|
6065a3d70c | ||
|
|
34e9d5084c | ||
|
|
a56101197a | ||
|
|
2a82393c8e | ||
|
|
418eada0b5 | ||
|
|
b2e70a7257 | ||
|
|
b7a2636e97 | ||
|
|
d0e5e169cc | ||
|
|
7e18424d11 | ||
|
|
00d307439c | ||
|
|
5df189d306 | ||
|
|
bd1c5c0330 | ||
|
|
77e75b9975 | ||
|
|
e506ac6b28 | ||
|
|
88bc63cf82 | ||
|
|
a05542c06c | ||
|
|
ffc4e06ac6 | ||
|
|
81076375b8 | ||
|
|
66f369f3a9 | ||
|
|
7cba7b432d | ||
|
|
dd98d96d7e | ||
|
|
8ede892c14 | ||
|
|
08c62ab8cd | ||
|
|
e8efef476b | ||
|
|
c6403a65d2 | ||
|
|
6a8d638853 | ||
|
|
782929b006 | ||
|
|
4997ad9115 | ||
|
|
857a30d980 | ||
|
|
a40e69cca0 | ||
|
|
e069a79a32 | ||
|
|
d9dfe15bfe | ||
|
|
1dbd2e5990 | ||
|
|
7e9da4133d | ||
|
|
163b5c9c81 | ||
|
|
c6a3638a2f | ||
|
|
44f78cb03e | ||
|
|
61552ad6ca | ||
|
|
fa66ed866c | ||
|
|
08b4bd1f03 | ||
|
|
c536a1c312 | ||
|
|
adb18fe711 | ||
|
|
7d1e748b0e | ||
|
|
3a9d22c76b | ||
|
|
417e1a65b6 | ||
|
|
5762167186 | ||
|
|
faa9eb0093 | ||
|
|
005363c5cd | ||
|
|
478723ca2c | ||
|
|
05dcfa2a13 | ||
|
|
eff95c09f1 | ||
|
|
097e641789 | ||
|
|
f107c94d05 | ||
|
|
93242edc86 | ||
|
|
eb395414ab | ||
|
|
04191efc51 | ||
|
|
9bbf49bc6b | ||
|
|
dfed7e3621 | ||
|
|
4dc4ae88de | ||
|
|
c50c657193 | ||
|
|
c04e13ef93 | ||
|
|
5f3dc152c3 | ||
|
|
a64491a59e | ||
|
|
a7319ce1d6 | ||
|
|
02c94ea7ed | ||
|
|
c7ed37789e | ||
|
|
744e522aea | ||
|
|
af03ae0a8b | ||
|
|
31ec9baa95 | ||
|
|
1fc93bd05d | ||
|
|
5bee683661 | ||
|
|
b3d74e56a0 | ||
|
|
b5d48a6503 | ||
|
|
2ea4da0f1e | ||
|
|
152b4391b8 | ||
|
|
4b2787a8b2 | ||
|
|
d8de4a7863 | ||
|
|
14f7155a23 | ||
|
|
d188c37466 | ||
|
|
14fcc5d485 | ||
|
|
1d8d934a8a | ||
|
|
bdb2949b4b | ||
|
|
4cf1c7eee4 | ||
|
|
04ec99f17a | ||
|
|
150561abf2 | ||
|
|
58a0597c86 | ||
|
|
a842581785 | ||
|
|
8dcf0b31a2 |
3
.gitattributes
vendored
@@ -35,3 +35,6 @@
|
||||
*.so binary
|
||||
*.dll binary
|
||||
*.webp binary
|
||||
|
||||
# autogenerated constrained solve pnp code
|
||||
photon-targeting/src/main/native/cpp/photon/constrained_solvepnp/generate/**/* linguist-generated
|
||||
|
||||
4
.github/CODEOWNERS
vendored
@@ -1,2 +1,6 @@
|
||||
# These owners will be the default owners for everything in the repo.
|
||||
* @PhotonVision/program-devs
|
||||
docs/* @PhotonVision/doc-maintainers
|
||||
photonlib-java-examples/* @PhotonVision/doc-maintainers
|
||||
photonlib-cpp-examples/* @PhotonVision/doc-maintainers
|
||||
photonlib-python-examples/* @PhotonVision/doc-maintainers
|
||||
|
||||
17
.github/pull_request_template.md
vendored
Normal file
@@ -0,0 +1,17 @@
|
||||
## Description
|
||||
|
||||
<!-- What changed? Why? (the code + comments should speak for itself on the "how") -->
|
||||
|
||||
<!-- Fun screenshots or a cool video or something are super helpful as well. If this touches platform-specific behavior, this is where test evidence should be collected. -->
|
||||
|
||||
<!-- Any issues this pull request closes or pull requests this supersedes should be linked with `Closes #issuenumber`. -->
|
||||
|
||||
## Meta
|
||||
|
||||
Merge checklist:
|
||||
- [ ] Pull Request title is [short, imperative summary](https://cbea.ms/git-commit/) of proposed changes
|
||||
- [ ] The description documents the _what_ and _why_
|
||||
- [ ] If this PR changes behavior or adds a feature, user documentation is updated
|
||||
- [ ] If this PR touches photon-serde, all messages have been regenerated and hashes have not changed unexpectedly
|
||||
- [ ] If this PR touches configuration, this is backwards compatible with settings back to v2024.3.1
|
||||
- [ ] If this PR addresses a bug, a regression test for it is added
|
||||
169
.github/workflows/build.yml
vendored
@@ -1,21 +1,18 @@
|
||||
name: Build
|
||||
|
||||
on:
|
||||
# Run on pushes to master and pushed tags, and on pull requests against master, but ignore the docs folder
|
||||
# Run on pushes to main and pushed tags, and on pull requests against main, but ignore the docs folder
|
||||
push:
|
||||
branches: [ master ]
|
||||
branches: [ main ]
|
||||
tags:
|
||||
- 'v*'
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
pull_request:
|
||||
branches: [ master ]
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
branches: [ main ]
|
||||
merge_group:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-client:
|
||||
@@ -39,8 +36,20 @@ jobs:
|
||||
name: built-client
|
||||
path: photon-client/dist/
|
||||
build-examples:
|
||||
name: "Build Examples"
|
||||
runs-on: ubuntu-22.04
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- os: windows-2022
|
||||
architecture: x64
|
||||
- os: macos-14
|
||||
architecture: aarch64
|
||||
- os: ubuntu-22.04
|
||||
|
||||
name: "Photonlib - Build Examples - ${{ matrix.os }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
@@ -56,23 +65,14 @@ jobs:
|
||||
- name: Install RoboRIO Toolchain
|
||||
run: ./gradlew installRoboRioToolchain
|
||||
# Need to publish to maven local first, so that C++ sim can pick it up
|
||||
# Still haven't figured out how to make the vendordep file be copied before trying to build examples
|
||||
- name: Publish photonlib to maven local
|
||||
run: |
|
||||
chmod +x gradlew
|
||||
./gradlew publishtomavenlocal -x check
|
||||
run: ./gradlew photon-targeting:publishtomavenlocal photon-lib:publishtomavenlocal -x check
|
||||
- name: Build Java examples
|
||||
working-directory: photonlib-java-examples
|
||||
run: |
|
||||
chmod +x gradlew
|
||||
./gradlew copyPhotonlib -x check
|
||||
./gradlew build -x check
|
||||
run: ./gradlew build
|
||||
- name: Build C++ examples
|
||||
working-directory: photonlib-cpp-examples
|
||||
run: |
|
||||
chmod +x gradlew
|
||||
./gradlew copyPhotonlib -x check
|
||||
./gradlew build -x check
|
||||
run: ./gradlew build
|
||||
build-gradle:
|
||||
name: "Gradle Build"
|
||||
runs-on: ubuntu-22.04
|
||||
@@ -92,19 +92,17 @@ jobs:
|
||||
- name: Install mrcal deps
|
||||
run: sudo apt-get update && sudo apt-get install -y libcholmod3 liblapack3 libsuitesparseconfig5
|
||||
- name: Gradle Build
|
||||
run: |
|
||||
chmod +x gradlew
|
||||
./gradlew photon-targeting:build photon-core:build photon-server:build -x check
|
||||
run: ./gradlew photon-targeting:build photon-core:build photon-server:build -x check
|
||||
- name: Gradle Tests
|
||||
run: ./gradlew testHeadless -i --stacktrace
|
||||
- name: Gradle Coverage
|
||||
run: ./gradlew jacocoTestReport
|
||||
- name: Publish Coverage Report
|
||||
uses: codecov/codecov-action@v3
|
||||
uses: codecov/codecov-action@v4
|
||||
with:
|
||||
file: ./photon-server/build/reports/jacoco/test/jacocoTestReport.xml
|
||||
- name: Publish Core Coverage Report
|
||||
uses: codecov/codecov-action@v3
|
||||
uses: codecov/codecov-action@v4
|
||||
with:
|
||||
file: ./photon-core/build/reports/jacoco/test/jacocoTestReport.xml
|
||||
build-offline-docs:
|
||||
@@ -133,6 +131,36 @@ jobs:
|
||||
with:
|
||||
name: built-docs
|
||||
path: docs/build/html
|
||||
|
||||
build-photonlib-vendorjson:
|
||||
name: "Build Vendor JSON"
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Install Java 17
|
||||
uses: actions/setup-java@v4
|
||||
with:
|
||||
java-version: 17
|
||||
distribution: temurin
|
||||
|
||||
# grab all tags
|
||||
- run: git fetch --tags --force
|
||||
|
||||
# Generate the JSON and give it the ""standard""" name maven gives it
|
||||
- run: |
|
||||
./gradlew photon-lib:generateVendorJson
|
||||
export VERSION=$(git describe --tags --match=v*)
|
||||
mv photon-lib/build/generated/vendordeps/photonlib.json photon-lib/build/generated/vendordeps/photonlib-$(git describe --tags --match=v*).json
|
||||
|
||||
# Upload it here so it shows up in releases
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: photonlib-vendor-json
|
||||
path: photon-lib/build/generated/vendordeps/photonlib-*.json
|
||||
|
||||
build-photonlib-host:
|
||||
env:
|
||||
MACOSX_DEPLOYMENT_TARGET: 13
|
||||
@@ -162,9 +190,8 @@ jobs:
|
||||
distribution: temurin
|
||||
architecture: ${{ matrix.architecture }}
|
||||
- run: git fetch --tags --force
|
||||
- run: |
|
||||
chmod +x gradlew
|
||||
./gradlew photon-targeting:build photon-lib:build -i
|
||||
- run: ./gradlew photon-targeting:build photon-lib:build -i
|
||||
name: Build with Gradle
|
||||
- run: ./gradlew photon-lib:publish photon-targeting:publish
|
||||
name: Publish
|
||||
env:
|
||||
@@ -182,7 +209,7 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- container: wpilib/roborio-cross-ubuntu:2024-22.04
|
||||
- container: wpilib/roborio-cross-ubuntu:2025-24.04
|
||||
artifact-name: Athena
|
||||
build-options: "-Ponlylinuxathena"
|
||||
- container: wpilib/raspbian-cross-ubuntu:bullseye-22.04
|
||||
@@ -204,13 +231,9 @@ jobs:
|
||||
git config --global --add safe.directory /__w/photonvision/photonvision
|
||||
- name: Build PhotonLib
|
||||
# We don't need to run tests, since we specify only non-native platforms
|
||||
run: |
|
||||
chmod +x gradlew
|
||||
./gradlew photon-targeting:build photon-lib:build ${{ matrix.build-options }} -i -x test
|
||||
run: ./gradlew photon-targeting:build photon-lib:build ${{ matrix.build-options }} -i -x test
|
||||
- name: Publish
|
||||
run: |
|
||||
chmod +x gradlew
|
||||
./gradlew photon-lib:publish photon-targeting:publish ${{ matrix.build-options }}
|
||||
run: ./gradlew photon-lib:publish photon-targeting:publish ${{ matrix.build-options }}
|
||||
env:
|
||||
ARTIFACTORY_API_KEY: ${{ secrets.ARTIFACTORY_API_KEY }}
|
||||
if: github.event_name == 'push' && github.repository_owner == 'photonvision'
|
||||
@@ -306,13 +329,9 @@ jobs:
|
||||
with:
|
||||
name: built-docs
|
||||
path: photon-server/src/main/resources/web/docs
|
||||
- run: |
|
||||
chmod +x gradlew
|
||||
./gradlew photon-targeting:jar photon-server:shadowJar -PArchOverride=${{ matrix.arch-override }}
|
||||
- run: ./gradlew photon-targeting:jar photon-server:shadowJar -PArchOverride=${{ matrix.arch-override }}
|
||||
if: ${{ (matrix.arch-override != 'none') }}
|
||||
- run: |
|
||||
chmod +x gradlew
|
||||
./gradlew photon-server:shadowJar
|
||||
- run: ./gradlew photon-server:shadowJar
|
||||
if: ${{ (matrix.arch-override == 'none') }}
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
@@ -372,7 +391,7 @@ jobs:
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: RaspberryPi
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-4/photonvision_raspi.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_raspi.img.xz
|
||||
cpu: cortex-a7
|
||||
image_additional_mb: 0
|
||||
extraOpts: -Djdk.lang.Process.launchMechanism=vfork
|
||||
@@ -411,49 +430,61 @@ jobs:
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: RaspberryPi
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_raspi.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_raspi.img.xz
|
||||
cpu: cortex-a7
|
||||
image_additional_mb: 0
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: limelight2
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_limelight.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_limelight.img.xz
|
||||
cpu: cortex-a7
|
||||
image_additional_mb: 0
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: limelight3
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_limelight3.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_limelight3.img.xz
|
||||
cpu: cortex-a7
|
||||
image_additional_mb: 0
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: limelight3G
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_limelight3g.img.xz
|
||||
cpu: cortex-a7
|
||||
image_additional_mb: 0
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: orangepi5
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_opi5.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_opi5.img.xz
|
||||
cpu: cortex-a8
|
||||
image_additional_mb: 1024
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: orangepi5b
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_opi5b.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_opi5b.img.xz
|
||||
cpu: cortex-a8
|
||||
image_additional_mb: 1024
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: orangepi5plus
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_opi5plus.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_opi5plus.img.xz
|
||||
cpu: cortex-a8
|
||||
image_additional_mb: 1024
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: orangepi5pro
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_opi5pro.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_opi5pro.img.xz
|
||||
cpu: cortex-a8
|
||||
image_additional_mb: 1024
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: orangepi5max
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.0-beta-6/photonvision_opi5max.img.xz
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_opi5max.img.xz
|
||||
cpu: cortex-a8
|
||||
image_additional_mb: 1024
|
||||
- os: ubuntu-22.04
|
||||
artifact-name: LinuxArm64
|
||||
image_suffix: rock5c
|
||||
image_url: https://github.com/PhotonVision/photon-image-modifier/releases/download/v2025.0.3/photonvision_rock5c.img.xz
|
||||
cpu: cortex-a8
|
||||
image_additional_mb: 1024
|
||||
|
||||
@@ -506,6 +537,11 @@ jobs:
|
||||
with:
|
||||
merge-multiple: true
|
||||
pattern: photonlib-offline
|
||||
# Download vendor json
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
merge-multiple: true
|
||||
pattern: photonlib-vendor-json
|
||||
# Download all images
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
@@ -528,20 +564,35 @@ jobs:
|
||||
# Upload all jars and xz archives
|
||||
# Split into two uploads to work around max size limits in action-gh-releases
|
||||
# https://github.com/softprops/action-gh-release/issues/353
|
||||
- uses: softprops/action-gh-release@v2.0.8
|
||||
- uses: softprops/action-gh-release@v2.0.9
|
||||
with:
|
||||
files: |
|
||||
**/*orangepi5*.xz
|
||||
**/@(*orangepi5*|*rock5*).xz
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
- uses: softprops/action-gh-release@v2.0.8
|
||||
- uses: softprops/action-gh-release@v2.0.9
|
||||
with:
|
||||
files: |
|
||||
**/!(*orangepi5*).xz
|
||||
**/!(*orangepi5*|*rock5*).xz
|
||||
**/*.jar
|
||||
**/photonlib*.json
|
||||
**/photonlib*.zip
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
dispatch:
|
||||
name: dispatch
|
||||
needs: [build-photonlib-vendorjson, release]
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: peter-evans/repository-dispatch@v3
|
||||
if: |
|
||||
github.repository == 'PhotonVision/photonvision' &&
|
||||
startsWith(github.ref, 'refs/tags/v')
|
||||
with:
|
||||
token: ${{ secrets.VENDOR_JSON_REPO_PUSH_TOKEN }}
|
||||
repository: PhotonVision/vendor-json-repo
|
||||
event-type: tag
|
||||
client-payload: '{"run_id": "${{ github.run_id }}", "package_version": "${{ github.ref_name }}"}'
|
||||
|
||||
17
.github/workflows/cut-new-tag.yml
vendored
Normal file
@@ -0,0 +1,17 @@
|
||||
name: Cut a new tag
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag_name:
|
||||
type: string
|
||||
description: The full name of the new tag to push to the latest commit to main
|
||||
|
||||
jobs:
|
||||
push_tag:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- run: git tag ${{ github.event.inputs.tag_name }}
|
||||
- run: git push origin ${{ github.event.inputs.tag_name }}
|
||||
40
.github/workflows/lint-format.yml
vendored
@@ -1,21 +1,14 @@
|
||||
name: Lint and Format
|
||||
|
||||
on:
|
||||
# Run on pushes to master and pushed tags, and on pull requests against master, but ignore the docs folder
|
||||
# Run on pushes to main and pushed tags, and on pull requests against main, but ignore the docs folder
|
||||
push:
|
||||
branches: [ master ]
|
||||
branches: [ main ]
|
||||
tags:
|
||||
- 'v*'
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
pull_request:
|
||||
branches: [ master ]
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
branches: [ main ]
|
||||
merge_group:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
|
||||
@@ -26,18 +19,18 @@ jobs:
|
||||
name: "wpiformat"
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v4
|
||||
- name: Fetch all history and metadata
|
||||
run: |
|
||||
git fetch --prune --unshallow
|
||||
git checkout -b pr
|
||||
git branch -f master origin/master
|
||||
git branch -f main origin/main
|
||||
- name: Set up Python 3.8
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: 3.11
|
||||
- name: Install wpiformat
|
||||
run: pip3 install wpiformat==2024.41
|
||||
run: pip3 install wpiformat==2024.45
|
||||
- name: Run
|
||||
run: wpiformat
|
||||
- name: Check output
|
||||
@@ -45,7 +38,7 @@ jobs:
|
||||
- name: Generate diff
|
||||
run: git diff HEAD > wpiformat-fixes.patch
|
||||
if: ${{ failure() }}
|
||||
- uses: actions/upload-artifact@v3
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: wpiformat fixes
|
||||
path: wpiformat-fixes.patch
|
||||
@@ -54,16 +47,15 @@ jobs:
|
||||
name: "Java Formatting"
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-java@v4
|
||||
with:
|
||||
java-version: 17
|
||||
distribution: temurin
|
||||
- run: |
|
||||
chmod +x gradlew
|
||||
./gradlew spotlessCheck
|
||||
- run: ./gradlew spotlessCheck
|
||||
name: Run spotless
|
||||
|
||||
client-lint-format:
|
||||
name: "PhotonClient Lint and Formatting"
|
||||
@@ -72,9 +64,9 @@ jobs:
|
||||
working-directory: photon-client
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v4
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v3
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 18
|
||||
- name: Install Dependencies
|
||||
@@ -87,11 +79,11 @@ jobs:
|
||||
name: "Check server index.html not changed"
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v4
|
||||
- name: Fetch all history and metadata
|
||||
run: |
|
||||
git fetch --prune --unshallow
|
||||
git checkout -b pr
|
||||
git branch -f master origin/master
|
||||
git branch -f main origin/main
|
||||
- name: Check index.html not changed
|
||||
run: git --no-pager diff --exit-code origin/master photon-server/src/main/resources/web/index.html
|
||||
run: git --no-pager diff --exit-code origin/main photon-server/src/main/resources/web/index.html
|
||||
|
||||
25
.github/workflows/photon-code-docs.yml
vendored
@@ -1,21 +1,18 @@
|
||||
name: Photon Code Documentation
|
||||
|
||||
on:
|
||||
# Run on pushes to master and pushed tags, and on pull requests against master, but ignore the docs folder
|
||||
# Run on pushes to main and pushed tags, and on pull requests against main, but ignore the docs folder
|
||||
push:
|
||||
branches: [ master ]
|
||||
branches: [ main ]
|
||||
tags:
|
||||
- 'v*'
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
pull_request:
|
||||
branches: [ master ]
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
branches: [ main ]
|
||||
merge_group:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
|
||||
permissions:
|
||||
@@ -23,10 +20,6 @@ permissions:
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-client:
|
||||
name: "PhotonClient Build"
|
||||
@@ -85,7 +78,7 @@ jobs:
|
||||
|
||||
- run: find .
|
||||
- name: copy file via ssh password
|
||||
if: github.ref == 'refs/heads/master'
|
||||
if: github.ref == 'refs/heads/main'
|
||||
uses: appleboy/scp-action@v0.1.7
|
||||
with:
|
||||
host: ${{ secrets.WEBMASTER_SSH_HOST }}
|
||||
|
||||
20
.github/workflows/photonvision-docs.yml
vendored
@@ -2,22 +2,24 @@ name: PhotonVision Sphinx Documentation Checks
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ master ]
|
||||
paths:
|
||||
- 'docs/**'
|
||||
- '.github/**'
|
||||
branches: [ main ]
|
||||
pull_request:
|
||||
branches: [ master ]
|
||||
paths:
|
||||
- 'docs/**'
|
||||
- '.github/**'
|
||||
branches: [ main ]
|
||||
merge_group:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-22.04
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
|
||||
26
.github/workflows/python.yml
vendored
@@ -5,19 +5,16 @@ permissions:
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ master ]
|
||||
branches: [ main ]
|
||||
tags:
|
||||
- 'v*'
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
pull_request:
|
||||
branches: [ master ]
|
||||
paths:
|
||||
- '**'
|
||||
- '!docs/**'
|
||||
- '.github/**'
|
||||
branches: [ main ]
|
||||
merge_group:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
buildAndDeploy:
|
||||
@@ -37,7 +34,7 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install setuptools wheel pytest
|
||||
pip install setuptools wheel pytest mypy
|
||||
|
||||
- name: Build wheel
|
||||
working-directory: ./photon-lib/py
|
||||
@@ -50,6 +47,13 @@ jobs:
|
||||
pip install --no-cache-dir dist/*.whl
|
||||
pytest
|
||||
|
||||
- name: Run mypy type checking
|
||||
uses: liskin/gh-problem-matcher-wrap@v3
|
||||
with:
|
||||
linters: mypy
|
||||
run: |
|
||||
mypy --show-column-numbers --config-file photon-lib/py/pyproject.toml photon-lib
|
||||
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@master
|
||||
|
||||
24
.gitignore
vendored
@@ -131,27 +131,12 @@ New client/photon-client/*
|
||||
*.jfr
|
||||
.DS_Store
|
||||
# *.iml
|
||||
photon-server/build
|
||||
photon-server/photon-vision
|
||||
photon-server/src/main/resources/web
|
||||
photon-server/src/main/java/org/photonvision/PhotonVersion.java
|
||||
photon-server/src/main/generated/native/include/org_photonvision_raspi_PicamJNI.h
|
||||
*.bin
|
||||
.gradle
|
||||
.gradle/*
|
||||
photonvision_config
|
||||
build/spotlessJava
|
||||
build/*
|
||||
build
|
||||
photon-lib/src/main/java/org/photonvision/PhotonVersion.java
|
||||
photon-lib/bin/main/images/*
|
||||
/photonlib-java-examples/bin/
|
||||
photon-lib/src/generate/native/include/PhotonVersion.h
|
||||
.gitattributes
|
||||
lib/*
|
||||
photon-server/lib/libapriltag.so
|
||||
photon-server/bin/main/nativelibraries/apriltag/*
|
||||
photon-server/src/main/resources/nativelibraries/apriltag/*
|
||||
bin*/
|
||||
build*/
|
||||
|
||||
photonlib-java-examples/*/vendordeps/*
|
||||
photonlib-cpp-examples/*/vendordeps/*
|
||||
@@ -161,10 +146,7 @@ photonlib-cpp-examples/*/vendordeps/*
|
||||
photonlib-cpp-examples/*/networktables.json.bck
|
||||
photonlib-java-examples/*/networktables.json.bck
|
||||
*.sqlite
|
||||
photon-server/src/main/resources/web/index.html
|
||||
photon-lib/src/generate/native/cpp/PhotonVersion.cpp
|
||||
|
||||
photon-server/src/main/resources/web/*
|
||||
venv
|
||||
|
||||
.venv/*
|
||||
.venv
|
||||
|
||||
@@ -20,6 +20,9 @@ modifiableFileExclude {
|
||||
\.ico$
|
||||
\.rknn$
|
||||
gradlew
|
||||
photon-lib/py/photonlibpy/generated/
|
||||
photon-targeting/src/main/native/cpp/photon/constrained_solvepnp/generate/
|
||||
photon-targeting/src/generated/
|
||||
}
|
||||
|
||||
includeProject {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# PhotonVision
|
||||
|
||||
[](https://github.com/PhotonVision/photonvision/actions?query=workflow%3ACI) [](https://codecov.io/gh/PhotonVision/photonvision) [](https://discord.gg/wYxTwym)
|
||||
[](https://discord.gg/wYxTwym)
|
||||
|
||||
PhotonVision is the free, fast, and easy-to-use computer vision solution for the *FIRST* Robotics Competition. You can read an overview of our features [on our website](https://photonvision.org). You can find our comprehensive documentation [here](https://docs.photonvision.org).
|
||||
|
||||
@@ -17,7 +17,7 @@ If you are interested in contributing code or documentation to the project, plea
|
||||
## Documentation
|
||||
|
||||
- Our main documentation page: [docs.photonvision.org](https://docs.photonvision.org)
|
||||
- Photon UI demo: [demo.photonvision.org](https://demo.photonvision.org) (or [manual link](https://photonvision.github.io/photonvision/built-client/))
|
||||
- Photon UI demo: [http://photonvision.global/](http://photonvision.global/) (or [manual link](https://photonvision.github.io/photonvision/built-client/))
|
||||
- Javadocs: [javadocs.photonvision.org](https://javadocs.photonvision.org) (or [manual link](https://photonvision.github.io/photonvision/built-docs/javadoc/))
|
||||
- C++ Doxygen [cppdocs.photonvision.org](https://cppdocs.photonvision.org) (or [manual link](https://photonvision.github.io/photonvision/built-docs/doxygen/html/))
|
||||
|
||||
@@ -67,7 +67,7 @@ sudo apt install libcholmod3 liblapack3 libsuitesparseconfig5
|
||||
|
||||
PhotonVision was forked from [Chameleon Vision](https://github.com/Chameleon-Vision/chameleon-vision/). Thank you to everyone who worked on the original project.
|
||||
|
||||
* [WPILib](https://github.com/wpilibsuite) - Specifically [cscore](https://github.com/wpilibsuite/allwpilib/tree/master/cscore), [CameraServer](https://github.com/wpilibsuite/allwpilib/tree/master/cameraserver), [NTCore](https://github.com/wpilibsuite/allwpilib/tree/master/ntcore), and [OpenCV](https://github.com/wpilibsuite/thirdparty-opencv).
|
||||
* [WPILib](https://github.com/wpilibsuite) - Specifically [cscore](https://github.com/wpilibsuite/allwpilib/tree/main/cscore), [CameraServer](https://github.com/wpilibsuite/allwpilib/tree/main/cameraserver), [NTCore](https://github.com/wpilibsuite/allwpilib/tree/main/ntcore), and [OpenCV](https://github.com/wpilibsuite/thirdparty-opencv).
|
||||
|
||||
* [Apache Commons](https://commons.apache.org/) - Specifically [Commons Math](https://commons.apache.org/proper/commons-math/), and [Commons Lang](https://commons.apache.org/proper/commons-lang/)
|
||||
|
||||
|
||||
33
build.gradle
@@ -1,14 +1,17 @@
|
||||
import edu.wpi.first.toolchain.*
|
||||
|
||||
plugins {
|
||||
id "java"
|
||||
id "cpp"
|
||||
id "com.diffplug.spotless" version "6.24.0"
|
||||
id "edu.wpi.first.wpilib.repositories.WPILibRepositoriesPlugin" version "2020.2"
|
||||
id "edu.wpi.first.GradleRIO" version "2025.1.1-beta-1"
|
||||
id "edu.wpi.first.GradleRIO" version "2025.3.1"
|
||||
id 'edu.wpi.first.WpilibTools' version '1.3.0'
|
||||
id 'com.google.protobuf' version '0.9.3' apply false
|
||||
id 'edu.wpi.first.GradleJni' version '1.1.0'
|
||||
id "org.ysb33r.doxygen" version "1.0.4" apply false
|
||||
id 'com.gradleup.shadow' version '8.3.4' apply false
|
||||
id "com.github.node-gradle.node" version "7.0.1" apply false
|
||||
id "org.hidetake.ssh" version "2.11.2" apply false
|
||||
}
|
||||
|
||||
allprojects {
|
||||
@@ -30,16 +33,16 @@ ext.allOutputsFolder = file("$project.buildDir/outputs")
|
||||
apply from: "versioningHelper.gradle"
|
||||
|
||||
ext {
|
||||
wpilibVersion = "2025.1.1-beta-1"
|
||||
wpilibVersion = "2025.3.1"
|
||||
wpimathVersion = wpilibVersion
|
||||
openCVYear = "2024"
|
||||
openCVversion = "4.8.0-4"
|
||||
openCVYear = "2025"
|
||||
openCVversion = "4.10.0-3"
|
||||
joglVersion = "2.4.0"
|
||||
javalinVersion = "5.6.2"
|
||||
libcameraDriverVersion = "dev-v2023.1.0-14-g787ab59"
|
||||
rknnVersion = "dev-v2024.0.1-4-g0db16ac"
|
||||
libcameraDriverVersion = "v2025.0.3"
|
||||
rknnVersion = "dev-v2025.0.0-1-g33b6263"
|
||||
frcYear = "2025"
|
||||
mrcalVersion = "dev-v2024.0.0-24-gc1efcf0";
|
||||
mrcalVersion = "v2025.0.0";
|
||||
|
||||
|
||||
pubVersion = versionString
|
||||
@@ -67,7 +70,7 @@ spotless {
|
||||
java {
|
||||
target fileTree('.') {
|
||||
include '**/*.java'
|
||||
exclude '**/build/**', '**/build-*/**', "photon-core\\src\\main\\java\\org\\photonvision\\PhotonVersion.java", "photon-lib\\src\\main\\java\\org\\photonvision\\PhotonVersion.java", "**/src/generated/**"
|
||||
exclude '**/build/**', '**/build-*/**', '**/src/generated/**'
|
||||
}
|
||||
toggleOffOn()
|
||||
googleJavaFormat()
|
||||
@@ -87,16 +90,6 @@ spotless {
|
||||
trimTrailingWhitespace()
|
||||
endWithNewline()
|
||||
}
|
||||
format 'xml', {
|
||||
target fileTree('.') {
|
||||
include '**/*.xml'
|
||||
exclude '**/build/**', '**/build-*/**', "**/.idea/**"
|
||||
}
|
||||
eclipseWtp('xml')
|
||||
trimTrailingWhitespace()
|
||||
indentWithSpaces(2)
|
||||
endWithNewline()
|
||||
}
|
||||
format 'misc', {
|
||||
target fileTree('.') {
|
||||
include '**/*.md', '**/.gitignore'
|
||||
@@ -109,7 +102,7 @@ spotless {
|
||||
}
|
||||
|
||||
wrapper {
|
||||
gradleVersion '8.4'
|
||||
gradleVersion '8.11'
|
||||
}
|
||||
|
||||
ext.getCurrentArch = {
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
import argparse
|
||||
import base64
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
import mrcal
|
||||
import numpy as np
|
||||
from wpimath.geometry import Quaternion as _Quat
|
||||
|
||||
|
||||
|
||||
@@ -12,5 +12,6 @@ modifiableFileExclude {
|
||||
\.ico$
|
||||
\.rknn$
|
||||
\.svg$
|
||||
\.woff2$
|
||||
gradlew
|
||||
}
|
||||
|
||||
@@ -6,4 +6,4 @@ PhotonVision is a free open-source vision processing software for FRC teams.
|
||||
|
||||
This repository is the source code for our ReadTheDocs documentation, which can be found [here](https://docs.photonvision.org).
|
||||
|
||||
[Contribution and formatting guidelines for this project](https://docs.photonvision.org/en/latest/docs/contributing/photonvision-docs/index.html)
|
||||
[Contribution and formatting guidelines for this project](https://docs.photonvision.org/en/latest/docs/contributing/index.html)
|
||||
|
||||
BIN
docs/source/_static/assets/poseest_demo.mp4
Normal file
@@ -1,16 +1,3 @@
|
||||
/*!
|
||||
* Font Awesome 4.7.0 by @davegandy - http://fontawesome.io - @fontawesome
|
||||
* License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License)
|
||||
*/
|
||||
|
||||
@font-face {
|
||||
font-family: FontAwesome;
|
||||
src: url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713);
|
||||
src: url(fonts/fontawesome-webfont.eot?674f50d287a8c48dc19ba404d20fe713?#iefix&v=4.7.0) format("embedded-opentype"), url(fonts/fontawesome-webfont.woff2?af7ae505a9eed503f8b8e6982036873e) format("woff2"), url(fonts/fontawesome-webfont.woff?fee66e712a8a08eef5805a46892932ad) format("woff"), url(fonts/fontawesome-webfont.ttf?b06871f281fee6b241d60582ae9369b9) format("truetype"), url(fonts/fontawesome-webfont.svg?912ec66d7572ff821749319396470bde#fontawesomeregular) format("svg");
|
||||
font-weight: 400;
|
||||
font-style:normal
|
||||
}
|
||||
|
||||
.code-block-caption>.headerlink, dl dt>.headerlink, h1>.headerlink, h2>.headerlink, h3>.headerlink, h4>.headerlink, h5>.headerlink, h6>.headerlink, p.caption>.headerlink, table>caption>.headerlink {
|
||||
font-family: FontAwesome;
|
||||
font-size: 0.75em;
|
||||
|
||||
6
docs/source/_static/css/v4-font-face.min.css
vendored
Normal file
@@ -0,0 +1,6 @@
|
||||
/*!
|
||||
* Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com
|
||||
* License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License)
|
||||
* Copyright 2024 Fonticons, Inc.
|
||||
*/
|
||||
@font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-solid-900.woff2) format("woff2"),url(../webfonts/fa-solid-900.ttf) format("truetype")}@font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-brands-400.woff2) format("woff2"),url(../webfonts/fa-brands-400.ttf) format("truetype")}@font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-regular-400.woff2) format("woff2"),url(../webfonts/fa-regular-400.ttf) format("truetype");unicode-range:u+f003,u+f006,u+f014,u+f016-f017,u+f01a-f01b,u+f01d,u+f022,u+f03e,u+f044,u+f046,u+f05c-f05d,u+f06e,u+f070,u+f087-f088,u+f08a,u+f094,u+f096-f097,u+f09d,u+f0a0,u+f0a2,u+f0a4-f0a7,u+f0c5,u+f0c7,u+f0e5-f0e6,u+f0eb,u+f0f6-f0f8,u+f10c,u+f114-f115,u+f118-f11a,u+f11c-f11d,u+f133,u+f147,u+f14e,u+f150-f152,u+f185-f186,u+f18e,u+f190-f192,u+f196,u+f1c1-f1c9,u+f1d9,u+f1db,u+f1e3,u+f1ea,u+f1f7,u+f1f9,u+f20a,u+f247-f248,u+f24a,u+f24d,u+f255-f25b,u+f25d,u+f271-f274,u+f278,u+f27b,u+f28c,u+f28e,u+f29c,u+f2b5,u+f2b7,u+f2ba,u+f2bc,u+f2be,u+f2c0-f2c1,u+f2c3,u+f2d0,u+f2d2,u+f2d4,u+f2dc}@font-face{font-family:"FontAwesome";font-display:block;src:url(../webfonts/fa-v4compatibility.woff2) format("woff2"),url(../webfonts/fa-v4compatibility.ttf) format("truetype");unicode-range:u+f041,u+f047,u+f065-f066,u+f07d-f07e,u+f080,u+f08b,u+f08e,u+f090,u+f09a,u+f0ac,u+f0ae,u+f0b2,u+f0d0,u+f0d6,u+f0e4,u+f0ec,u+f10a-f10b,u+f123,u+f13e,u+f148-f149,u+f14c,u+f156,u+f15e,u+f160-f161,u+f163,u+f175-f178,u+f195,u+f1f8,u+f219,u+f27a}
|
||||
BIN
docs/source/_static/webfonts/fa-brands-400.woff2
Normal file
BIN
docs/source/_static/webfonts/fa-regular-400.woff2
Normal file
BIN
docs/source/_static/webfonts/fa-solid-900.woff2
Normal file
BIN
docs/source/_static/webfonts/fa-v4compatibility.woff2
Normal file
@@ -10,7 +10,8 @@
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
# import os
|
||||
import os
|
||||
|
||||
# import sys
|
||||
# sys.path.insert(0, os.path.abspath('.'))
|
||||
|
||||
@@ -44,7 +45,7 @@ extensions = [
|
||||
|
||||
ogp_site_url = "https://docs.photonvision.org/en/latest/"
|
||||
ogp_site_name = "PhotonVision Documentation"
|
||||
ogp_image = "https://raw.githubusercontent.com/PhotonVision/photonvision-docs/master/source/assets/RectLogo.png"
|
||||
ogp_image = "https://raw.githubusercontent.com/PhotonVision/photonvision-docs/main/source/assets/RectLogo.png"
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ["_templates"]
|
||||
@@ -78,6 +79,7 @@ source_suffix = [".rst", ".md"]
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_css_file("css/v4-font-face.min.css")
|
||||
app.add_css_file("css/pv-icons.css")
|
||||
|
||||
|
||||
@@ -120,15 +122,33 @@ html_theme_options = {
|
||||
"color-api-overall": "#101010",
|
||||
"color-inline-code-background": "#0d0d0d",
|
||||
},
|
||||
"footer_icons": [
|
||||
{
|
||||
"name": "GitHub",
|
||||
"url": "https://github.com/photonvision/photonvision",
|
||||
"html": """
|
||||
<svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 16 16">
|
||||
<path fill-rule="evenodd" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0 0 16 8c0-4.42-3.58-8-8-8z"></path>
|
||||
</svg>
|
||||
""",
|
||||
"class": "",
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
suppress_warnings = ["epub.unknown_project_files"]
|
||||
|
||||
sphinx_tabs_valid_builders = ["epub", "linkcheck"]
|
||||
|
||||
# -- Options for linkcheck -------------------------------------------------
|
||||
|
||||
# Excluded links for linkcheck
|
||||
# These should be periodically checked by hand to ensure that they are still functional
|
||||
linkcheck_ignore = ["https://www.raspberrypi.com/software/"]
|
||||
linkcheck_ignore = [R"https://www.raspberrypi.com/software/", R"http://10\..+"]
|
||||
|
||||
token = os.environ.get("GITHUB_TOKEN", None)
|
||||
if token:
|
||||
linkcheck_auth = [(R"https://github.com/.+", token)]
|
||||
|
||||
# MyST configuration (https://myst-parser.readthedocs.io/en/latest/configuration.html)
|
||||
myst_enable_extensions = ["colon_fence"]
|
||||
|
||||
@@ -3,27 +3,31 @@
|
||||
## Before Competition
|
||||
|
||||
- Ensure you have spares of the relevant electronics if you can afford it (switch, coprocessor, cameras, etc.).
|
||||
- Download the latest release .jar onto your computer and update your Pi if necessary (only update if the release is labeled "critical" or similar, we do not recommend updating right before an event in case there are unforeseen bugs).
|
||||
- Test out PhotonVision at your home setup.
|
||||
- Ensure that you have set up SmartDashboard / Shuffleboard to view your camera streams during matches.
|
||||
- Follow all the recommendations under the Networking section in installation (network switch and static IP).
|
||||
- Use high quality ethernet cables that have been rigorously tested.
|
||||
- Set up port forwarding using the guide in the Networking section in installation.
|
||||
- Stay on the latest version of PhotonVision until you have tested your full robot system to be functional.
|
||||
- Some time before the competition, lock down the version you are using and do not upgrade unless you encounter a critical bug.
|
||||
- Have a copy of the installation image for the version you are using on your programming laptop, in case re-imaging (without internet) is needed.
|
||||
- Extensively test at your home setup. Practice tuning from scratch under different lighting conditions.
|
||||
- Use SmartDashboard / Shuffleboard to view your camera streams during practice.
|
||||
- Confirm you have followed all the recommendations under the Networking section in installation (network switch and static IP).
|
||||
- Only use high quality ethernet cables that have been rigorously tested.
|
||||
- Set up RIO USB port forwarding using the guide in the Networking section in installation.
|
||||
|
||||
## During the Competition
|
||||
|
||||
- Make sure you take advantage of the field calibration time given at the start of the event:
|
||||
- Bring your robot to the field at the allotted time.
|
||||
- Turn on your robot and pull up the dashboard on your driver station.
|
||||
- Point your robot at the AprilTags(s) and ensure you get a consistent tracking (you hold one AprilTag consistently, the ceiling lights aren't detected, etc.).
|
||||
- If you have problems with your pipeline, go to the pipeline tuning section and retune the pipeline using the guide there.
|
||||
- Move the robot close, far, angled, and around the field to ensure no extra AprilTags are found.
|
||||
- Go to a practice match to ensure everything is working correctly.
|
||||
- Use the field calibration time given at the start of the event:
|
||||
- Bring your robot to the field at the allotted time.
|
||||
- Make sure the field has match-accurate lighting conditions active.
|
||||
- Turn on your robot and pull up the dashboard on your driver station.
|
||||
- Point your robot at the targets and ensure you get a consistent tracking (you hold one targets consistently, the ceiling lights aren't detected, etc.).
|
||||
- If you have problems with your pipeline, go to the pipeline tuning section and retune the pipeline using the guide there.
|
||||
- Move the robot close, far, angled, and around the field to ensure no extra targets are found.
|
||||
- Monitor camera feeds during a practice match to ensure everything is working correctly.
|
||||
- After field calibration, use the "Export Settings" button in the "Settings" page to create a backup.
|
||||
- Do this for each coprocessor on your robot that runs PhotonVision, and name your exports with meaningful names.
|
||||
- This will contain camera information/calibration, pipeline information, network settings, etc.
|
||||
- In the event of software/hardware failures (IE lost SD Card, broken device), you can then use the "Import Settings" button and select "All Settings" to restore your settings.
|
||||
- This effectively works as a snapshot of your PhotonVision data that can be restored at any point.
|
||||
- Before every match, check the ethernet connection going into your coprocessor and that it is seated fully.
|
||||
- Ensure that exposure is as low as possible and that you don't have the dashboard up when you don't need it to reduce bandwidth.
|
||||
- Do this for each coprocessor on your robot that runs PhotonVision, and name your exports with meaningful names.
|
||||
- This will contain camera information/calibration, pipeline information, network settings, etc.
|
||||
- In the event of software/hardware failures (IE lost SD Card, broken device), you can then use the "Import Settings" button and select "All Settings" to restore your settings.
|
||||
- This effectively works as a snapshot of your PhotonVision data that can be restored at any point.
|
||||
- Before every match:
|
||||
- Check the ethernet and USB connectors are seated fully.
|
||||
- Close streaming dashboards when you don't need them to reduce bandwidth.
|
||||
- Stream at as low of a resolution as possible while still detecting AprilTags to stay within field bandwidth limits.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Filesystem Directory
|
||||
|
||||
PhotonVision stores and loads settings in the {code}`photonvision_config` directory, in the same folder as the PhotonVision JAR is stored. On the Pi image as well as the Gloworm, this is in the {code}`/opt/photonvision` directory. The contents of this directory can be exported as a zip archive from the settings page of the interface, under "export settings". This export will contain everything detailed below. These settings can later be uploaded using "import settings", to restore configurations from previous backups.
|
||||
PhotonVision stores and loads settings in the {code}`photonvision_config` directory, in the same folder as the PhotonVision JAR is stored. On supported hardware, this is in the {code}`/opt/photonvision` directory. The contents of this directory can be exported as a zip archive from the settings page of the interface, under "export settings". This export will contain everything detailed below. These settings can later be uploaded using "import settings", to restore configurations from previous backups.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
@@ -12,20 +12,20 @@ The directory structure is outlined below.
|
||||
```
|
||||
|
||||
- calibImgs
|
||||
- Images saved from the last run of the calibration routine
|
||||
- Images saved from the last run of the calibration routine
|
||||
- cameras
|
||||
- Contains a subfolder for each camera. This folder contains the following files:
|
||||
- pipelines folder, which contains a {code}`json` file for each user-created pipeline.
|
||||
- config.json, which contains all camera-specific configuration. This includes FOV, pitch, current pipeline index, and calibration data
|
||||
- drivermode.json, which contains settings for the driver mode pipeline
|
||||
- Contains a subfolder for each camera. This folder contains the following files:
|
||||
- pipelines folder, which contains a {code}`json` file for each user-created pipeline.
|
||||
- config.json, which contains all camera-specific configuration. This includes FOV, pitch, current pipeline index, and calibration data
|
||||
- drivermode.json, which contains settings for the driver mode pipeline
|
||||
- imgSaves
|
||||
- Contains images saved with the input/output save commands.
|
||||
- Contains images saved with the input/output save commands.
|
||||
- logs
|
||||
- Contains timestamped logs in the format {code}`photonvision-YYYY-MM-D_HH-MM-SS.log`. Note that on Pi or Gloworm these timestamps will likely be significantly behind the real time.
|
||||
- Contains timestamped logs in the format {code}`photonvision-YYYY-MM-D_HH-MM-SS.log`. These timestamps will likely be significantly behind the real time. Coprocessors on the robot have no way to get current time.
|
||||
- hardwareSettings.json
|
||||
- Contains hardware settings. Currently this includes only the LED brightness.
|
||||
- Contains hardware settings. Currently this includes only the LED brightness.
|
||||
- networkSettings.json
|
||||
- Contains network settings, including team number (or remote network tables address), static/dynamic settings, and hostname.
|
||||
- Contains network settings, including team number (or remote network tables address), static/dynamic settings, and hostname.
|
||||
|
||||
## Importing and Exporting Settings
|
||||
|
||||
@@ -41,10 +41,13 @@ The entire settings directory can be exported as a ZIP archive from the settings
|
||||
A variety of files can be imported back into PhotonVision:
|
||||
|
||||
- ZIP Archive ({code}`.zip`)
|
||||
- Useful for restoring a full configuration from a different PhotonVision instance.
|
||||
- Useful for restoring a full configuration from a different PhotonVision instance.
|
||||
- Single Config File
|
||||
- Currently-supported Files
|
||||
- {code}`hardwareConfig.json`
|
||||
- {code}`hardwareSettings.json`
|
||||
- {code}`networkSettings.json`
|
||||
- Useful for simple hardware or network configuration tasks without overwriting all settings.
|
||||
- Currently-supported Files
|
||||
- {code}`hardwareConfig.json`
|
||||
- {code}`hardwareSettings.json`
|
||||
- {code}`networkSettings.json`
|
||||
- Useful for simple hardware or network configuration tasks without overwriting all settings.
|
||||
|
||||
|
||||
|
||||
|
||||
|
Before Width: | Height: | Size: 81 KiB After Width: | Height: | Size: 81 KiB |
|
Before Width: | Height: | Size: 139 KiB After Width: | Height: | Size: 139 KiB |
|
Before Width: | Height: | Size: 122 KiB After Width: | Height: | Size: 122 KiB |
@@ -1,6 +1,6 @@
|
||||
# Installation & Setup
|
||||
# Advanced Installation
|
||||
|
||||
This page will help you install PhotonVision on your coprocessor, wire it, and properly setup the networking in order to start tracking targets.
|
||||
This page will help you install PhotonVision on non-supported coprocessor.
|
||||
|
||||
## Step 1: Software Install
|
||||
|
||||
@@ -14,25 +14,5 @@ You only need to install PhotonVision on the coprocessor/device that is being us
|
||||
:maxdepth: 3
|
||||
|
||||
sw_install/index
|
||||
updating
|
||||
```
|
||||
|
||||
## Step 2: Wiring
|
||||
|
||||
This section will walk you through how to wire your coprocessor to get power.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
wiring
|
||||
```
|
||||
|
||||
## Step 3: Networking
|
||||
|
||||
This section will walk you though how to connect your coprocessor to a network. This section is very important (and easy to get wrong), so we recommend you read it thoroughly.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
networking
|
||||
prerelease-software
|
||||
```
|
||||
@@ -0,0 +1,23 @@
|
||||
# Installing Pre-Release Versions
|
||||
|
||||
Pre-release/development version of PhotonVision can be tested by installing/downloading artifacts from Github Actions (see below), which are built automatically on commits to open pull requests and to PhotonVision's `main` branch, or by {ref}`compiling PhotonVision locally <docs/contributing/building-photon:Build Instructions>`.
|
||||
|
||||
:::{warning}
|
||||
If testing a pre-release version of PhotonVision with a robot, PhotonLib must be updated to match the version downloaded! If not, packet schema definitions may not match and unexpected things will occur. To update PhotonLib, refer to {ref}`installing specific version of PhotonLib<docs/programming/photonlib/adding-vendordep:Install Specific Version - Java/C++>`.
|
||||
:::
|
||||
|
||||
GitHub Actions builds pre-release version of PhotonVision automatically on PRs and on each commit merged to main. To test a particular commit to main, navigate to the [PhotonVision commit list](https://github.com/PhotonVision/photonvision/commits/main/) and click on the check mark (below). Scroll to "Build / Build fat JAR - PLATFORM", click details, and then summary. From here, JAR and image files can be downloaded to be flashed or uploaded using "Offline Update".
|
||||
|
||||
```{image} images/gh_actions_1.png
|
||||
:alt: Github Actions Badge
|
||||
```
|
||||
|
||||
```{image} images/gh_actions_2.png
|
||||
:alt: Github Actions artifact list
|
||||
```
|
||||
|
||||
Built JAR files (but not image files) can also be downloaded from PRs before they are merged. Navigate to the PR in GitHub, and select Checks at the top. Click on "Build" to display the same artifact list as above.
|
||||
|
||||
```{image} images/gh_actions_3.png
|
||||
:alt: Github Actions artifacts from PR
|
||||
```
|
||||
|
Before Width: | Height: | Size: 115 KiB After Width: | Height: | Size: 115 KiB |
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 13 KiB |
@@ -1,16 +1,5 @@
|
||||
# Software Installation
|
||||
|
||||
## Supported Coprocessors
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
raspberry-pi
|
||||
limelight
|
||||
orange-pi
|
||||
snakeyes
|
||||
```
|
||||
|
||||
## Desktop Environments
|
||||
|
||||
```{toctree}
|
||||
@@ -29,5 +18,4 @@ mac-os
|
||||
other-coprocessors
|
||||
advanced-cmd
|
||||
romi
|
||||
gloworm
|
||||
```
|
||||
@@ -8,14 +8,14 @@ You do not need to install PhotonVision on a Windows PC in order to access the w
|
||||
|
||||
## Installing Java
|
||||
|
||||
PhotonVision requires a JDK installed and on the system path. JDK 11 is needed (different versions will not work). If you don't have JDK 11 already, run the following to install it:
|
||||
PhotonVision requires a JDK installed and on the system path. JDK 17 is needed (different versions will not work). If you don't have JDK 17 already, run the following to install it:
|
||||
|
||||
```
|
||||
$ sudo apt-get install openjdk-11-jdk
|
||||
$ sudo apt-get install openjdk-17-jdk
|
||||
```
|
||||
|
||||
:::{warning}
|
||||
Using a JDK other than JDK11 will cause issues when running PhotonVision and is not supported.
|
||||
Using a JDK other than JDK17 will cause issues when running PhotonVision and is not supported.
|
||||
:::
|
||||
|
||||
## Downloading the Latest Stable Release of PhotonVision
|
||||
@@ -5,17 +5,17 @@ Due to current [cscore](https://github.com/wpilibsuite/allwpilib/tree/main/cscor
|
||||
:::
|
||||
|
||||
:::{note}
|
||||
You do not need to install PhotonVision on a Windows PC in order to access the webdashboard (assuming you are using an external coprocessor like a Raspberry Pi).
|
||||
You do not need to install PhotonVision on a Mac in order to access the webdashboard (assuming you are using an external coprocessor like a Raspberry Pi).
|
||||
:::
|
||||
|
||||
VERY Limited macOS support is available.
|
||||
|
||||
## Installing Java
|
||||
|
||||
PhotonVision requires a JDK installed and on the system path. JDK 11 is needed (different versions will not work). You may already have this if you have installed WPILib. If not, [download and install it from here](https://adoptium.net/temurin/releases?version=11).
|
||||
PhotonVision requires a JDK installed and on the system path. JDK 17 is needed (different versions will not work). You may already have this if you have installed WPILib 2025+. If not, [download and install it from here](https://adoptium.net/temurin/releases?version=17).
|
||||
|
||||
:::{warning}
|
||||
Using a JDK other than JDK11 will cause issues when running PhotonVision and is not supported.
|
||||
Using a JDK other than JDK17 will cause issues when running PhotonVision and is not supported.
|
||||
:::
|
||||
|
||||
## Downloading the Latest Stable Release of PhotonVision
|
||||
@@ -23,13 +23,13 @@ $ sudo reboot now
|
||||
Your co-processor will require an Internet connection for this process to work correctly.
|
||||
:::
|
||||
|
||||
For installation on any other co-processors, we recommend reading the {ref}`advanced command line documentation <docs/installation/sw_install/advanced-cmd:Advanced Command Line Usage>`.
|
||||
For installation on any other co-processors, we recommend reading the {ref}`advanced command line documentation <docs/advanced-installation/sw_install/advanced-cmd:Advanced Command Line Usage>`.
|
||||
|
||||
## Updating PhotonVision
|
||||
|
||||
PhotonVision can be updated by downloading the latest jar file, copying it onto the processor, and restarting the service.
|
||||
|
||||
For example, from another computer, run the following commands. Substitute the correct username for "\[user\]" (e.g. Raspberry Pi uses "pi", Orange Pi uses "orangepi".)
|
||||
For example, from another computer, run the following commands. Substitute the correct username for "\[user\]" ( Provided images use username "pi")
|
||||
|
||||
```bash
|
||||
$ scp [jar name].jar [user]@photonvision.local:~/
|
||||
43
docs/source/docs/advanced-installation/sw_install/romi.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Romi Installation
|
||||
|
||||
The [Romi](https://docs.wpilib.org/en/latest/docs/romi-robot/index.html) is a small robot that can be controlled with the WPILib software. The main controller is a Raspberry Pi that must be imaged with [WPILibPi](https://docs.wpilib.org/en/latest/docs/romi-robot/imaging-romi.html) .
|
||||
|
||||
## Installation
|
||||
|
||||
The WPILibPi image includes FRCVision, which reserves USB cameras; to use PhotonVision, we need to edit the `/home/pi/runCamera` script to disable it. First we will need to make the file system writeable; the easiest way to do this is to go to `10.0.0.2` and choose "Writable" at the top.
|
||||
|
||||
SSH into the Raspberry Pi (using Windows command line, or a tool like [Putty](https://www.chiark.greenend.org.uk/~sgtatham/putty/) ) at the Romi's default address `10.0.0.2`. The default user is `pi`, and the password is `raspberry`.
|
||||
|
||||
:::.. The following paragraph can be restored when WPILibPi becomes compatible with the current version of PhotonVision.
|
||||
:::.. Follow the process for installing PhotonVision on {ref}`"Other Debian-Based Co-Processor Installation" <docs/advanced-installation/sw_install/other-coprocessors:Other Debian-Based Co-Processor Installation>`. As it mentions, this will require an internet connection so connecting the Raspberry Pi to an internet-connected router via an Ethernet cable will be the easiest solution. The pi must remain writable while you are following these steps!
|
||||
|
||||
:::..Temporary instructions explaining how to install the older version of PhotonVision on a Romi. Remove when no longer needed.
|
||||
:::{attention}
|
||||
The version of WPILibPi for the Romi is 2023.2.1, which is not compatible with the current version of PhotonVision. **If you are using WPILibPi 2023.2.1 on your Romi, you must install PhotonVision v2023.4.2 or earlier!**
|
||||
|
||||
To install a compatible version of PhotonVision, enter these commands in the SSH terminal connected to the Raspberry Pi. This will download and run the install script, which will install PhotonVision on your Raspberry Pi and configure it to run at startup.
|
||||
|
||||
```bash
|
||||
$ wget https://git.io/JJrEP -O install.sh
|
||||
$ sudo chmod +x install.sh
|
||||
$ sudo ./install.sh -v v2023.4.2
|
||||
```
|
||||
The install script requires an internet connection, so connecting the Raspberry Pi to an internet-connected router via an Ethernet cable will be the easiest solution. The pi must remain writable while you are following these steps!
|
||||
:::
|
||||
:::..End of temporary instructions.
|
||||
|
||||
Next, from the SSH terminal, run `sudo nano /home/pi/runCamera` then arrow down to the start of the exec line and press "Enter" to add a new line. Then add `#` before the exec command to comment it out. Then, arrow up to the new line and type `sleep 10000`. Hit "Ctrl + O" and then "Enter" to save the file. Finally press "Ctrl + X" to exit nano. Now, reboot the Romi by typing `sudo reboot now`.
|
||||
|
||||
```{image} images/nano.png
|
||||
|
||||
```
|
||||
|
||||
After the Romi reboots, you should be able to open the PhotonVision UI at: [`http://10.0.0.2:5800/`](http://10.0.0.2:5800/). From here, you can adjust settings and configure {ref}`Pipelines <docs/pipelines/index:Pipelines>`.
|
||||
|
||||
:::{warning}
|
||||
In order for settings, logs, etc. to be saved / take effect, ensure that PhotonVision is in writable mode.
|
||||
:::
|
||||
|
||||
:::{attention}
|
||||
When using an older version of PhotonVision, the user interface and features may be different than what appears in the online documentation. The [Documentation](http://10.0.0.2:5800/#/docs) link in the User Interface will open a bundled version of the documentation that matches the PhotonVision version running on your coprocessor.
|
||||
:::
|
||||
@@ -12,10 +12,14 @@ Bonjour provides more stable networking when using Windows PCs. Install [Bonjour
|
||||
|
||||
## Installing Java
|
||||
|
||||
PhotonVision requires a JDK installed and on the system path. **JDK 11 is needed** (different versions will not work). You may already have this if you have installed WPILib, but ensure that running `java -version` shows JDK 11. If not, [download and install it from here](https://adoptium.net/temurin/releases?version=11) and ensure that the new JDK is being used.
|
||||
PhotonVision requires a JDK installed and on the system path. **JDK 17 is needed. Windows Users must use the JDK that ships with WPILib.** [Download and install it from here.](https://github.com/wpilibsuite/allwpilib/releases/tag/v2025.3.1) Either ensure the only Java on your PATH is the WPILIB Java or specify it to gradle with `-Dorg.gradle.java.home=C:\Users\Public\wpilib\2025\jdk`:
|
||||
|
||||
```
|
||||
> ./gradlew run "-Dorg.gradle.java.home=C:\Users\Public\wpilib\2025\jdk"
|
||||
```
|
||||
|
||||
:::{warning}
|
||||
Using a JDK other than JDK11 will cause issues when running PhotonVision and is not supported.
|
||||
Using a JDK other than WPILIB's JDK17 will cause issues when running PhotonVision and is not supported.
|
||||
:::
|
||||
|
||||
## Downloading the Latest Stable Release of PhotonVision
|
||||
@@ -1,6 +1,6 @@
|
||||
# 2D AprilTag Tuning / Tracking
|
||||
|
||||
## Tracking Apriltags
|
||||
## Tracking AprilTags
|
||||
|
||||
Before you get started tracking AprilTags, ensure that you have followed the previous sections on installation, wiring and networking. Next, open the Web UI, go to the top right card, and switch to the "AprilTag" or "Aruco" type. You should see a screen similar to the one below.
|
||||
|
||||
@@ -21,7 +21,9 @@ AprilTag pipelines come with reasonable defaults to get you up and running with
|
||||
|
||||
### Target Family
|
||||
|
||||
Target families are defined by two numbers (before and after the h). The first number is the number of bits the tag is able to encode (which means more tags are available in the respective family) and the second is the hamming distance. Hamming distance describes the ability for error correction while identifying tag ids. A high hamming distance generally means that it will be easier for a tag to be identified even if there are errors. However, as hamming distance increases, the number of available tags decreases. The 2024 FRC game will be using 36h11 tags, which can be found [here](https://github.com/AprilRobotics/apriltag-imgs/tree/master/tag36h11).
|
||||
Target families are defined by two numbers (before and after the h). The first number is the number of bits the tag is able to encode (which means more tags are available in the respective family) and the second is the hamming distance. Hamming distance describes the ability for error correction while identifying tag ids. A high hamming distance generally means that it will be easier for a tag to be identified even if there are errors. However, as hamming distance increases, the number of available tags decreases.
|
||||
|
||||
The 2025 FRC game will be using 36h11 tags, which can be found [here](https://github.com/AprilRobotics/apriltag-imgs/tree/main/tag36h11).
|
||||
|
||||
### Decimate
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# About Apriltags
|
||||
# About AprilTags
|
||||
|
||||
```{image} images/pv-apriltag.png
|
||||
:align: center
|
||||
@@ -10,5 +10,5 @@ AprilTags are a common type of visual fiducial marker. Visual fiducial markers a
|
||||
A more technical explanation can be found in the [WPILib documentation](https://docs.wpilib.org/en/latest/docs/software/vision-processing/apriltag/apriltag-intro.html).
|
||||
|
||||
:::{note}
|
||||
You can get FIRST's [official PDF of the targets used in 2024 here](https://firstfrc.blob.core.windows.net/frc2024/FieldAssets/Apriltag_Images_and_User_Guide.pdf).
|
||||
You can get FIRST's [official PDF of the targets used in 2025 here](https://firstfrc.blob.core.windows.net/frc2025/FieldAssets/Apriltag_Images_and_User_Guide.pdf).
|
||||
:::
|
||||
|
||||
@@ -12,4 +12,4 @@ The AprilTag pipeline type is based on the [AprilTag](https://april.eecs.umich.e
|
||||
|
||||
## AruCo
|
||||
|
||||
The AruCo pipeline is based on the [AruCo](https://docs.opencv.org/4.8.0/d9/d6a/group__aruco.html) library implementation from OpenCV. It is ~2x higher fps and ~2x lower latency than the AprilTag pipeline type, but is less accurate. We recommend this pipeline type for teams that need to run at a higher framerate or have a lower powered device. This pipeline type is new for the 2024 season and is not as well tested as AprilTag.
|
||||
The AruCo pipeline is based on the [AruCo](https://docs.opencv.org/4.8.0/d9/d6a/group__aruco.html) library implementation from OpenCV. It is ~2x higher fps and ~2x lower latency than the AprilTag pipeline type, but is less accurate. We recommend this pipeline type for teams that need to run at a higher framerate or have a lower powered device. This pipeline type was new for the 2024 season.
|
||||
|
||||
@@ -6,6 +6,10 @@ PhotonVision can combine AprilTag detections from multiple simultaneously observ
|
||||
MultiTag requires an accurate field layout JSON to be uploaded! Differences between this layout and the tags' physical location will drive error in the estimated pose output.
|
||||
:::
|
||||
|
||||
:::{warning}
|
||||
For the 2025 Reefscape Season, there are two different field layouts. The first is the [welded field layout](https://github.com/wpilibsuite/allwpilib/blob/main/apriltag/src/main/native/resources/edu/wpi/first/apriltag/2025-reefscape-welded.json), which photonvision ships with. The second is the [Andymark field layout](https://github.com/wpilibsuite/allwpilib/blob/main/apriltag/src/main/native/resources/edu/wpi/first/apriltag/2025-reefscape-andymark.json). It is very important to ensure that you use the correct field layout, both in the [PhotonPoseEstimator](https://docs.photonvision.org/en/latest/docs/programming/photonlib/robot-pose-estimator.html#apriltags-and-photonposeestimator) and on the [coprocessor](https://docs.photonvision.org/en/latest/docs/apriltag-pipelines/multitag.html#updating-the-field-layout).
|
||||
:::
|
||||
|
||||
## Enabling MultiTag
|
||||
|
||||
Ensure that your camera is calibrated and 3D mode is enabled. Navigate to the Output tab and enable "Do Multi-Target Estimation". This enables MultiTag to use the uploaded field layout JSON to calculate your camera's pose in the field. This 3D transform will be shown as an additional table in the "targets" tab, along with the IDs of AprilTags used to compute this transform.
|
||||
@@ -51,7 +55,7 @@ The returned field to camera transform is a transform from the fixed field origi
|
||||
|
||||
## Updating the Field Layout
|
||||
|
||||
PhotonVision ships by default with the [2024 field layout JSON](https://github.com/wpilibsuite/allwpilib/blob/main/apriltag/src/main/native/resources/edu/wpi/first/apriltag/2024-crescendo.json). The layout can be inspected by navigating to the settings tab and scrolling down to the "AprilTag Field Layout" card, as shown below.
|
||||
PhotonVision ships by default with the [2025 welded field layout JSON](https://github.com/wpilibsuite/allwpilib/blob/main/apriltag/src/main/native/resources/edu/wpi/first/apriltag/2025-reefscape-welded.json). The layout can be inspected by navigating to the settings tab and scrolling down to the "AprilTag Field Layout" card, as shown below.
|
||||
|
||||
```{image} images/field-layout.png
|
||||
:alt: The currently saved field layout in the Photon UI
|
||||
|
||||
@@ -51,6 +51,10 @@ We'll next select a resolution to calibrate and populate our pattern spacing, ma
|
||||
:::{warning} Old OpenCV Pattern selector. This should be used in the case that the calibration image is generated from a version of OpenCV before version 4.6.0. This would include targets created by calib.io. If this selector is not set correctly the calibration will be completely invalid. For more info view [this GitHub issue](https://github.com/opencv/opencv_contrib/issues/3291).
|
||||
:::
|
||||
|
||||
:::{note}
|
||||
If you have a [calib.io](https://calib.io/) CharuCo Target you will have to enter the paramaters of your target. For example if your taget says "9x12 | Chceker Size: 30 mm | Marker Size: 22 mm | Dictionary: AruCo DICT 5x5", you would have to set the board type to Dict_5x5_1000, the pattern spacing to 1.1811 in (30 mm converted to inches), the marker size 0.866142 in (22 mm converted to inches), the board width to 12 and the board height to 9. If you chose the wrong tag family the baord wont be detected duting calibration. If you swap the width and height your calibration will have a very high error.
|
||||
:::
|
||||
|
||||
### 4. Take at calibration images from various angles.
|
||||
|
||||
Now, we'll capture images of our board from various angles. It's important to check that the board overlay matches the board in your image. The further the overdrawn points are from the true position of the chessboard corners, the less accurate the final calibration will be. We'll want to capture enough images to cover the whole camera's FOV (with a minimum of 12). Once we've got our images, we'll click "Finish calibration" and wait for the calibration process to complete. If all goes well, the mean error and FOVs will be shown in the table on the right. The FOV should be close to the camera's specified FOV (usually found in a datasheet) usually within + or - 10 degrees. The mean error should also be low, usually less than 1 pixel.
|
||||
@@ -83,7 +87,7 @@ More info on what these parameters mean can be found in [OpenCV's docs](https://
|
||||
|
||||
Below these outputs are the snapshots collected for calibration, along with a per-snapshot mean reprojection error. A snapshot with a larger reprojection error might indicate a bad snapshot, due to effects such as motion blur or misidentified chessboard corners.
|
||||
|
||||
Calibration images can also be extracted from the downloaded JSON file using [this Python script](https://raw.githubusercontent.com/PhotonVision/photonvision/master/devTools/calibrationUtils.py). This script will unpack calibration images, and also generate a VNL file for use [with mrcal](https://mrcal.secretsauce.net/).
|
||||
Calibration images can also be extracted from the downloaded JSON file using [this Python script](https://raw.githubusercontent.com/PhotonVision/photonvision/main/devTools/calibrationUtils.py). This script will unpack calibration images, and also generate a VNL file for use [with mrcal](https://mrcal.secretsauce.net/).
|
||||
|
||||
```
|
||||
python3 /path/to/calibrationUtils.py path/to/photon_calibration.json /path/to/output/folder
|
||||
|
||||
BIN
docs/source/docs/contributing/assets/vscode-gradle-args.png
Normal file
|
After Width: | Height: | Size: 87 KiB |
BIN
docs/source/docs/contributing/assets/vscode-gradle-tests.png
Normal file
|
After Width: | Height: | Size: 42 KiB |
BIN
docs/source/docs/contributing/assets/vscode-runner-tests.png
Normal file
|
After Width: | Height: | Size: 334 KiB |
@@ -18,7 +18,7 @@ You must install a set of Python dependencies in order to build the documentatio
|
||||
|
||||
In order to build the documentation, you can run the following command in the docs sub-folder. This will automatically build docs every time a file changes, and serves them locally at `localhost:8000` by default.
|
||||
|
||||
`~/photonvision/docs$ sphinx-autobuild --open-browser source/_build/html`
|
||||
`~/photonvision/docs$ sphinx-autobuild --open-browser source source/_build/html`
|
||||
|
||||
## Opening the Documentation
|
||||
|
||||
|
||||
@@ -139,25 +139,7 @@ The `deploy` command is tested against Raspberry Pi coprocessors. Other similar
|
||||
|
||||
### Using PhotonLib Builds
|
||||
|
||||
The build process includes the following task:
|
||||
|
||||
```{eval-rst}
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: Linux
|
||||
|
||||
``./gradlew generateVendorJson``
|
||||
|
||||
.. tab-item:: macOS
|
||||
|
||||
``./gradlew generateVendorJson``
|
||||
|
||||
.. tab-item:: Windows (cmd)
|
||||
|
||||
``gradlew generateVendorJson``
|
||||
```
|
||||
|
||||
This generates a vendordep JSON of your local build at `photon-lib/build/generated/vendordeps/photonlib.json`.
|
||||
The build process automatically generates a vendordep JSON of your local build at `photon-lib/build/generated/vendordeps/photonlib.json`.
|
||||
|
||||
The photonlib source can be published to your local maven repository after building:
|
||||
|
||||
@@ -185,30 +167,31 @@ repositories {
|
||||
}
|
||||
```
|
||||
|
||||
### VSCode Test Runner Extension
|
||||
|
||||
With the VSCode [Extension Pack for Java](https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-pack), you can get the Test Runner for Java and Gradle for Java extensions. This lets you easily run specific tests through the IDE:
|
||||
|
||||
```{image} assets/vscode-runner-tests.png
|
||||
:alt: An image showing how unit tests can be ran in VSCode through the Test Runner for Java extension.
|
||||
```
|
||||
|
||||
To correctly run PhotonVision tests this way, you must [delegate the tests to Gradle](https://code.visualstudio.com/docs/java/java-build#_delegate-tests-to-gradle). Debugging tests like this will [**not** currently](https://github.com/microsoft/build-server-for-gradle/issues/119) collect outputs.
|
||||
|
||||
### Debugging PhotonVision Running Locally
|
||||
|
||||
One way is by running the program using gradle with the {code}`--debug-jvm` flag. Run the program with {code}`./gradlew run --debug-jvm`, and attach to it with VSCode by adding the following to {code}`launch.json`. Note args can be passed with {code}`--args="foobar"`.
|
||||
Unit tests can instead be debugged through the ``test`` Gradle task for a specific subproject in VSCode, found in the Gradle tab:
|
||||
|
||||
```
|
||||
{
|
||||
// Use IntelliSense to learn about possible attributes.
|
||||
// Hover to view descriptions of existing attributes.
|
||||
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
|
||||
"version": "0.2.0",
|
||||
"configurations": [
|
||||
{
|
||||
"type": "java",
|
||||
"name": "Attach to Remote Program",
|
||||
"request": "attach",
|
||||
"hostName": "localhost",
|
||||
"port": "5005",
|
||||
"projectName": "photon-core",
|
||||
}
|
||||
]
|
||||
}
|
||||
```{image} assets/vscode-gradle-tests.png
|
||||
:alt: An image showing how unit tests can be debugged in VSCode through the Gradle for Java extension.
|
||||
```
|
||||
|
||||
PhotonVision can also be run using the gradle tasks plugin with {code}`"args": "--debug-jvm"` added to launch.json.
|
||||
However, this will run all tests in a subproject.
|
||||
|
||||
Similarly, a local instance of PhotonVision can be debugged in the same way using the Gradle ``run`` task. In both cases, additional arguments can be specified:
|
||||
|
||||
```{image} assets/vscode-gradle-args.png
|
||||
:alt: An image showing how VSCode gradle tasks can specify additional arguments.
|
||||
```
|
||||
|
||||
### Debugging PhotonVision Running on a CoProcessor
|
||||
|
||||
@@ -247,17 +230,15 @@ You can run one of the many built in examples straight from the command line, to
|
||||
|
||||
#### Running C++/Java
|
||||
|
||||
PhotonLib must first be published to your local maven repository, then the copy PhotonLib task will copy the generated vendordep json file into each example. After that, the simulateJava/simulateNative task can be used like a normal robot project. Robot simulation with attached debugger is technically possible by using simulateExternalJava and modifying the launch script it exports, though not yet supported.
|
||||
PhotonLib must first be published to your local maven repository. This will also copy the generated vendordep json file into each example. After that, the simulateJava/simulateNative task can be used like a normal robot project. Robot simulation with attached debugger is technically possible by using simulateExternalJava and modifying the launch script it exports, though not yet supported.
|
||||
|
||||
```
|
||||
~/photonvision$ ./gradlew publishToMavenLocal
|
||||
|
||||
~/photonvision$ cd photonlib-java-examples
|
||||
~/photonvision/photonlib-java-examples$ ./gradlew copyPhotonlib
|
||||
~/photonvision/photonlib-java-examples$ ./gradlew <example-name>:simulateJava
|
||||
|
||||
~/photonvision$ cd photonlib-cpp-examples
|
||||
~/photonvision/photonlib-cpp-examples$ ./gradlew copyPhotonlib
|
||||
~/photonvision/photonlib-cpp-examples$ ./gradlew <example-name>:simulateNative
|
||||
```
|
||||
|
||||
@@ -284,3 +265,11 @@ Then, run the examples:
|
||||
> cd photonlib-python-examples
|
||||
> run.bat <example name>
|
||||
```
|
||||
|
||||
#### Downloading Pipeline Artifacts
|
||||
|
||||
Using the [GitHub CLI](https://cli.github.com/), we can download artifacts from pipelines by run ID and name:
|
||||
|
||||
```
|
||||
~/photonvision$ gh run download 11759699679 -n jar-Linux
|
||||
```
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# Camera Matching
|
||||
|
||||
Diagrams generated by the [PlantUML UML editor](https://www.plantuml.com/plantuml/). Copy the image URLs below and decode in the editor to make changes.
|
||||
|
||||
## Initial Setup
|
||||
|
||||
When PhotonVision first starts, settings are loaded from disk and [VisionSources](https://javadocs.photonvision.org/org/photonvision/vision/processes/VisionSource.html) are created for every serialized & active [Camera Configuration](https://javadocs.photonvision.org/org/photonvision/common/configuration/CameraConfiguration.html)
|
||||
|
||||

|
||||
|
||||
## UI Workflow
|
||||
|
||||
A [background thread](https://javadocs.photonvision.org/org/photonvision/common/util/TimedTaskManager.html) will periodically query CSCore and Libcamera for what cameras we currently see connected. This list is provided to the web UI for display.
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
This UI allows users to "Activate" a camera that's never been seen before, or activate a CameraConfiguration we've seen before but was disabled. Allowing camera configurations to be saved but not loaded by default lets us support temporarily disabling/unplugging a camera without flooding log files.
|
||||
|
||||
Since our backend logic intentionally does not protect users from plugging camera B into the port that camera A was active on, the UI shall show a warning but vision processing will (attempt to) continue like normal.
|
||||
|
||||
### Activate New Camera
|
||||
|
||||
When a new camera (ie, one we can't match by-path to a deserialized CameraConfiguration) is activated, we'll create a spin up a new Vision Module for it
|
||||
|
||||

|
||||
|
||||
### Deactivate Camera
|
||||
|
||||
Deactivating a camera will release the native resources it owns, and return the CameraConfiguration to the pool of currently disabled cameras we can re-enable later.
|
||||
|
||||

|
||||
|
||||
### Reactivate a CameraConfig
|
||||
|
||||
When a new camera (ie, one we can't match by-path to a deserialized CameraConfiguration) is activated, we'll create and spin up a new Vision Module for it.
|
||||
|
||||

|
||||
|
||||
# Camera Matching Requirements
|
||||
|
||||
## Definitions
|
||||
- VALID USB PATH: a path in the form `/dev/v4l/by-path/[UUID]`
|
||||
- VIDEO DEVICE PATH: a CSCore-provided identifier derived from the V4L path `/dev/video[N]` on Linux, or an opaque string on Windows
|
||||
- UNIQUE NAME: an identifier that is unique within the set of all deserialized CameraConfigurations and unmatched USB cameras
|
||||
- I don't love this, it means that a USB camera matched to a VisionModule will share a UNIQUE NAME, right?
|
||||
- DESERIALIZED CAMERA CONFIGURATIONS: The set of camera configurations loaded from disk and provided to the VisionSourceManager. This configuration data structure includes the UNIQUE NAME
|
||||
- CURRENTLY ACTIVE CAMERAS: The set of VisionModules currently active and processing vision data, and associated metadata
|
||||
|
||||
## Startup:
|
||||
|
||||
- GIVEN An empty set of deserialized Camera Configurations
|
||||
<br>WHEN PhotonVision starts
|
||||
<br>THEN no VisionModules will be started
|
||||
|
||||
- GIVEN A valid set of deserialized Camera Configurations
|
||||
<br>WHEN PhotonVision starts
|
||||
<br>THEN VisionModules will be started FOR EACH un-DISABLED config
|
||||
|
||||
- GIVEN A valid set of deserialized Camera Configurations
|
||||
<br>WHEN PhotonVision starts
|
||||
<br>THEN VisionModules will NOT be started FOR EACH DISABLED config
|
||||
|
||||
- GIVEN A CameraConfiguration with a VALID USB PATH
|
||||
<br>WHEN a VisionModule is created
|
||||
<br>THEN The VisionModule shall open the camera using the USB path
|
||||
|
||||
- GIVEN A CameraConfiguration without a valid USB path
|
||||
<br>WHEN a VisionModule is created
|
||||
<br>THEN The VisionModule shall open the camera using the VIDEO DEVICE PATH
|
||||
|
||||
## Camera (re)enumeration:
|
||||
|
||||
- GIVEN a NEW USB CAMERA is available for enumeration
|
||||
<br>WHEN a USB camera is discovered by VisionSourceManager
|
||||
<br>AND the USB camera's VIDEO DEVICE PATH is not in the set of DESERIALIZED CAMERA CONFIGURATIONS
|
||||
<br>THEN a UNIQUE NAME will be assigned to the camera info
|
||||
|
||||
- GIVEN a NEW USB CAMERA is available for enumeration
|
||||
<br>WHEN a USB camera is discovered by VisionSourceManager
|
||||
<br>AND the USB camera's VIDEO DEVICE PATH is in the set of DESERIALIZED CAMERA CONFIGURATIONS
|
||||
<br>THEN a UNIQUE NAME equal to the matching DESERIALIZED CAMERA CONFIGURATION will be assigned to the camera info
|
||||
- This is a weird case. How -should- we handle this? see above
|
||||
|
||||
## Creating from a new camera
|
||||
|
||||
- Given: A UNIQUE NAME from a NEW USB CAMERA
|
||||
<br>WHEN I request a new VisionModule is created for this NEW USB CAMERA
|
||||
<br>AND the camera has a VALID USB PATH
|
||||
<br>AND the camera's VALID USB PATH is not in use by any CURRENTLY ACTIVE CAMERAS
|
||||
<br>THEN a NEW VisionModule will be started for the NEW USB CAMERA using the VALID USB PATH
|
||||
|
||||
- Given: A UNIQUE NAME from a NEW USB CAMERA
|
||||
<br>WHEN I request a new VisionModule is created for this NEW USB CAMERA
|
||||
<br>AND the camera does not have a VALID USB PATH
|
||||
<br>AND the camera's VIDEO DEVICE PATH is not in use by any CURRENTLY ACTIVE CAMERAS
|
||||
<br>THEN a NEW VisionModule will be started for the NEW USB CAMERA using the VIDEO DEVICE PATH
|
||||
|
||||
## Deactivate
|
||||
|
||||
- Given: A UNIQUE NAME from a CURRENTLY ACTIVE CAMERA
|
||||
<br>WHEN I request the VisionModule be DEACTIVATED
|
||||
<br>THEN the VisionModule will be stopped for the given CURRENTLY ACTIVE CAMERA
|
||||
<br>AND the CameraConfiguration DISABLED flag will be set to TRUE
|
||||
|
||||
## Reactivate
|
||||
|
||||
- Given: A UNIQUE NAME from a DESERIALIZED CAMERA CONFIGURATIONS
|
||||
<br>WHEN I request the VisionModule be ACTIVATED
|
||||
<br>AND the CameraConfiguration's DISABLED flag is TRUE
|
||||
<br>THEN a VisionModule will be created and started for the camera
|
||||
136
docs/source/docs/contributing/design-descriptions/e2e-latency.md
Normal file
@@ -0,0 +1,136 @@
|
||||
# Latency Characterization
|
||||
|
||||
|
||||
## A primer on time
|
||||
|
||||
Especially starting around 2022 with AprilTags making localization easier, providing a way to know when a camera image was captured at became more important for localization.
|
||||
Since the [creation of USBFrameProvider](https://github.com/PhotonVision/photonvision/commit/f92bf670ded52b59a00352a4a49c277f01bae305), we used the time [provided by CSCore](https://github.wpilib.org/allwpilib/docs/release/java/edu/wpi/first/cscore/CvSink.html#grabFrame(org.opencv.core.Mat)) to tell when a camera image was captured at, but just keeping track of "CSCore told us frame N was captured 104.21s after the Raspberry Pi turned on" isn't very helpful. We can decompose this into asking:
|
||||
|
||||
- At what time was a particular image captured at, in the coprocessor's timebase?
|
||||
- How do I convert a time in a coprocessor's timebase into the RoboRIO's timebase, so I can integrate the measurement with my other sensor measurements (like encoders)?
|
||||
|
||||
The first one seems easy - CSCore tells us the time, so just keep track of that? Should be easy. For the second, translating this time, as measured by the coprocessor's clock, into a timebase also used by user code on the RoboRIO, is actually a [fairly hard problem](time-sync.md) that involved reinventing [PTP](https://en.wikipedia.org/wiki/PTP).
|
||||
|
||||
And on latency vs timestamps - PhotonVision has exposed a magic "latency" number since forever, but latency (as in, the time from image capture to acting on data) can be useful for benchmarking code, but robots actually want to answer "what time was this image from, relative to "?
|
||||
|
||||
|
||||
## CSCore's Frame Time
|
||||
|
||||
WPILib's CSCore is a platform-agnostic wrapper around Windows, Linux, and MacOS camera APIs. On Linux, CSCore uses [Video4Linux](https://en.wikipedia.org/wiki/Video4Linux) to access USB Video Class (UVC) devices like webcams, as well as CSI cameras on some platforms. At a high level, CSCore's [Linux USB Camera driver](https://github.com/wpilibsuite/allwpilib/blob/17a03514bad6de195639634b3d57d5ac411d601e/cscore/src/main/native/linux/UsbCameraImpl.cpp) works by:
|
||||
|
||||
- Opening a camera with `open`
|
||||
- Creating and `mmap`ing a handful of buffers V4L will fill with frame data into program memory
|
||||
- Asking V4L to start streaming
|
||||
- While the camera is running:
|
||||
- Wait for new frames
|
||||
- Dequeue one buffer
|
||||
- Call `SourceImpl::PutFrame`, which will copy the image out and convert as needed
|
||||
- Return the buffer to V4L to fill again
|
||||
|
||||
Prior to https://github.com/wpilibsuite/allwpilib/pull/7609, CSCore used the [time it dequeued the buffer at](https://github.com/wpilibsuite/allwpilib/blob/17a03514bad6de195639634b3d57d5ac411d601e/cscore/src/main/native/linux/UsbCameraImpl.cpp#L559) as the image capture time. But this doesn't account for exposure time or latency introduced by the camera + USB stack + Linux itself.
|
||||
|
||||
V4L does expose (with some [very heavy caveats](https://github.com/torvalds/linux/blob/fc033cf25e612e840e545f8d5ad2edd6ba613ed5/drivers/media/usb/uvc/uvc_video.c#L600) for some troublesome cameras) its best guess at the time an image was captured at via [buffer flags](https://www.kernel.org/doc/html/v4.9/media/uapi/v4l/buffer.html#buffer-flags). In my testing, all my cameras were able to provide timestamps with both these flags set:
|
||||
- `V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC`: The buffer timestamp has been taken from the CLOCK_MONOTONIC clock [...] accessible via `clock_gettime()`.
|
||||
- `V4L2_BUF_FLAG_TSTAMP_SRC_SOE`: Start Of Exposure. The buffer timestamp has been taken when the exposure of the frame has begun.
|
||||
|
||||
I'm sure that we'll find a camera that doesn't play nice, because we can't have nice things :). But until then, using this timestamp gets us a free accuracy bump.
|
||||
|
||||
Other things to note: This gets us an estimate at when the camera *started* collecting photons. The camera's sensor will remain collecting light for up to the total integration time, plus readout time for rolling shutter cameras.
|
||||
|
||||
## Latency Testing
|
||||
|
||||
Here, I've got a RoboRIO with an LED, an Orange Pi 5, and a network switch on a test bench. The LED is assumed to turn on basically instantly once we apply current, and based on DMA testing, the total time to switch a digital output on is on the order of 10uS. The RoboRIO is running a TimeSync Server, and the Orange Pi is running a TimeSync Client.
|
||||
|
||||
### Test Setup
|
||||
|
||||
<details>
|
||||
<summary>Show RoboRIO Test Code</summary>
|
||||
|
||||
```java
|
||||
package frc.robot;
|
||||
|
||||
import org.photonvision.PhotonCamera;
|
||||
|
||||
import edu.wpi.first.wpilibj.DigitalOutput;
|
||||
import edu.wpi.first.wpilibj.TimedRobot;
|
||||
import edu.wpi.first.wpilibj.Timer;
|
||||
import edu.wpi.first.wpilibj.smartdashboard.SmartDashboard;
|
||||
|
||||
public class Robot extends TimedRobot {
|
||||
PhotonCamera camera;
|
||||
DigitalOutput light;
|
||||
|
||||
@Override
|
||||
public void robotInit() {
|
||||
camera = new PhotonCamera("Arducam_OV9782_USB_Camera");
|
||||
|
||||
light = new DigitalOutput(0);
|
||||
light.set(false);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void robotPeriodic() {
|
||||
super.robotPeriodic();
|
||||
|
||||
try {
|
||||
light.set(false);
|
||||
for (int i = 0; i < 50; i++) {
|
||||
Thread.sleep(20);
|
||||
camera.getAllUnreadResults();
|
||||
}
|
||||
|
||||
var t1 = Timer.getFPGATimestamp();
|
||||
light.set(true);
|
||||
var t2 = Timer.getFPGATimestamp();
|
||||
|
||||
|
||||
for (int i = 0; i < 100; i++) {
|
||||
for (var result : camera.getAllUnreadResults()) {
|
||||
if (result.hasTargets()) {
|
||||
var t3 = result.getTimestampSeconds();
|
||||
var t1p5 = (t1 + t2) / 2;
|
||||
var error = t3-t1p5;
|
||||
SmartDashboard.putNumber("blink_error_ms", error * 1000);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
Thread.sleep(20);
|
||||
}
|
||||
} catch (InterruptedException e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
</details>
|
||||
|
||||
I've decreased camera exposure as much as possible (so we know with reasonable confidence that the image was collected right at the start of the exposure time reported by V4L), but we only get back new images at 60fps. So we don't know when between frame N and N+1 the LED turned on - just that sometime between now and 1/60th of a second a go, the LED turned on.
|
||||
|
||||
The test coprocessor was an Orange Pi 5 running a PhotonVision 2025 (Ubuntu 24.04 based) image, with an ArduCam OV9782 at 1280x800, 60fps, MJPG running a reflective pipeline.
|
||||
|
||||
|
||||
### Test Results
|
||||
|
||||
The videos above show the difference between when the RoboRIO turned the LED on and when PhotonVision first seeing a camera frame with the LED on, what I've called error and plotted in yellow with units of seconds. This error decreases when I use the frame time reported by V4L from a mean delta of 26 ms to a mean delta of 11 ms (below the maximum temporal resolution of my camera).
|
||||
|
||||
Old CSCore:
|
||||
```{raw} html
|
||||
<video width="85%" controls>
|
||||
<source src="../../../_static/assets/latency-tests/ov9782_1280x720x60xMJPG_old.mp4" type="video/mp4">
|
||||
Your browser does not support the video tag.
|
||||
</video>
|
||||
```
|
||||
CSCore using V4L frame time:
|
||||
```{raw} html
|
||||
<video width="85%" controls>
|
||||
<source src="../../../_static/assets/latency-tests/ov9782_1280x720x60xMJPG_new.mp4" type="video/mp4">
|
||||
Your browser does not support the video tag.
|
||||
</video>
|
||||
```
|
||||
|
||||
With the camera capturing at 60fps, the time between successive frames is only ~16.7 ms, so I don't expect to be able to resolve anything smaller. Given sufficient time and with perfect latency compensation, and with more noise in the robot program to make sure we vary LED toggle times, I'd expect the error to converge to ~half the interval between frames - so being within this frame interval with CSCore updates is a very good sign.
|
||||
|
||||
### Future Work
|
||||
|
||||
This test also makes no effort to isolate error from time synchronization from error introduced by frame time measurement - we're just interested in overall error. Future work could investigate the latency contribution
|
||||
|
After Width: | Height: | Size: 215 KiB |
@@ -4,4 +4,6 @@
|
||||
:maxdepth: 1
|
||||
image-rotation
|
||||
time-sync
|
||||
camera-matching
|
||||
e2e-latency
|
||||
```
|
||||
|
||||
@@ -76,7 +76,7 @@ Communication between server and clients shall occur over the User Datagram Prot
|
||||
|
||||
## Message Format
|
||||
|
||||
The message format forgoes CRCs (as these are provided by the Ethernet physical layer) or packet delimination (as our packetsa are assumed be under the network MTU). **TSP Ping** and **TSP Pong** messages shall be encoded in a manor compatible with a WPILib packed struct with respect to byte alignment and endienness.
|
||||
The message format forgoes CRCs (as these are provided by the Ethernet physical layer) or packet delineation (as our packets are assumed be under the network MTU). **TSP Ping** and **TSP Pong** messages shall be encoded in a manor compatible with a WPILib packed struct with respect to byte alignment and endianness.
|
||||
|
||||
### TSP Ping
|
||||
|
||||
@@ -98,10 +98,10 @@ The message format forgoes CRCs (as these are provided by the Ethernet physical
|
||||
|
||||
## Optional Protocol Extensions
|
||||
|
||||
Clients may publish statistics to NetworkTables. If they do, they shall publish to a key that is globally unique per participant in the Time Synronization network. If a client implements this, it shall provide the following publishers:
|
||||
Clients may publish statistics to NetworkTables. If they do, they shall publish to a key that is globally unique per participant in the Time Synchronization network. If a client implements this, it shall provide the following publishers:
|
||||
|
||||
| Key | Type | Notes |
|
||||
| ------ | ------ | ---- | ----- |
|
||||
| ------ | ------ | ---- |
|
||||
| offset_us | Integer | The time offset that, when added to the client's local clock, provides server time |
|
||||
| ping_tx_count | Integer | The total number of TSP Ping packets transmitted |
|
||||
| ping_rx_count | Integer | The total number of TSP Ping packets received |
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Description
|
||||
|
||||
PhotonVision is a free, fast, and easy-to-use vision processing solution for the *FIRST*Robotics Competition. PhotonVision is designed to get vision working on your robot *quickly*, without the significant cost of other similar solutions.
|
||||
PhotonVision is a free, fast, and easy-to-use vision processing solution for the _FIRST_ Robotics Competition. PhotonVision is designed to get vision working on your robot _quickly_, but with lower cost than other solutions.
|
||||
Using PhotonVision, teams can go from setting up a camera and coprocessor to detecting and tracking AprilTags and other targets by simply tuning sliders. With an easy to use interface, comprehensive documentation, and a feature rich vendor dependency, no experience is necessary to use PhotonVision. No matter your resources, using PhotonVision is easy compared to its alternatives.
|
||||
|
||||
## Advantages
|
||||
@@ -19,19 +19,15 @@ The PhotonVision user interface is simple and modular, making things easier for
|
||||
|
||||
### PhotonLib Vendor Dependency
|
||||
|
||||
The PhotonLib vendor dependency allows you to easily get necessary target data (without having to work directly with NetworkTables) while also providing utility methods to get distance and position on the field. This helps your team focus less on getting data and more on using it to do cool things. This also has the benefit of having a structure that ensures all data is from the same timestamp, which is helpful for latency compensation.
|
||||
The PhotonLib vendor dependency allows you to easily get necessary target data (without having to work directly with NetworkTables) while also providing utility methods to get distance and position on the field. A serialization strategy is used to guarantees data coherency, which is helpful for latency compensation. This helps your team focus less on getting data and more on using it to do cool things.
|
||||
|
||||
### User Calibration
|
||||
|
||||
Using PhotonVision allows the user to calibrate for their specific camera, which will get you the best tracking results. This is extremely important as every camera (even if it is the same model) will have it's own quirks and user calibration allows for those to be accounted for.
|
||||
|
||||
### High FPS Processing
|
||||
### Low Latency, High FPS Processing
|
||||
|
||||
Compared to alternative solutions, PhotonVision boasts higher frames per second which allows for a smoother video stream and detection of targets to ensure you aren't losing out on any performance.
|
||||
|
||||
### Low Latency
|
||||
|
||||
PhotonVision provides low latency processing to make sure you get vision measurements as fast as possible, which makes complex vision tasks easier. We guarantee that all measurements are sent from the same timestamp, making life easier for your programmers.
|
||||
PhotonVision exposes specialized hardware on select coprocessors to maximize processing speed. This allows for lower-latency detection of targets to ensure you aren't losing out on any performance.
|
||||
|
||||
### Fully Open Source and Active Developer Community
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Combining Aiming and Getting in Range
|
||||
|
||||
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/master/photonlib-java-examples/aimandrange)/[C++](https://github.com/PhotonVision/photonvision/tree/master/photonlib-cpp-examples/aimandrange)).
|
||||
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/main/photonlib-java-examples/aimandrange)/[C++](https://github.com/PhotonVision/photonvision/tree/main/photonlib-cpp-examples/aimandrange)).
|
||||
|
||||
## Knowledge and Equipment Needed
|
||||
|
||||
|
||||
@@ -1,21 +1,21 @@
|
||||
# Aiming at a Target
|
||||
|
||||
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/master/photonlib-java-examples/aimattarget)).
|
||||
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/main/photonlib-java-examples/aimattarget)).
|
||||
|
||||
## Knowledge and Equipment Needed
|
||||
|
||||
- A Robot
|
||||
- A camera mounted rigidly to the robot's frame, cenetered and pointed forward.
|
||||
- A coprocessor running PhotonVision with an AprilTag or Aurco 2D Pipeline.
|
||||
- [A printout of Apriltag 7](https://firstfrc.blob.core.windows.net/frc2024/FieldAssets/Apriltag_Images_and_User_Guide.pdf), mounted on a rigid and flat surface.
|
||||
- A camera mounted rigidly to the robot's frame, centered and pointed forward.
|
||||
- A coprocessor running PhotonVision with an AprilTag or Aruco 2D Pipeline.
|
||||
- [A printout of AprilTag 7](https://firstfrc.blob.core.windows.net/frc2025/FieldAssets/Apriltag_Images_and_User_Guide.pdf), mounted on a rigid and flat surface.
|
||||
|
||||
## Code
|
||||
|
||||
Now that you have properly set up your vision system and have tuned a pipeline, you can now aim your robot at an AprilTag using the data from PhotonVision. The *yaw* of the target is the critical piece of data that will be needed first.
|
||||
Now that you have properly set up your vision system and have tuned a pipeline, you can now aim your robot at an AprilTag using the data from PhotonVision. The _yaw_ of the target is the critical piece of data that will be needed first.
|
||||
|
||||
Yaw is reported to the roboRIO over Network Tables. PhotonLib, our vender dependency, is the easiest way to access this data. The documentation for the Network Tables API can be found {ref}`here <docs/additional-resources/nt-api:Getting Target Information>` and the documentation for PhotonLib {ref}`here <docs/programming/photonlib/adding-vendordep:What is PhotonLib?>`.
|
||||
|
||||
In this example, while the operator holds a button down, the robot will turn towards the AprilTag using the P term of a PID loop. To learn more about how PID loops work, how WPILib implements them, and more, visit [Advanced Controls (PID)](https://docs.wpilib.org/en/stable/docs/software/advanced-control/introduction/index.html) and [PID Control in WPILib](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/controllers/pidcontroller.html#pid-control-in-wpilib).
|
||||
In this example, while the operator holds a button down, the robot will turn towards the AprilTag using the P term of a PID loop. To learn more about how PID loops work, how WPILib implements them, and more, visit [Advanced Controls (PID)](https://docs.wpilib.org/en/stable/docs/software/advanced-control/introduction/index.html) and [PID Control in WPILib](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/controllers/pidcontroller.html#pid-control-in-wpilib).
|
||||
|
||||
```{eval-rst}
|
||||
.. tab-set::
|
||||
|
||||
|
Before Width: | Height: | Size: 24 MiB |
@@ -1,6 +1,6 @@
|
||||
# Using WPILib Pose Estimation, Simulation, and PhotonVision Together
|
||||
|
||||
The following example comes from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/master/photonlib-java-examples/poseest)/[C++](https://github.com/PhotonVision/photonvision/tree/master/photonlib-cpp-examples/poseest)/[Python](https://github.com/PhotonVision/photonvision/tree/master/photonlib-python-examples/poseest)). Full code is available at that links.
|
||||
The following example comes from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/main/photonlib-java-examples/poseest)/[C++](https://github.com/PhotonVision/photonvision/tree/main/photonlib-cpp-examples/poseest)/[Python](https://github.com/PhotonVision/photonvision/tree/main/photonlib-python-examples/poseest)). Full code is available at that links.
|
||||
|
||||
## Knowledge and Equipment Needed
|
||||
|
||||
@@ -205,7 +205,9 @@ During simulation, we periodically update the simulated vision system.
|
||||
|
||||
The rest is done behind the scenes.
|
||||
|
||||
```{image} images/poseest_demo.gif
|
||||
:alt: Simulated swerve drive and vision system working together in teleoperated mode.
|
||||
:width: 1200
|
||||
```{raw} html
|
||||
<video width="85%" controls>
|
||||
<source src="../../_static/assets/poseest_demo.mp4" type="video/mp4">
|
||||
Your browser does not support the video tag.
|
||||
</video>
|
||||
```
|
||||
|
||||
|
Before Width: | Height: | Size: 1.0 MiB After Width: | Height: | Size: 1.0 MiB |
@@ -1,5 +1,7 @@
|
||||
# Pi Camera Configuration
|
||||
|
||||
This page covers specifics about the _Raspberry Pi_ CSI camera configuration.
|
||||
|
||||
## Background
|
||||
|
||||
The Raspberry Pi CSI Camera port is routed through and processed by the GPU. Since the GPU boots before the CPU, it must be configured properly for the attached camera. Additionally, this configuration cannot be changed without rebooting.
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
# Selecting Hardware
|
||||
|
||||
In order to use PhotonVision, you need a coprocessor and a camera. This page will help you select the right hardware for your team depending on your budget, needs, and experience.
|
||||
:::{note}
|
||||
See the {ref}`quick start guide<docs/quick-start/common-setups:Common Hardware Setups>`, for latest, specific recommendations on hardware to use for PhotonVision.
|
||||
:::
|
||||
|
||||
In order to use PhotonVision, you need a coprocessor and a camera. This page discusses the specifics of why that hardware is recommended.
|
||||
|
||||
## Choosing a Coprocessor
|
||||
|
||||
@@ -11,69 +15,82 @@ In order to use PhotonVision, you need a coprocessor and a camera. This page wil
|
||||
- CPU: ARM Cortex-A53 (the CPU on Raspberry Pi 3) or better
|
||||
- At least 8GB of storage
|
||||
- 2GB of RAM
|
||||
- PhotonVision isn't very RAM intensive, but you'll need at least 2GB to run the OS and PhotonVision.
|
||||
- PhotonVision isn't very RAM intensive, but you'll need at least 2GB to run the OS and PhotonVision.
|
||||
- The following IO:
|
||||
- At least 1 USB or MIPI-CSI port for the camera
|
||||
- Note that we only support using the Raspberry Pi's MIPI-CSI port, other MIPI-CSI ports from other coprocessors may not work.
|
||||
- Ethernet port for networking
|
||||
- At least 1 USB or MIPI-CSI port for the camera
|
||||
- Note that we only support using the Raspberry Pi's MIPI-CSI port, other MIPI-CSI ports from other coprocessors will probably not work.
|
||||
- Ethernet port for networking
|
||||
|
||||
Note these are bare minimums. Most high-performance vision processing will require higher specs.
|
||||
|
||||
### Coprocessor Recommendations
|
||||
|
||||
When selecting a coprocessor, it is important to consider various factors, particularly when it comes to AprilTag detection. Opting for a coprocessor with a more powerful CPU can generally result in higher FPS AprilTag detection, leading to more accurate pose estimation. However, it is important to note that there is a point of diminishing returns, where the benefits of a more powerful CPU may not outweigh the additional cost. Below is a list of supported hardware, along with some notes on each.
|
||||
Vision processing on one camera stream is usually a CPU-bound operation. Some operations are able to be done in parallel, but not all. USB bandwidth and network data transfer also cause a fixed overhead.
|
||||
|
||||
Faster CPU's generally result in lower latency, but eventually with diminishing returns. More cores allow for some improvement, especially if multiple camera streams are being processed.
|
||||
|
||||
PhotonVision is most commonly tested around Raspbian (Debian-based) operating systems.
|
||||
|
||||
Other coprocessors can be used but may require some extra work / command line usage in order to get it working properly.
|
||||
|
||||
### Power Supply
|
||||
|
||||
Coprocessors need a steady, regulated power supply. Under-volting the processor will result in CPU throttling, low performance, unexpected reboots, and sometimes electrical damage. Many coprocessors draw 5-10 amps of current.
|
||||
|
||||
Be sure to select a power supply which regulate's the robot's variable battery voltage into something steady that the robot can use.
|
||||
|
||||
### Storage Media
|
||||
|
||||
Most single-board computer coprocessors use micro SD cards as their storage media.
|
||||
|
||||
Three important considerations include total storage space, read/write speed, and robustness.
|
||||
|
||||
PhotonVision is not usually disk-bound, other than during coprocessor boot-up and initial startup. Some disk writing is done at runtime for logging, settings, and saving camera images on command.
|
||||
|
||||
Better storage space and read/write speed mostly matter if image capture is used frequently on the field.
|
||||
|
||||
Industrial-grade SD cards are recommended for their stability under shock, vibration, variable voltage, and power-off. Raspberry Pi and Orange Pi coprocessors are generally robust against robot power interruptions, teams have anecdotally reported that Sandisk industrial SD cards reduce the chances of an unexpected settings or log file corruption on shutdown.
|
||||
|
||||
- Orange Pi 5 (\$99)
|
||||
- This is the recommended coprocessor for most teams. It has a powerful CPU that can handle AprilTag detection at high FPS, and is relatively cheap compared to processors of a similar power.
|
||||
- Raspberry Pi 4/5 (\$55-\$80)
|
||||
- This is the recommended coprocessor for teams on a budget. It has a less powerful CPU than the Orange Pi 5, but is still capable of running PhotonVision at a reasonable FPS.
|
||||
- Mini PCs (such as Beelink N5095)
|
||||
- This coprocessor will likely have similar performance to the Orange Pi 5 but has a higher performance ceiling (when using more powerful CPUs). Do note that this would require extra effort to wire to the robot / get set up. More information can be found in the set up guide [here.](https://docs.google.com/document/d/1lOSzG8iNE43cK-PgJDDzbwtf6ASyf4vbW8lQuFswxzw/edit?usp=drivesdk)
|
||||
- Other coprocessors can be used but may require some extra work / command line usage in order to get it working properly.
|
||||
|
||||
## Choosing a Camera
|
||||
|
||||
PhotonVision works with Pi Cameras and most USB Cameras, the recommendations below are known to be working and have been tested. Other cameras such as webcams, virtual cameras, etc. are not officially supported and may not work. It is important to note that fisheye cameras should only be used as a driver camera and not for detecting targets.
|
||||
PhotonVision relies on [CSCore](https://github.com/wpilibsuite/allwpilib/tree/main/cscore) to detect and process cameras, so camera support is determined based off compatibility with CScore along with native support for the camera within your OS (ex. [V4L compatibility](https://en.wikipedia.org/wiki/Video4Linux)).
|
||||
|
||||
PhotonVision relies on [CSCore](https://github.com/wpilibsuite/allwpilib/tree/main/cscore) to detect and process cameras, so camera support is determined based off compatibility with CScore along with native support for the camera within your OS (ex. [V4L compatibility](https://en.wikipedia.org/wiki/Video4Linux) if using a Linux machine like a Raspberry Pi).
|
||||
PhotonVision attempts to support most USB Cameras. Exceptions include:
|
||||
|
||||
:::{note}
|
||||
Logitech Cameras and integrated laptop cameras will not work with PhotonVision due to oddities with their drivers. We recommend using a different camera.
|
||||
:::
|
||||
- All Logitech brand cameras
|
||||
- Logitech uses a non-standard driver which is not currently supported
|
||||
- Built-in webcams
|
||||
- Driver support is too varied. Some may happen to work, but most have been found to be non-functional
|
||||
- virtual cameras (OBS, Snapchat camera, etc.)
|
||||
- PhotonVision assumes the camera has real physical hardware to control - these do not expose the minimum number of controls.
|
||||
|
||||
:::{note}
|
||||
We do not currently support the usage of two of the same camera on the same coprocessor. You can only use two or more cameras if they are of different models or they are from Arducam, which has a [tool that allows for cameras to be renamed](https://docs.arducam.com/UVC-Camera/Serial-Number-Tool-Guide/).
|
||||
:::
|
||||
Use caution when using multiple identical cameras, as only the physical USB port they are plugged into can differentiate them. PhotonVision provides a "strict matching" setting which can reduce errors related to identical cameras. Arducam has a [tool that allows for identical cameras to be renamed](https://docs.arducam.com/UVC-Camera/Serial-Number-Tool-Guide/) by their physical location or purpose.
|
||||
|
||||
### Recommended Cameras
|
||||
|
||||
For colored shape detection, any non-fisheye camera supported by PhotonVision will work. We recommend the Pi Camera V1 or a high fps USB camera.
|
||||
### Cameras Attributes
|
||||
|
||||
For driver camera, we recommend a USB camera with a fisheye lens, so your driver can see more of the field.
|
||||
For colored shape detection, any non-fisheye camera supported by PhotonVision will work.
|
||||
|
||||
For AprilTag detection, we recommend you use a global shutter camera that has ~100 degree diagonal FOV. This will allow you to see more AprilTags in frame, and will allow for more accurate pose estimation. You also want a camera that supports high FPS, as this will allow you to update your pose estimator at a higher frequency.
|
||||
For driver camera, we recommend a USB camera with a fisheye lens, so your driver can see more of the field. Use the minimum acceptable resolution to help keep latency low.
|
||||
|
||||
- Recommendations For AprilTag Detection
|
||||
- Arducam USB OV9281
|
||||
- This is the recommended camera for AprilTag detection as it is a high FPS, global shutter camera USB camera that has a ~70 degree FOV.
|
||||
- Innomaker OV9281
|
||||
- Spinel AR0144
|
||||
- Pi Camera Module V1
|
||||
- The V1 is strongly preferred over the V2 due to the V2 having undesirable FOV choices
|
||||
For AprilTag detection, we recommend you use a camera that has ~100 degree diagonal FOV. This will allow you to see more AprilTags in frame, and will allow for more accurate pose estimation. You also want a camera that supports high FPS, as this will allow you to update your pose estimator at a higher frequency.
|
||||
|
||||
### AprilTags and Motion Blur
|
||||
For object detection, we recommend a USB camera. Some fisheye lenses may be ok, but very wide angle cameras may distort the gamepiece beyond recognition.
|
||||
|
||||
When detecting AprilTags, you want to reduce the "motion blur" as much as possible. Motion blur is the visual streaking/smearing on the camera stream as a result of movement of the camera or object of focus. You want to mitigate this as much as possible because your robot is constantly moving and you want to be able to read as many tags as you possibly can. The possible solutions to this include:
|
||||
Global shutter cameras are recommended in all cases, to reduce rolling-shutter image sheer while the robot is moving.
|
||||
|
||||
1. Cranking your exposure as low as it goes and increasing your gain/brightness. This will decrease the effects of motion blur and increase FPS.
|
||||
2. Using a global shutter (as opposed to rolling shutter) camera. This should eliminate most, if not all motion blur.
|
||||
3. Only rely on tags when not moving.
|
||||
|
||||
```{image} images/motionblur.gif
|
||||
```{image} images/rollingshutter.gif
|
||||
:align: center
|
||||
```
|
||||
|
||||
Cameras capable of capturing a good image with very short exposures will also help reduce image blur. Usually, high-FPS-capable cameras designed for computer vision are better at this than "consumer-grade" USB webcams.
|
||||
|
||||
### Using Multiple Cameras
|
||||
|
||||
Using multiple cameras on your robot will help you detect more AprilTags at once and improve your pose estimation as a result. In order to use multiple cameras, you will need to create multiple PhotonPoseEstimators and add all of their measurements to a single drivetrain pose estimator. Please note that the accuracy of your robot to camera transform is especially important when using multiple cameras as any error in the transform will cause your pose estimations to "fight" each other. For more information, see {ref}`the programming reference. <docs/programming/index:programming reference>`.
|
||||
Keeping the target(s) in view of the robot often requires more than one camera. PhotonVision has no hardcoded limit on the number of cameras supported. The limit is usually dependant on CPU (can all frames be processed fast enough?) and USB bandwidth (Can all cameras send their images without overwhelming the bus?).
|
||||
|
||||
Note that cameras are not synchronized together. Frames are captured and processed asynchronously. Robot Code must fuse estimates together. For more information, see {ref}`the programming reference. <docs/programming/index:programming reference>`.
|
||||
|
||||
## Performance Matrix
|
||||
|
||||
|
||||
|
Before Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 17 KiB |
|
Before Width: | Height: | Size: 152 KiB |
@@ -1,66 +0,0 @@
|
||||
# Networking
|
||||
|
||||
## Physical Networking
|
||||
|
||||
:::{note}
|
||||
When using PhotonVision off robot, you *MUST* plug the coprocessor into a physical router/radio. You can then connect your laptop/device used to view the webdashboard to the same network. Any other networking setup will not work and will not be supported in any capacity.
|
||||
:::
|
||||
|
||||
After imaging your coprocessor, run an ethernet cable from your coprocessor to a router/radio and power on your coprocessor by plugging it into the wall. Then connect whatever device you're using to view the webdashboard to the same network and navigate to photonvision.local:5800.
|
||||
|
||||
PhotonVision *STRONGLY* recommends the usage of a network switch on your robot. This is because the second radio port on the current FRC radios is known to be buggy and cause frequent connection issues that are detrimental during competition. An in-depth guide on how to install a network switch can be found [on FRC 900's website](https://team900.org/blog/ZebraSwitch/).
|
||||
|
||||
```{image} images/networking-diagram.png
|
||||
:alt: Correctly set static IP
|
||||
```
|
||||
|
||||
## Digital Networking
|
||||
|
||||
PhotonVision *STRONGLY* recommends the usage of Static IPs as it increases reliability on the field and when using PhotonVision in general. To properly set up your static IP, follow the steps below:
|
||||
|
||||
:::{warning}
|
||||
Only use a static IP when connected to the **robot radio**, and never when testing at home, unless you are well versed in networking or have the relevant "know how".
|
||||
:::
|
||||
|
||||
1. Ensure your robot is on and you are connected to the robot network.
|
||||
2. Navigate to `photonvision.local:5800` (this may be different if you are using a Gloworm / Limelight) in your browser.
|
||||
3. Open the settings tab on the left pane.
|
||||
4. Under the Networking section, set your team number.
|
||||
5. Change your IP to Static.
|
||||
6. Set your coprocessor's IP address to “10.TE.AM.11”. More information on IP format can be found [here](https://docs.wpilib.org/en/stable/docs/networking/networking-introduction/ip-configurations.html#on-the-field-static-configuration).
|
||||
7. Click the “Save” button.
|
||||
8. Set your roboRIO to the following static IP address: “10.TE.AM.2”. This can be done via the [roboRIO web dashboard](https://docs.wpilib.org/en/stable/docs/software/roborio-info/roborio-web-dashboard.html#roborio-web-dashboard).
|
||||
|
||||
Power-cycle your robot and then you will now be access the PhotonVision dashboard at `10.TE.AM.11:5800`.
|
||||
|
||||
```{image} images/static.png
|
||||
:alt: Correctly set static IP
|
||||
```
|
||||
|
||||
## Port Forwarding
|
||||
|
||||
If you would like to access your Ethernet-connected vision device from a computer when tethered to the USB port on the roboRIO, you can use [WPILib's](https://docs.wpilib.org/en/stable/docs/networking/networking-utilities/portforwarding.html) `PortForwarder`.
|
||||
|
||||
```{eval-rst}
|
||||
.. tab-set-code::
|
||||
|
||||
.. code-block:: Java
|
||||
|
||||
PortForwarder.add(5800, "photonvision.local", 5800);
|
||||
|
||||
.. code-block:: C++
|
||||
|
||||
wpi::PortForwarder::GetInstance().Add(5800, "photonvision.local", 5800);
|
||||
|
||||
.. code-block:: Python
|
||||
|
||||
# Coming Soon!
|
||||
```
|
||||
|
||||
:::{note}
|
||||
The address in the code above (`photonvision.local`) is the hostname of the coprocessor. This can be different depending on your hardware, and can be checked in the settings tab under "hostname".
|
||||
:::
|
||||
|
||||
## Camera Stream Ports
|
||||
|
||||
The camera streams start at they begin at 1181 with two ports for each camera (ex. 1181 and 1182 for camera one, 1183 and 1184 for camera two, etc.). The easiest way to identify the port of the camera that you want is by double clicking on the stream, which opens it in a separate page. The port will be listed below the stream.
|
||||
@@ -1,60 +0,0 @@
|
||||
# Gloworm Installation
|
||||
|
||||
While not currently in production, PhotonVision still supports Gloworm vision processing cameras.
|
||||
|
||||
## Downloading the Gloworm Image
|
||||
|
||||
Download the latest [Gloworm/Limelight release of PhotonVision](https://github.com/photonvision/photonvision/releases); the image will be suffixed with "image_limelight2.xz". You do not need to extract the downloaded archive.
|
||||
|
||||
## Flashing the Gloworm Image
|
||||
|
||||
Plug a USB C cable from your computer into the USB C port on Gloworm labeled with a download icon.
|
||||
|
||||
Use the 1.18.11 version of [Balena Etcher](https://github.com/balena-io/etcher/releases/tag/v1.18.11) to flash an image onto the coprocessor.
|
||||
|
||||
Run BalenaEtcher as an administrator. Select the downloaded `.zip` file.
|
||||
|
||||
Select the compute module. If it doesn't show up after 30s try using another USB port, initialization may take a while. If prompted, install the recommended missing drivers.
|
||||
|
||||
Hit flash. Wait for flashing to complete, then disconnect your USB C cable.
|
||||
|
||||
:::{warning}
|
||||
Using a version of Balena Etcher older than 1.18.11 may cause bootlooping (the system will repeatedly boot and restart) when imaging your Gloworm. Updating to 1.18.11 will fix this issue.
|
||||
:::
|
||||
|
||||
## Final Steps
|
||||
|
||||
Power your device per its documentation and connect it to a robot network.
|
||||
|
||||
You should be able to locate the camera at `http://photonvision.local:5800/` in your browser on your computer when connected to the robot.
|
||||
|
||||
## Troubleshooting/Setting a Static IP
|
||||
|
||||
A static IP address may be used as an alternative to the mDNS `photonvision.local` address.
|
||||
|
||||
Download and run [Angry IP Scanner](https://angryip.org/download/#windows) to find PhotonVision/your coprocessor on your network.
|
||||
|
||||
```{image} images/angryIP.png
|
||||
```
|
||||
|
||||
Once you find it, set the IP to a desired {ref}`static IP in PhotonVision. <docs/settings:Networking>`
|
||||
|
||||
## Updating PhotonVision
|
||||
|
||||
Download the latest stable .jar from [the releases page](https://github.com/PhotonVision/photonvision/releases), go to the settings tab, and upload the .jar using the Offline Update button.
|
||||
|
||||
:::{note}
|
||||
If you are updating PhotonVision on a Gloworm/Limelight, download the LinuxArm64 .jar file.
|
||||
:::
|
||||
|
||||
As an alternative option - Export your settings, reimage your coprocessor using the instructions above, and import your settings back in.
|
||||
|
||||
## Hardware Troubleshooting
|
||||
|
||||
To turn the LED lights off or on you need to modify the `ledMode` network tables entry or the `camera.setLED` of PhotonLib.
|
||||
|
||||
## Support Links
|
||||
|
||||
- [Website/Documentation](https://photonvision.github.io/gloworm-docs/docs/quickstart/#finding-gloworm) (Note: Gloworm is no longer in production)
|
||||
- [Image](https://github.com/gloworm-vision/pi-img-updator/releases)
|
||||
- [Discord](https://discord.com/invite/DncQRky)
|
||||
@@ -1,24 +0,0 @@
|
||||
# Limelight Installation
|
||||
|
||||
## Imaging
|
||||
|
||||
Limelight imaging is a very similar process to Gloworm, but with extra steps.
|
||||
|
||||
### Base Install Steps
|
||||
|
||||
Due to the similarities in hardware, follow the {ref}`Gloworm install instructions <docs/installation/sw_install/gloworm:Gloworm Installation>`.
|
||||
|
||||
## Hardware-Specific Steps
|
||||
|
||||
Download the hardwareConfig.json file for the version of your Limelight:
|
||||
|
||||
- {download}`Limelight Version 2 <files/Limelight2/hardwareConfig.json>`.
|
||||
- {download}`Limelight Version 2+ <files/Limelight2+/hardwareConfig.json>`.
|
||||
|
||||
:::{note}
|
||||
No hardware config is provided for the Limelight 3 as AprilTags do not require the LEDs (meaning nobody has reverse-engineered what I/O pins drive the LEDs) and the camera FOV is determined as part of calibration.
|
||||
:::
|
||||
|
||||
{ref}`Import the hardwareConfig.json file <docs/additional-resources/config:Importing and Exporting Settings>`. Again, this is **REQUIRED** or target measurements will be incorrect, and LEDs will not work.
|
||||
|
||||
After installation you should be able to [locate the camera](https://photonvision.github.io/gloworm-docs/docs/quickstart/#finding-gloworm) at: `http://photonvision.local:5800/` (not `gloworm.local`, as previously)
|
||||
@@ -1,39 +0,0 @@
|
||||
# Orange Pi Installation
|
||||
|
||||
## Downloading Linux Image
|
||||
|
||||
Starting in 2024, PhotonVision provides pre-configured system images for Orange Pi 5 devices. Download the latest release of the PhotonVision Orange Pi 5 image (.xz file suffixed with `orangepi5.xz`) from the [releases page](https://github.com/PhotonVision/photonvision/releases). You do not need to extract the downloaded archive file. This image is configured with a `pi` user with password `raspberry`.
|
||||
|
||||
For an Orange Pi 4, download the latest release of the Armbian Bullseye CLI image from [here](https://armbian.tnahosting.net/archive/orangepi4/archive/Armbian_23.02.2_Orangepi4_bullseye_current_5.15.93.img.xz).
|
||||
|
||||
## Flashing the Pi Image
|
||||
|
||||
An 8GB or larger SD card is recommended.
|
||||
|
||||
Use the 1.18.11 version of [Balena Etcher](https://github.com/balena-io/etcher/releases/tag/v1.18.11) to flash an image onto a Orange Pi. Select the downloaded image file, select your microSD card, and flash.
|
||||
|
||||
For more detailed instructions on using Etcher, please see the [Etcher website](https://www.balena.io/etcher/).
|
||||
|
||||
:::{warning}
|
||||
Using a version of Balena Etcher older than 1.18.11 may cause bootlooping (the system will repeatedly boot and restart) when imaging your Orange Pi. Updating to 1.18.11 will fix this issue.
|
||||
:::
|
||||
|
||||
Alternatively, you can use the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) to flash the image.
|
||||
|
||||
Select "Choose OS" and then "Use custom" to select the downloaded image file. Select your microSD card and flash.
|
||||
|
||||
:::{note}
|
||||
If you are working on Linux, "dd" can be used in the command line to flash an image.
|
||||
:::
|
||||
|
||||
If you're using an Orange Pi 5, that's it! Orange Pi 4 users will need to install PhotonVision (see below).
|
||||
|
||||
### Initial User Setup (Orange Pi 4 Only)
|
||||
|
||||
Insert the flashed microSD card into your Orange Pi and boot it up. The first boot may take a few minutes as the Pi expands the filesystem. Be sure not to unplug during this process.
|
||||
|
||||
Plug your Orange Pi into a display via HDMI and plug in a keyboard via USB once its powered up. For an Orange Pi 4, complete the initial set up which involves creating a root password and adding a user, as well as setting localization language. Additionally, choose “bash” when prompted.
|
||||
|
||||
## Installing PhotonVision (Orange Pi 4 Only)
|
||||
|
||||
From here, you can follow {ref}`this guide <docs/installation/sw_install/other-coprocessors:Installing Photonvision>`.
|
||||
@@ -1,50 +0,0 @@
|
||||
# Raspberry Pi Installation
|
||||
|
||||
A Pre-Built Raspberry Pi image is available for ease of installation.
|
||||
|
||||
## Downloading the Pi Image
|
||||
|
||||
Download the latest release of the PhotonVision Raspberry image (.xz file) from the [releases page](https://github.com/PhotonVision/photonvision/releases). You do not need to extract the downloaded ZIP file.
|
||||
|
||||
:::{note}
|
||||
Make sure you download the image that ends in '-RaspberryPi.xz'.
|
||||
:::
|
||||
|
||||
## Flashing the Pi Image
|
||||
|
||||
An 8GB or larger card is recommended.
|
||||
|
||||
Use the 1.18.11 version of [Balena Etcher](https://github.com/balena-io/etcher/releases/tag/v1.18.11) to flash an image onto a Raspberry Pi. Select the downloaded `.tar.xz` file, select your microSD card, and flash.
|
||||
|
||||
For more detailed instructions on using Etcher, please see the [Etcher website](https://www.balena.io/etcher/).
|
||||
|
||||
:::{warning}
|
||||
Using a version of Balena Etcher older than 1.18.11 may cause bootlooping (the system will repeatedly boot and restart) when imaging your Raspberry Pi. Updating to 1.18.11 will fix this issue.
|
||||
:::
|
||||
|
||||
Alternatively, you can use the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) to flash the image.
|
||||
|
||||
Select "Choose OS" and then "Use custom" to select the downloaded image file. Select your microSD card and flash.
|
||||
|
||||
If you are using a non-standard Pi Camera connected to the CSI port, {ref}`additional configuration may be required. <docs/hardware/picamconfig:Pi Camera Configuration>`
|
||||
|
||||
## Final Steps
|
||||
|
||||
Insert the flashed microSD card into your Raspberry Pi and boot it up. The first boot may take a few minutes as the Pi expands the filesystem. Be sure not to unplug during this process.
|
||||
|
||||
After the initial setup process, your Raspberry Pi should be configured for PhotonVision. You can verify this by making sure your Raspberry Pi and computer are connected to the same network and navigating to `http://photonvision.local:5800` in your browser on your computer.
|
||||
|
||||
## Troubleshooting/Setting a Static IP
|
||||
|
||||
A static IP address may be used as an alternative to the mDNS `photonvision.local` address.
|
||||
|
||||
Download and run [Angry IP Scanner](https://angryip.org/download/#windows) to find PhotonVision/your coprocessor on your network.
|
||||
|
||||
```{image} images/angryIP.png
|
||||
```
|
||||
|
||||
Once you find it, set the IP to a desired {ref}`static IP in PhotonVision. <docs/settings:Networking>`
|
||||
|
||||
## Updating PhotonVision
|
||||
|
||||
To upgrade a Raspberry Pi device with PhotonVision already installed, follow the {ref}`Raspberry Pi update instructions<docs/installation/updating:offline update>`.
|
||||
@@ -1,22 +0,0 @@
|
||||
# Romi Installation
|
||||
|
||||
The [Romi](https://docs.wpilib.org/en/latest/docs/romi-robot/index.html) is a small robot that can be controlled with the WPILib software. The main controller is a Raspberry Pi that must be imaged with [WPILibPi](https://docs.wpilib.org/en/latest/docs/romi-robot/imaging-romi.html) .
|
||||
|
||||
## Installation
|
||||
|
||||
The WPILibPi image includes FRCVision, which reserves USB cameras; to use PhotonVision, we need to edit the `/home/pi/runCamera` script to disable it. First we will need to make the file system writeable; the easiest way to do this is to go to `10.0.0.2` and choose "Writable" at the top.
|
||||
|
||||
SSH into the Raspberry Pi (using Windows command line, or a tool like [Putty](https://www.chiark.greenend.org.uk/~sgtatham/putty/) ) at the Romi's default address `10.0.0.2`. The default user is `pi`, and the password is `raspberry`.
|
||||
|
||||
Follow the process for installing PhotonVision on {ref}`"Other Debian-Based Co-Processor Installation" <docs/installation/sw_install/other-coprocessors:Other Debian-Based Co-Processor Installation>`. As it mentions this will require an internet connection so plugging into the ethernet jack on the Raspberry Pi will be the easiest solution. The pi must remain writable!
|
||||
|
||||
Next, from the SSH terminal, run `sudo nano /home/pi/runCamera` then arrow down to the start of the exec line and press "Enter" to add a new line. Then add `#` before the exec command to comment it out. Then, arrow up to the new line and type `sleep 10000`. Hit "Ctrl + O" and then "Enter" to save the file. Finally press "Ctrl + X" to exit nano. Now, reboot the Romi by typing `sudo reboot`.
|
||||
|
||||
```{image} images/nano.png
|
||||
```
|
||||
|
||||
After it reboots, you should be able to [locate the PhotonVision UI](https://photonvision.github.io/gloworm-docs/docs/quickstart/#finding-gloworm) at: `http://10.0.0.2:5800/`.
|
||||
|
||||
:::{warning}
|
||||
In order for settings, logs, etc. to be saved / take effect, ensure that PhotonVision is in writable mode.
|
||||
:::
|
||||
@@ -1,56 +0,0 @@
|
||||
# SnakeEyes Installation
|
||||
|
||||
A Pre-Built Raspberry Pi image with configuration for [the SnakeEyes Raspberry Pi Hat](https://www.playingwithfusion.com/productview.php?pdid=133&catid=1014) is available for ease of setup.
|
||||
|
||||
## Downloading the SnakeEyes Image
|
||||
|
||||
Download the latest release of the SnakeEyes-specific PhotonVision Pi image from the [releases page](https://github.com/PlayingWithFusion/SnakeEyesDocs/releases). You do not need to extract the downloaded ZIP file.
|
||||
|
||||
## Flashing the SnakeEyes Image
|
||||
|
||||
An 8GB or larger card is recommended.
|
||||
|
||||
Use the 1.18.11 version of [Balena Etcher](https://github.com/balena-io/etcher/releases/tag/v1.18.11) to flash an image onto a Raspberry Pi. Select the downloaded `.zip` file, select your microSD card, and flash.
|
||||
|
||||
For more detailed instructions on using Etcher, please see the [Etcher website](https://www.balena.io/etcher/).
|
||||
|
||||
:::{warning}
|
||||
Using a version of Balena Etcher older than 1.18.11 may cause bootlooping (the system will repeatedly boot and restart) when imaging your Raspberry Pi. Updating to 1.18.11 will fix this issue.
|
||||
:::
|
||||
|
||||
Alternatively, you can use the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) to flash the image.
|
||||
|
||||
Select "Choose OS" and then "Use custom" to select the downloaded image file. Select your microSD card and flash.
|
||||
|
||||
## Final Steps
|
||||
|
||||
Insert the flashed microSD card into your Raspberry Pi and boot it up. The first boot may take a few minutes as the Pi expands the filesystem. Be sure not to unplug during this process.
|
||||
|
||||
After the initial setup process, your Raspberry Pi should be configured for PhotonVision. You can verify this by making sure your Raspberry Pi and computer are connected to the same network and navigating to `http://photonvision.local:5800` in your browser on your computer.
|
||||
|
||||
## Troubleshooting/Setting a Static IP
|
||||
|
||||
A static IP address may be used as an alternative to the mDNS `photonvision.local` address.
|
||||
|
||||
Download and run [Angry IP Scanner](https://angryip.org/download/#windows) to find PhotonVision/your coprocessor on your network.
|
||||
|
||||
```{image} images/angryIP.png
|
||||
```
|
||||
|
||||
Once you find it, set the IP to a desired {ref}`static IP in PhotonVision. <docs/settings:Networking>`
|
||||
|
||||
## Updating PhotonVision
|
||||
|
||||
Download the latest xxxxx-LinuxArm64.jar from [our releases page](https://github.com/PhotonVision/photonvision/releases), go to the settings tab, and upload the .jar using the Offline Update button.
|
||||
|
||||
As an alternative option - Export your settings, reimage your coprocessor using the instructions above, and import your settings back in.
|
||||
|
||||
## Hardware Troubleshooting
|
||||
|
||||
To turn the LED lights off or on you need to modify the `ledMode` network tables entry or the `camera.setLED` of PhotonLib.
|
||||
|
||||
## Support Links
|
||||
|
||||
- [Website](https://www.playingwithfusion.com/productview.php?pdid=133)
|
||||
- [Image](https://github.com/PlayingWithFusion/SnakeEyesDocs/releases/latest)
|
||||
- [Documentation](https://github.com/PlayingWithFusion/SnakeEyesDocs/blob/master/PhotonVision/readme.md)
|
||||
@@ -1,54 +0,0 @@
|
||||
# Updating PhotonVision
|
||||
|
||||
PhotonVision provides many different files on a single release page. Each release contains JAR files for performing "offline updates" of a device with PhotonVision already installed, as well as full image files to "flash" to supported coprocessors.
|
||||
|
||||
```{image} images/release-page.png
|
||||
:alt: Example GitHub release page
|
||||
```
|
||||
|
||||
In the example release above, we see:
|
||||
|
||||
- Image files for flashing directly to supported coprocessors.
|
||||
|
||||
- Raspberry Pi 3/4/5/CM4: follow our {ref}`Raspberry Pi flashing instructions<docs/installation/sw_install/raspberry-pi:raspberry pi installation>`.
|
||||
- For LimeLight devices: follow our {ref}`LimeLight flashing instructions<docs/installation/sw_install/limelight:imaging>`.
|
||||
- For Orange Pi 5 devices: follow our {ref}`Orange Pi flashing instructions<docs/installation/sw_install/orange-pi:orange pi installation>`.
|
||||
|
||||
- JAR files for the suite of supported operating systems for use with Offline Update. In general:
|
||||
|
||||
- Raspberry Pi, Limelight, and Orange Pi: use images suffixed with -linuxarm64.jar. For example: {code}`photonvision-v2024.1.1-linuxarm64.jar`
|
||||
- Beelink and other Intel/AMD-based Mini-PCs: use images suffixed with -linuxx64.jar. For example: {code}`photonvision-v2024.1.1-linuxx64.jar`
|
||||
|
||||
## Offline Update
|
||||
|
||||
Unless noted in the release page, an offline update allows you to quickly upgrade the version of PhotonVision running on a coprocessor with PhotonVision already installed on it.
|
||||
|
||||
Unless otherwise noted on the release page, config files should be backward compatible with previous version of PhotonVision, and this offline update process should preserve any pipelines and calibrations previously performed. For paranoia, we suggest exporting settings from the Settings tab prior to performing an offline update.
|
||||
|
||||
:::{note}
|
||||
Carefully review release notes to ensure that reflashing the device (for supported devices) or other installation steps are not required, as dependencies needed for PhotonVision may change between releases
|
||||
:::
|
||||
|
||||
## Installing Pre-Release Versions
|
||||
|
||||
Pre-release/development version of PhotonVision can be tested by installing/downloading artifacts from Github Actions (see below), which are built automatically on commits to open pull requests and to PhotonVision's `master` branch, or by {ref}`compiling PhotonVision locally <docs/contributing/building-photon:Build Instructions>`.
|
||||
|
||||
:::{warning}
|
||||
If testing a pre-release version of PhotonVision with a robot, PhotonLib must be updated to match the version downloaded! If not, packet schema definitions may not match and unexpected things will occur. To update PhotonLib, refer to {ref}`installing specific version of PhotonLib<docs/programming/photonlib/adding-vendordep:Install Specific Version - Java/C++>`.
|
||||
:::
|
||||
|
||||
GitHub Actions builds pre-release version of PhotonVision automatically on PRs and on each commit merged to master. To test a particular commit to master, navigate to the [PhotonVision commit list](https://github.com/PhotonVision/photonvision/commits/master/) and click on the check mark (below). Scroll to "Build / Build fat JAR - PLATFORM", click details, and then summary. From here, JAR and image files can be downloaded to be flashed or uploaded using "Offline Update".
|
||||
|
||||
```{image} images/gh_actions_1.png
|
||||
:alt: Github Actions Badge
|
||||
```
|
||||
|
||||
```{image} images/gh_actions_2.png
|
||||
:alt: Github Actions artifact list
|
||||
```
|
||||
|
||||
Built JAR files (but not image files) can also be downloaded from PRs before they are merged. Navigate to the PR in GitHub, and select Checks at the top. Click on "Build" to display the same artifact list as above.
|
||||
|
||||
```{image} images/gh_actions_3.png
|
||||
:alt: Github Actions artifacts from PR
|
||||
```
|
||||
@@ -1,42 +0,0 @@
|
||||
# Wiring
|
||||
|
||||
## Off-Robot Wiring
|
||||
|
||||
Plugging your coprocessor into the wall via a power brick will suffice for off robot wiring.
|
||||
|
||||
:::{note}
|
||||
Please make sure your chosen power supply can provide enough power for your coprocessor. Undervolting (where enough power isn't being supplied) can cause many issues.
|
||||
:::
|
||||
|
||||
## On-Robot Wiring
|
||||
|
||||
:::{note}
|
||||
We recommend users use the [SnakeEyes Pi Hat](https://www.playingwithfusion.com/productview.php?pdid=133) as it provides passive power over ethernet (POE) and other useful features to simplify wiring and make your life easier.
|
||||
:::
|
||||
|
||||
### Recommended: Coprocessor with Passive POE (Gloworm, Pi with SnakeEyes, Limelight)
|
||||
|
||||
1. Plug the [passive POE injector](https://www.revrobotics.com/rev-11-1210/) into the coprocessor and wire it to PDP/PDH (NOT the VRM).
|
||||
2. Add a breaker to relevant slot in your PDP/PDH
|
||||
3. Run an ethernet cable from the passive POE injector to your network switch / radio (we *STRONGLY* recommend the usage of a network switch, see the [networking](networking.md) section for more info.)
|
||||
|
||||
### Coprocessor without Passive POE
|
||||
|
||||
1a. Option 1: Get a micro USB (may be USB-C if using a newer Pi) pigtail cable and connect the wire ends to a regulator like [this](https://www.pololu.com/product/4082). Then, wire the regulator into your PDP/PDH and the Micro USB / USB C into your coprocessor.
|
||||
|
||||
1b. Option 2: Use a USB power bank to power your coprocessor. Refer to this year's robot rulebook on legal implementations of this.
|
||||
|
||||
2. Run an ethernet cable from your Pi to your network switch / radio (we *STRONGLY* recommend the usage of a network switch, see the [networking](networking.md) section for more info.)
|
||||
|
||||
This diagram shows how to use the recommended regulator to power a coprocessor.
|
||||
|
||||
```{image} images/pololu-diagram.png
|
||||
:alt: A flowchart-type diagram showing how to connect wires from the PDP or PDH to
|
||||
: the recommended voltage regulator and then a Coprocessor.
|
||||
```
|
||||
|
||||
:::{note}
|
||||
The regulator comes with optional screw terminals that may be used to connect the PDP/PDH and Coprocessor power wires if you do not wish to solder them.
|
||||
:::
|
||||
|
||||
Once you have wired your coprocessor, you are now ready to install PhotonVision.
|
||||
@@ -2,21 +2,18 @@
|
||||
|
||||
## How does it work?
|
||||
|
||||
PhotonVision supports object detection using neural network accelerator hardware built into Orange Pi 5/5+ coprocessors. The Neural Processing Unit, or NPU, is [used by PhotonVision](https://github.com/PhotonVision/rknn_jni/tree/main) to massively accelerate certain math operations like those needed for running ML-based object detection.
|
||||
PhotonVision supports object detection using neural network accelerator hardware built into Orange Pi 5/5+ coprocessors. Please note that the Orange Pi 5/5+ are the only coprocessors that are currently supported. The Neural Processing Unit, or NPU, is [used by PhotonVision](https://github.com/PhotonVision/rknn_jni/tree/main) to massively accelerate certain math operations like those needed for running ML-based object detection.
|
||||
|
||||
For the 2024 season, PhotonVision ships with a **pre-trained NOTE detector** (shown above), as well as a mechanism for swapping in custom models. Future development will focus on enabling lower friction management of multiple custom models.
|
||||
|
||||
```{image} images/notes-ui.png
|
||||
```
|
||||
For the 2025 season, PhotonVision ships with a pretrained ALGAE model. A model to detect coral is not currently stable, and interested teams should ask in the Photonvision discord.
|
||||
|
||||
## Tracking Objects
|
||||
|
||||
Before you get started with object detection, ensure that you have followed the previous sections on installation, wiring, and networking. Next, open the Web UI, go to the top right card, and switch to the “Object Detection” type. You should see a screen similar to the image above.
|
||||
|
||||
PhotonVision currently ships with a NOTE detector based on a [YOLOv5 model](https://docs.ultralytics.com/yolov5/). This model is trained to detect one or more object "classes" (such as cars, stoplights, or in our case, NOTES) in an input image. For each detected object, the model outputs a bounding box around where in the image the object is located, what class the object belongs to, and a unitless confidence between 0 and 1.
|
||||
Models are trained to detect one or more object "classes" (such as cars, stoplights) in an input image. For each detected object, the model outputs a bounding box around where in the image the object is located, what class the object belongs to, and a unitless confidence between 0 and 1.
|
||||
|
||||
:::{note}
|
||||
This model output means that while its fairly easy to say that "this rectangle probably contains a NOTE", we don't have any information about the NOTE's orientation or location. Further math in user code would be required to make estimates about where an object is physically located relative to the camera.
|
||||
This model output means that while its fairly easy to say that "this rectangle probably contains an object", we don't have any information about the object's orientation or location. Further math in user code would be required to make estimates about where an object is physically located relative to the camera.
|
||||
:::
|
||||
|
||||
## Tuning and Filtering
|
||||
@@ -32,16 +29,27 @@ Compared to other pipelines, object detection exposes very few tuning handles. T
|
||||
|
||||
The same area, aspect ratio, and target orientation/sort parameters from {ref}`reflective pipelines <docs/reflectiveAndShape/contour-filtering:Reflective>` are also exposed in the object detection card.
|
||||
|
||||
## Letterboxing
|
||||
|
||||
Photonvision will letterbox your camera frame to 640x640. This means that if you select a resolution that is larger than 640 it will be scaled down to fit inside a 640x640 frame with black bars if needed. Smaller frames will be scaled up with black bars if needed.
|
||||
|
||||
## Training Custom Models
|
||||
|
||||
Coming soon!
|
||||
:::{warning}
|
||||
Power users only. This requires some setup, such as obtaining your own dataset and installing various tools. It's additionally advised to have a general knowledge of ML before attempting to train your own model. Additionally, this is not officially supported by Photonvision, and any problems that may arise are not attributable to Photonvision.
|
||||
:::
|
||||
|
||||
Before beginning, it is necessary to install the [rknn-toolkit2](https://github.com/airockchip/rknn-toolkit2). Then, install the relevant [Ultralytics repository](https://github.com/airockchip?tab=repositories&q=yolo&type=&language=&sort=) from this list. After training your model, export it to `rknn`. This will give you an `onnx` file, formatted for conversion. Copy this file to the relevant folder in [rknn_model_zoo](https://github.com/airockchip/rknn_model_zoo), and use the conversion script located there to convert it. If necessary, modify the script to provide the path to your training database for quantization.
|
||||
|
||||
## Uploading Custom Models
|
||||
|
||||
:::{warning}
|
||||
PhotonVision currently ONLY supports YOLOv5 models trained and converted to `.rknn` format for RK3588 CPUs! Other models require different post-processing code and will NOT work. The model conversion process is also highly particular. Proceed with care.
|
||||
PhotonVision currently ONLY supports 640x640 Ultralytics YOLOv5, YOLOv8, and YOLO11 models trained and converted to `.rknn` format for RK3588 CPUs! Other models require different post-processing code and will NOT work. The model conversion process is also highly particular. Proceed with care.
|
||||
:::
|
||||
|
||||
Our [pre-trained NOTE model](https://github.com/PhotonVision/photonvision/blob/master/photon-server/src/main/resources/models/note-640-640-yolov5s.rknn) is automatically extracted from the JAR when PhotonVision starts, only if a file named “note-640-640-yolov5s.rknn” and "labels.txt" does not exist in the folder `photonvision_config/models/`. This technically allows power users to replace the model and label files with new ones without rebuilding Photon from source and uploading a new JAR.
|
||||
|
||||
Use a program like WinSCP or FileZilla to access your coprocessor's filesystem, and copy the new `.rknn` model file into /home/pi. Next, SSH into the coprocessor and `sudo mv /path/to/new/model.rknn /opt/photonvision/photonvision_config/models/note-640-640-yolov5s.rknn`. Repeat this process with the labels file, which should contain one line per label the model outputs with no training newline. Next, restart PhotonVision via the web UI.
|
||||
In the settings, under `Device Control`, there's an option to upload a new object detection model. Naming convention
|
||||
should be `name-verticalResolution-horizontalResolution-modelType`. The
|
||||
`name` should only include alphanumeric characters, periods, and underscores. Additionally, the labels
|
||||
file ought to have the same name as the RKNN file, with `-labels` appended to the end. For
|
||||
example, if the RKNN file is named `Algae_1.03.2025-640-640-yolov5s.rknn`, the labels file should be
|
||||
named `Algae_1.03.2025-640-640-yolov5s-labels.txt`.
|
||||
|
||||
@@ -60,7 +60,7 @@ Use the `getLatestResult()`/`GetLatestResult()` (Java and C++ respectively) to o
|
||||
```
|
||||
|
||||
:::{note}
|
||||
Unlike other vision software solutions, using the latest result guarantees that all information is from the same timestamp. This is achievable because the PhotonVision backend sends a byte-packed string of data which is then deserialized by PhotonLib to get target data. For more information, check out the [PhotonLib source code](https://github.com/PhotonVision/photonvision/tree/master/photon-lib).
|
||||
Unlike other vision software solutions, using the latest result guarantees that all information is from the same timestamp. This is achievable because the PhotonVision backend sends a byte-packed string of data which is then deserialized by PhotonLib to get target data. For more information, check out the [PhotonLib source code](https://github.com/PhotonVision/photonvision/tree/main/photon-lib).
|
||||
:::
|
||||
|
||||
## Checking for Existence of Targets
|
||||
|
||||
@@ -17,12 +17,12 @@ The API documentation can be found in here: [Java](https://github.wpilib.org/all
|
||||
.. code-block:: Java
|
||||
|
||||
// The field from AprilTagFields will be different depending on the game.
|
||||
AprilTagFieldLayout aprilTagFieldLayout = AprilTagFields.k2024Crescendo.loadAprilTagLayoutField();
|
||||
AprilTagFieldLayout aprilTagFieldLayout = AprilTagFieldLayout.loadField(AprilTagFields.kDefaultField);
|
||||
|
||||
.. code-block:: C++
|
||||
|
||||
// The parameter for LoadAPrilTagLayoutField will be different depending on the game.
|
||||
frc::AprilTagFieldLayout aprilTagFieldLayout = frc::LoadAprilTagLayoutField(frc::AprilTagField::k2024Crescendo);
|
||||
frc::AprilTagFieldLayout aprilTagFieldLayout = frc::LoadAprilTagLayoutField(frc::AprilTagField::kDefaultField);
|
||||
|
||||
.. code-block:: Python
|
||||
|
||||
@@ -47,6 +47,19 @@ The PhotonPoseEstimator has a constructor that takes an `AprilTagFieldLayout` (s
|
||||
- Choose the Pose which is closest to the last pose calculated.
|
||||
- AVERAGE_BEST_TARGETS
|
||||
- Choose the Pose which is the average of all the poses from each tag.
|
||||
- MULTI_TAG_PNP_ON_RIO
|
||||
- A slower, older version of MULTI_TAG_PNP_ON_COPROCESSOR, not recommended for use.
|
||||
- PNP_DISTANCE_TRIG_SOLVE
|
||||
- Use distance data from best visible tag to compute a Pose. This runs on the RoboRIO in order
|
||||
to access the robot's yaw heading, and MUST have addHeadingData called every frame so heading
|
||||
data is up-to-date. Based on a reference implementation by [FRC Team 6328 Mechanical Advantage](https://www.chiefdelphi.com/t/frc-6328-mechanical-advantage-2025-build-thread/477314/98).
|
||||
- CONSTRAINED_SOLVEPNP
|
||||
- Solve a constrained version of the Perspective-n-Point problem with the robot's drivebase
|
||||
flat on the floor. This computation takes place on the RoboRIO, and should not take more than 2ms.
|
||||
This also requires addHeadingData to be called every frame so heading data is up to date.
|
||||
If Multi-Tag PNP is enabled on the coprocessor, it will be used to provide an initial seed to
|
||||
the optimization algorithm -- otherwise, the multi-tag fallback strategy will be used as the
|
||||
seed.
|
||||
|
||||
```{eval-rst}
|
||||
.. tab-set-code::
|
||||
@@ -91,7 +104,7 @@ The PhotonPoseEstimator has a constructor that takes an `AprilTagFieldLayout` (s
|
||||
self.cam = PhotonCamera("YOUR CAMERA NAME")
|
||||
|
||||
self.camPoseEst = PhotonPoseEstimator(
|
||||
loadAprilTagLayoutField(AprilTagField.k2024Crescendo),
|
||||
loadAprilTagLayoutField(AprilTagField.kDefaultField),
|
||||
PoseStrategy.CLOSEST_TO_REFERENCE_POSE,
|
||||
self.cam,
|
||||
kRobotToCam
|
||||
|
||||
@@ -106,7 +106,7 @@ You can get a [translation](https://docs.wpilib.org/en/latest/docs/software/adva
|
||||
.. code-block:: C++
|
||||
|
||||
// Calculate a translation from the camera to the target.
|
||||
frc::Translation2d translation = photonlib::PhotonUtils::EstimateCameraToTargetTranslationn(
|
||||
frc::Translation2d translation = photonlib::PhotonUtils::EstimateCameraToTargetTranslation(
|
||||
distance, frc::Rotation2d(units::degree_t(-target.GetYaw())));
|
||||
|
||||
.. code-block:: Python
|
||||
|
||||
22
docs/source/docs/quick-start/arducam-cameras.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Arducam Cameras
|
||||
|
||||
Arducam cameras are supported for setups with multiple devices. This is possible because Arducam provides software that allows you to assign truly different device names to each camera. This feature is particularly useful in complex setups where multiple cameras are used simultaneously.
|
||||
|
||||
## Setting Up Arducam Cameras
|
||||
|
||||
1. **Download Arducam Software**: [Download and install the Arducam software from their official website.](https://docs.arducam.com/UVC-Camera/Serial-Number-Tool-Guide/)
|
||||
|
||||
2. **Assign Device Names**: Use the Arducam software and Arducam [documentation](https://docs.arducam.com/UVC-Camera/Serial-Number-Tool-Guide/) to give each camera a unique device name. This will help in distinguishing between multiple cameras in your setup.
|
||||
|
||||
## Steps to Configure in PhotonVision
|
||||
|
||||
1. **Open PhotonVision Settings**: Navigate to the cameras page in PhotonVision.
|
||||
|
||||
2. **Select Camera Model**: Select the proper camera. Use the Arducam model selector to specify the model of each Arducam camera connected to your system.
|
||||
|
||||
3. **Save Settings**: Ensure that you save the settings after selecting the appropriate camera model for each device.
|
||||
|
||||
```{image} images/setArducamModel.png
|
||||
:alt: The camera model can be selected from the Arducam model selector in the cameras tab
|
||||
:align: center
|
||||
```
|
||||
33
docs/source/docs/quick-start/camera-calibration.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Camera Calibration
|
||||
|
||||
:::{important}
|
||||
In order to detect AprilTags and use 3D mode, your camera must be calibrated at the desired resolution! Inaccurate calibration will lead to poor performance.
|
||||
:::
|
||||
|
||||
If you’re not using cameras in 3D mode, calibration is optional, but it can still offer benefits. Calibrating cameras helps refine the pitch and yaw values, leading to more accurate positional data in every mode. {ref}`For a more in-depth view<docs/calibration/calibration:Calibrating Your Camera>`.
|
||||
|
||||
## Print the Calibration Target
|
||||
|
||||
- Downloaded from our [demo site](http://photonvision.global/#/cameras), or directly from your coprocessors cameras tab.
|
||||
- Use the Charuco calibration board:
|
||||
- Board Type: Charuco
|
||||
- Tag Family: 4x4
|
||||
- Pattern Spacing: 1.00in
|
||||
- Marker Size: 0.75in
|
||||
- Board Height : 8
|
||||
- Board Width : 8
|
||||
|
||||
## Prepare the Calibration Target
|
||||
|
||||
- Measure Accurately: Use calipers to measure the actual size of the squares and markers. Accurate measurements are crucial for effective calibration.
|
||||
- Ensure Flatness: The calibration board must be perfectly flat, without any wrinkles or bends, to avoid introducing errors into the calibration process.
|
||||
|
||||
## Calibrate your Camera
|
||||
|
||||
- Take lots of photos: It's recommended to capture more than 50 images to properly calibrate your camera for accuracy. 12 is the bare minimum and may not provide good results.
|
||||
- Other Tips
|
||||
- Move the board not the camera.
|
||||
- Take photos of lots of angles: The more angles the more better (up to 45 deg).
|
||||
- A couple of up close images is good.
|
||||
- Cover the entire cameras fov.
|
||||
- Avoid images with the board facing straight towards the camera.
|
||||
46
docs/source/docs/quick-start/common-setups.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Common Hardware Setups
|
||||
|
||||
PhotonVision requires dedicated hardware, above and beyond a roboRIO. This page lists hardware that is frequently used with PhotonVision.
|
||||
|
||||
## Coprocessors
|
||||
|
||||
- Orange Pi 5 4GB
|
||||
- Supports up to 2 object detection streams, along with 2 AprilTag streams at 1280x800 (30fps).
|
||||
- Raspberry Pi 5 2GB
|
||||
- Supports up to 2 AprilTag streams at 1280x800 (30fps).
|
||||
|
||||
:::{note}
|
||||
The Orange Pi 5 is the only currently supported device for object detection.
|
||||
:::
|
||||
|
||||
## SD Cards
|
||||
|
||||
- 8GB or larger micro SD card
|
||||
|
||||
:::{important}
|
||||
Industrial grade SD cards from major manufacturers are recommended for robotics applications. For example: Sandisk SDSDQAF3-016G-I .
|
||||
:::
|
||||
|
||||
## Cameras
|
||||
|
||||
Innomaker and Arducam are common manufacturers of hardware designed specifically for vision processing.
|
||||
|
||||
- AprilTag Detection
|
||||
- OV9281
|
||||
|
||||
- Object Detection
|
||||
- OV9782
|
||||
|
||||
- Driver Camera
|
||||
- OV9281
|
||||
- OV9782
|
||||
- Pi Camera Module V1 {ref}`(More setup info)<docs/hardware/picamconfig:Pi Camera Configuration>`
|
||||
|
||||
Feel free to get started with any color webcam you have sitting around.
|
||||
|
||||
## Power
|
||||
|
||||
- Pololu S13V30F5 Regulator
|
||||
- Redux Robotics Zinc-V Regulator
|
||||
|
||||
See {ref}`(Selecting Hardware)<docs/hardware/selecting-hardware:Selecting Hardware>` for info on why these are recommended.
|
||||
BIN
docs/source/docs/quick-start/images/OrangePiPololu.png
Normal file
|
After Width: | Height: | Size: 2.3 MiB |
BIN
docs/source/docs/quick-start/images/OrangePiPololuPigtail.png
Normal file
|
After Width: | Height: | Size: 2.3 MiB |
BIN
docs/source/docs/quick-start/images/OrangePiZinc.png
Normal file
|
After Width: | Height: | Size: 2.1 MiB |
BIN
docs/source/docs/quick-start/images/OrangePiZincUSBC.png
Normal file
|
After Width: | Height: | Size: 2.1 MiB |
BIN
docs/source/docs/quick-start/images/RPiPololu.png
Normal file
|
After Width: | Height: | Size: 2.3 MiB |
BIN
docs/source/docs/quick-start/images/RPiPololuPigtail.png
Normal file
|
After Width: | Height: | Size: 2.3 MiB |
BIN
docs/source/docs/quick-start/images/RPiZinc.png
Normal file
|
After Width: | Height: | Size: 2.0 MiB |
BIN
docs/source/docs/quick-start/images/RPiZincUSBC.png
Normal file
|
After Width: | Height: | Size: 2.0 MiB |
BIN
docs/source/docs/quick-start/images/editCameraName.png
Normal file
|
After Width: | Height: | Size: 138 KiB |
BIN
docs/source/docs/quick-start/images/editHostname.png
Normal file
|
After Width: | Height: | Size: 132 KiB |
BIN
docs/source/docs/quick-start/images/motionblur.png
Normal file
|
After Width: | Height: | Size: 394 KiB |
|
After Width: | Height: | Size: 826 KiB |
BIN
docs/source/docs/quick-start/images/networking-diagram.png
Normal file
|
After Width: | Height: | Size: 55 KiB |
BIN
docs/source/docs/quick-start/images/setArducamModel.png
Normal file
|
After Width: | Height: | Size: 142 KiB |