Team2890/PhotonVision
4
0
Fork 0
mirror of https://github.com/PhotonVision/photonvision synced 2026-06-22 01:11:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

Examples Clean-Up (#1408)

Browse Source
This commit is contained in:
Chris Gerth
2024-09-14 23:10:02 -05:00
committed by GitHub
parent 596c87519c
commit 9e6a066561
269 changed files with 9346 additions and 3734 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/docs/additional-resources/best-practices.md
View File

@@ -15,9 +15,9 @@
- Make sure you take advantage of the field calibration time given at the start of the event:
- Bring your robot to the field at the allotted time.
- Turn on your robot and pull up the dashboard on your driver station.
- Point your robot at the target(s) and ensure you get a consistent tracking (you hold one target consistently, the ceiling lights aren't detected, etc.).
- If you have problems with your pipeline, go to the pipeline tuning section and retune the pipeline using the guide there. You want to make your exposure as low as possible with a tight hue value to ensure no extra targets are detected.
- Move the robot close, far, angled, and around the field to ensure no extra targets are found anywhere when looking for a target.
- Point your robot at the AprilTags(s) and ensure you get a consistent tracking (you hold one AprilTag consistently, the ceiling lights aren't detected, etc.).
- If you have problems with your pipeline, go to the pipeline tuning section and retune the pipeline using the guide there.
- Move the robot close, far, angled, and around the field to ensure no extra AprilTags are found.
- Go to a practice match to ensure everything is working correctly.
- After field calibration, use the "Export Settings" button in the "Settings" page to create a backup.
- Do this for each coprocessor on your robot that runs PhotonVision, and name your exports with meaningful names.
@@ -26,4 +26,4 @@
- This effectively works as a snapshot of your PhotonVision data that can be restored at any point.
- Before every match, check the ethernet connection going into your coprocessor and that it is seated fully.
- Ensure that exposure is as low as possible and that you don't have the dashboard up when you don't need it to reduce bandwidth.
- Stream at as low of a resolution as possible while still detecting targets to stay within bandwidth limits.
- Stream at as low of a resolution as possible while still detecting AprilTags to stay within field bandwidth limits.

7
docs/source/docs/apriltag-pipelines/multitag.md
View File

@@ -24,7 +24,7 @@ This multi-target pose estimate can be accessed using PhotonLib. We suggest usin
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
var result = camera.getLatestResult();
if (result.getMultiTagResult().estimatedPose.isPresent) {
@@ -38,6 +38,11 @@ This multi-target pose estimate can be accessed using PhotonLib. We suggest usin
if (result.MultiTagResult().result.isPresent) {
frc::Transform3d fieldToCamera = result.MultiTagResult().result.best;
}
.. code-block:: Python
# Coming Soon!
```
:::{note}

38
docs/source/docs/contributing/building-photon.md
View File

@@ -243,19 +243,11 @@ The program will wait for the VSCode debugger to attach before proceeding.
### Running examples
You can run one of the many built in examples straight from the command line, too! They contain a fully featured robot project, and some include simulation support. The projects can be found inside the photonlib-java-examples and photonlib-cpp-examples subdirectories, respectively. The projects currently available include:
You can run one of the many built in examples straight from the command line, too! They contain a fully featured robot project, and some include simulation support. The projects can be found inside the photonlib-*-examples subdirectories for each language.
- photonlib-java-examples:
- aimandrange:simulateJava
- aimattarget:simulateJava
- getinrange:simulateJava
- simaimandrange:simulateJava
- simposeest:simulateJava
- photonlib-cpp-examples:
- aimandrange:simulateNative
- getinrange:simulateNative
#### Running C++/Java
To run them, use the commands listed below. PhotonLib must first be published to your local maven repository, then the copy PhotonLib task will copy the generated vendordep json file into each example. After that, the simulateJava/simulateNative task can be used like a normal robot project. Robot simulation with attached debugger is technically possible by using simulateExternalJava and modifying the launch script it exports, though unsupported.
PhotonLib must first be published to your local maven repository, then the copy PhotonLib task will copy the generated vendordep json file into each example. After that, the simulateJava/simulateNative task can be used like a normal robot project. Robot simulation with attached debugger is technically possible by using simulateExternalJava and modifying the launch script it exports, though not yet supported.
```
~/photonvision$ ./gradlew publishToMavenLocal
@@ -268,3 +260,27 @@ To run them, use the commands listed below. PhotonLib must first be published to
~/photonvision/photonlib-cpp-examples$ ./gradlew copyPhotonlib
~/photonvision/photonlib-cpp-examples$ ./gradlew <example-name>:simulateNative
```
#### Running Python
PhotonLibPy must first be built into a wheel.
```
> cd photon-lib/py
> buildAndTest.bat
```
Then, you must enable using the development wheels. robotpy will use pip behind the scenes, and this bat file tells pip about your development artifacts.
Note: This is best done in a virtual environment.
```
> enableUsingDevBuilds.bat
```
Then, run the examples:
```
> cd photonlib-python-examples
> run.bat <example name>
```

2
docs/source/docs/description.md
View File

@@ -3,7 +3,7 @@
## Description
PhotonVision is a free, fast, and easy-to-use vision processing solution for the *FIRST*Robotics Competition. PhotonVision is designed to get vision working on your robot *quickly*, without the significant cost of other similar solutions.
Using PhotonVision, teams can go from setting up a camera and coprocessor to detecting and tracking targets by simply tuning sliders. With an easy to use interface, comprehensive documentation, and a feature rich vendor dependency, no experience is necessary to use PhotonVision. No matter your resources, using PhotonVision is easy compared to its alternatives.
Using PhotonVision, teams can go from setting up a camera and coprocessor to detecting and tracking AprilTags and other targets by simply tuning sliders. With an easy to use interface, comprehensive documentation, and a feature rich vendor dependency, no experience is necessary to use PhotonVision. No matter your resources, using PhotonVision is easy compared to its alternatives.
## Advantages

37
docs/source/docs/examples/aimandrange.md
View File

@@ -4,36 +4,47 @@ The following example is from the PhotonLib example repository ([Java](https://g
## Knowledge and Equipment Needed
- Everything required in {ref}`Aiming at a Target <docs/examples/aimingatatarget:Knowledge and Equipment Needed>` and {ref}`Getting in Range of the Target <docs/examples/gettinginrangeofthetarget:Knowledge and Equipment Needed>`.
- Everything required in {ref}`Aiming at a Target <docs/examples/aimingatatarget:Knowledge and Equipment Needed>`.
## Code
Now that you know how to both aim and get in range of the target, it is time to combine them both at the same time. This example will take the previous two code examples and make them into one function using the same tools as before. With this example, you now have all the knowledge you need to use PhotonVision on your robot in any game.
Now that you know how to aim toward the AprilTag, let's also drive the correct distance from the AprilTag.
To do this, we'll use the *pitch* of the target in the camera image and trigonometry to figure out how far away the robot is from the AprilTag. Then, like before, we'll use the P term of a PID controller to drive the robot to the correct distance.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/aimandrange/src/main/java/frc/robot/Robot.java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/aimandrange/src/main/java/frc/robot/Robot.java
:language: java
:lines: 42-111
:lines: 84-131
:linenos:
:lineno-start: 42
:lineno-start: 84
.. tab-item:: C++ (Header)
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-cpp-examples/aimandrange/src/main/include/Robot.h
:language: cpp
:lines: 27-71
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/aimandrange/src/main/include/Robot.h
:language: c++
:lines: 25-63
:linenos:
:lineno-start: 27
:lineno-start: 25
.. tab-item:: C++ (Source)
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-cpp-examples/aimandrange/src/main/cpp/Robot.cpp
:language: cpp
:lines: 25-67
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/aimandrange/src/main/cpp/Robot.cpp
:language: c++
:lines: 58-107
:linenos:
:lineno-start: 25
:lineno-start: 58
.. tab-item:: Python
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-python-examples/aimandrange/robot.py
:language: python
:lines: 44-98
:linenos:
:lineno-start: 44
```

42
docs/source/docs/examples/aimingatatarget.md
View File

@@ -1,45 +1,55 @@
# Aiming at a Target
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/master/photonlib-java-examples/aimattarget)/[C++](https://github.com/PhotonVision/photonvision/tree/master/photonlib-cpp-examples/aimattarget)).
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/master/photonlib-java-examples/aimattarget)).
## Knowledge and Equipment Needed
- Robot with a vision system running PhotonVision
- Target
- Ability to track a target by properly tuning a pipeline
- A Robot
- A camera mounted rigidly to the robot's frame, cenetered and pointed forward.
- A coprocessor running PhotonVision with an AprilTag or Aurco 2D Pipeline.
- [A printout of Apriltag 7](https://firstfrc.blob.core.windows.net/frc2024/FieldAssets/Apriltag_Images_and_User_Guide.pdf), mounted on a rigid and flat surface.
## Code
Now that you have properly set up your vision system and have tuned a pipeline, you can now aim your robot/turret at the target using the data from PhotonVision. This data is reported over NetworkTables and includes: latency, whether there is a target detected or not, pitch, yaw, area, skew, and target pose relative to the robot. This data will be used/manipulated by our vendor dependency, PhotonLib. The documentation for the Network Tables API can be found {ref}`here <docs/additional-resources/nt-api:Getting Target Information>` and the documentation for PhotonLib {ref}`here <docs/programming/photonlib/adding-vendordep:What is PhotonLib?>`.
Now that you have properly set up your vision system and have tuned a pipeline, you can now aim your robot at an AprilTag using the data from PhotonVision. The *yaw* of the target is the critical piece of data that will be needed first.
For this simple example, only yaw is needed.
Yaw is reported to the roboRIO over Network Tables. PhotonLib, our vender dependency, is the easiest way to access this data. The documentation for the Network Tables API can be found {ref}`here <docs/additional-resources/nt-api:Getting Target Information>` and the documentation for PhotonLib {ref}`here <docs/programming/photonlib/adding-vendordep:What is PhotonLib?>`.
In this example, while the operator holds a button down, the robot will turn towards the goal using the P term of a PID loop. To learn more about how PID loops work, how WPILib implements them, and more, visit [Advanced Controls (PID)](https://docs.wpilib.org/en/stable/docs/software/advanced-control/introduction/index.html) and [PID Control in WPILib](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/controllers/pidcontroller.html#pid-control-in-wpilib).
In this example, while the operator holds a button down, the robot will turn towards the AprilTag using the P term of a PID loop. To learn more about how PID loops work, how WPILib implements them, and more, visit [Advanced Controls (PID)](https://docs.wpilib.org/en/stable/docs/software/advanced-control/introduction/index.html) and [PID Control in WPILib](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/controllers/pidcontroller.html#pid-control-in-wpilib).
```{eval-rst}
.. tab-set::
.. tab-item:: Java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/aimattarget/src/main/java/frc/robot/Robot.java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/aimattarget/src/main/java/frc/robot/Robot.java
:language: java
:lines: 41-98
:lines: 77-117
:linenos:
:lineno-start: 41
:lineno-start: 77
.. tab-item:: C++ (Header)
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-cpp-examples/aimattarget/src/main/include/Robot.h
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/aimattarget/src/main/include/Robot.h
:language: c++
:lines: 27-53
:lines: 25-60
:linenos:
:lineno-start: 27
:lineno-start: 25
.. tab-item:: C++ (Source)
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-cpp-examples/aimattarget/src/main/cpp/Robot.cpp
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/aimattarget/src/main/cpp/Robot.cpp
:language: c++
:lines: 25-52
:lines: 56-96
:linenos:
:lineno-start: 25
:lineno-start: 56
.. tab-item:: Python
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-python-examples/aimattarget/robot.py
:language: python
:lines: 46-70
:linenos:
:lineno-start: 46
```

58
docs/source/docs/examples/gettinginrangeofthetarget.md
View File

@@ -1,58 +0,0 @@
# Getting in Range of the Target
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/master/photonlib-java-examples/getinrange)/[C++](https://github.com/PhotonVision/photonvision/tree/master/photonlib-cpp-examples/getinrange)).
## Knowledge and Equipment Needed
- Everything required in {ref}`Aiming at a Target <docs/examples/aimingatatarget:Knowledge and Equipment Needed>`.
- Large space where your robot can move around freely
## Code
In FRC, a mechanism usually has to be a certain distance away from its target in order to be effective and score. In the previous example, we showed how to aim your robot at the target. Now we will show how to move to a certain distance from the target.
For proper functionality of just this example, ensure that your robot is pointed towards the target.
While the operator holds down a button, the robot will drive towards the target and get in range.
This example uses P term of the PID loop and PhotonLib and the distance function of PhotonUtils.
:::{warning}
The PhotonLib utility to calculate distance depends on the camera being at a different vertical height than the target. If this is not the case, a different method for estimating distance, such as target width or area, should be used. In general, this method becomes more accurate as range decreases and as the height difference increases.
:::
:::{note}
There is no strict minimum delta-height necessary for this method to be applicable, just a requirement that a delta exists.
:::
```{eval-rst}
.. tab-set::
.. tab-item:: Java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/getinrange/src/main/java/frc/robot/Robot.java
:language: java
:lines: 42-107
:linenos:
:lineno-start: 42
.. tab-item:: C++ (Header)
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-cpp-examples/getinrange/src/main/include/Robot.h
:language: c++
:lines: 27-67
:linenos:
:lineno-start: 27
.. tab-item:: C++ (Source)
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-cpp-examples/getinrange/src/main/cpp/Robot.cpp
:language: c++
:lines: 25-58
:linenos:
:lineno-start: 25
```
:::{hint}
The accuracy of the measurement of the camera's pitch ({code}`CAMERA_PITCH_RADIANS` in the above example), as well as the camera's FOV, will determine the overall accuracy of this method.
:::

BIN
docs/source/docs/examples/images/poseest_demo.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 MiB

4
docs/source/docs/examples/index.md
View File

@@ -4,8 +4,6 @@
:maxdepth: 1
aimingatatarget
gettinginrangeofthetarget
aimandrange
simaimandrange
simposeest
poseest
```

211
docs/source/docs/examples/poseest.md Normal file
View File

@@ -0,0 +1,211 @@
# Using WPILib Pose Estimation, Simulation, and PhotonVision Together
The following example comes from the PhotonLib example repository ([Java](https://github.com/gerth2/photonvision/tree/master/photonlib-java-examples/poseest)/[C++](https://github.com/gerth2/photonvision/tree/master/photonlib-cpp-examples/poseest)/[Python](https://github.com/gerth2/photonvision/tree/master/photonlib-python-examples/poseest)). Full code is available at that links.
## Knowledge and Equipment Needed
- Everything required in {ref}`Combining Aiming and Getting in Range <docs/examples/aimandrange:Knowledge and Equipment Needed>`, plus some familiarity with WPILib pose estimation functionality.
## Background
This example demonstrates integration of swerve drive control, a basic swerve physics simulation, and PhotonLib's simulated vision system functionality.
## Walkthrough
### Estimating Pose
The {code}`Drivetrain` class includes functionality to fuse multiple sensor readings together (including PhotonVision) into a best-guess of the pose on the field.
Please reference the [WPILib documentation](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/state-space/state-space-pose_state-estimators.html) on using the {code}`SwerveDrivePoseEstimator` class.
We use the 2024 game's AprilTag Locations:
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/poseest/src/main/java/frc/robot/Vision.java
:language: java
:lines: 68-68
:linenos:
:lineno-start: 68
.. tab-item:: C++
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/poseest/src/main/include/Constants.h
:language: c++
:lines: 42-43
:linenos:
:lineno-start: 42
.. tab-item:: Python
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 46-46
:linenos:
:lineno-start: 46
```
To incorporate PhotonVision, we need to create a {code}`PhotonCamera`:
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/poseest/src/main/java/frc/robot/Vision.java
:language: java
:lines: 57-57
:linenos:
:lineno-start: 57
.. tab-item:: C++
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/poseest/src/main/include/Vision.h
:language: c++
:lines: 145-145
:linenos:
:lineno-start: 145
.. tab-item:: Python
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 44-44
:linenos:
:lineno-start: 44
```
During periodic execution, we read back camera results. If we see AprilTags in the image, we calculate the camera-measured pose of the robot and pass it to the {code}`Drivetrain`.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/poseest/src/main/java/frc/robot/Robot.java
:language: java
:lines: 64-74
:linenos:
:lineno-start: 64
.. tab-item:: C++
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/poseest/src/main/cpp/Robot.cpp
:language: c++
:lines: 38-46
:linenos:
:lineno-start: 38
.. tab-item:: Python
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 54-56
:linenos:
:lineno-start: 54
```
### Simulating the Camera
First, we create a new {code}`VisionSystemSim` to represent our camera and coprocessor running PhotonVision, and moving around our simulated field.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/poseest/src/main/java/frc/robot/Vision.java
:language: java
:lines: 65-69
:linenos:
:lineno-start: 65
.. tab-item:: C++
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/poseest/src/main/include/Vision.h
:language: c++
:lines: 49-52
:linenos:
:lineno-start: 49
.. tab-item:: Python
# Coming Soon!
```
Then, we add configure the simulated vision system to match the camera system being simulated.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/poseest/src/main/java/frc/robot/Vision.java
:language: java
:lines: 69-82
:linenos:
:lineno-start: 69
.. tab-item:: C++
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/poseest/src/main/include/Vision.h
:language: c++
:lines: 53-65
:linenos:
:lineno-start: 53
.. tab-item:: Python
# Coming Soon!
```
### Updating the Simulated Vision System
During simulation, we periodically update the simulated vision system.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-java-examples/poseest/src/main/java/frc/robot/Robot.java
:language: java
:lines: 114-132
:linenos:
:lineno-start: 114
.. tab-item:: C++
.. rli:: https://raw.githubusercontent.com/gerth2/photonvision/adb3098fbe0cdbc1a378c6d5a41126dd1d6d6955/photonlib-cpp-examples/poseest/src/main/cpp/Robot.cpp
:language: c++
:lines: 95-109
:linenos:
:lineno-start: 95
.. tab-item:: Python
# Coming Soon!
```
The rest is done behind the scenes.
```{image} images/poseest_demo.gif
:alt: Simulated swerve drive and vision system working together in teleoperated mode.
:width: 1200
```

96
docs/source/docs/examples/simaimandrange.md
View File

@@ -1,96 +0,0 @@
# Simulating Aiming and Getting in Range
The following example comes from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/661f8b2c0495474015f6ea9a89d65f9788436a05/photonlib-java-examples/src/main/java/org/photonlib/examples/simaimandrange)/[C++](https://github.com/PhotonVision/photonvision/tree/661f8b2c0495474015f6ea9a89d65f9788436a05/photonlib-cpp-examples/src/main/cpp/examples/simaimandrange)). Full code is available at those links.
## Knowledge and Equipment Needed
- Everything required in {ref}`Combining Aiming and Getting in Range <docs/examples/aimandrange:Knowledge and Equipment Needed>`.
## Background
The previous examples show how to run PhotonVision on a real robot, with a physical robot drivetrain moving around and interacting with the software.
This example builds upon that, adding support for simulating robot motion and incorporating that motion into a {code}`SimVisionSystem`. This allows you to test control algorithms on your development computer, without requiring access to a real robot.
```{raw} html
<video width="85%" controls>
<source src="../../_static/assets/simaimandrange.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
```
## Walkthrough
First, in the main {code}`Robot` source file, we add support to periodically update a new simulation-specific object. This logic only gets used while running in simulation:
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/Robot.java
:language: java
:lines: 118-128
:linenos:
:lineno-start: 118
```
Then, we add in the implementation of our new `DrivetrainSim` class. Please reference the [WPILib documentation on physics simulation](https://docs.wpilib.org/en/stable/docs/software/wpilib-tools/robot-simulation/physics-sim.html).
Simulated Vision support is added with the following steps:
### Creating the Simulated Vision System
First, we create a new {code}`SimVisionSystem` to represent our camera and coprocessor running PhotonVision.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/sim/DrivetrainSim.java
:language: java
:lines: 73-93
:linenos:
:lineno-start: 72
```
Next, we create objects to represent the physical location and size of the vision targets we are calibrated to detect. This example models the down-field high goal vision target from the 2020 and 2021 games.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/sim/DrivetrainSim.java
:language: java
:lines: 95-111
:linenos:
:lineno-start: 95
```
Finally, we add our target to the simulated vision system.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/sim/DrivetrainSim.java
:language: java
:lines: 116-117
:linenos:
:lineno-start: 113
```
If you have additional targets you want to detect, you can add them in the same way as the first one.
### Updating the Simulated Vision System
Once we have all the properties of our simulated vision system defined, the work to do at runtime becomes very minimal. Simply pass in the robot's pose periodically to the simulated vision system.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/ebef19af3d926cf87292177c9a16d01b71219306/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/sim/DrivetrainSim.java
:language: java
:lines: 124-142
:linenos:
:lineno-start: 122
```
The rest is done behind the scenes.

133
docs/source/docs/examples/simposeest.md
View File

@@ -1,133 +0,0 @@
# Using WPILib Pose Estimation, Simulation, and PhotonVision Together
The following example comes from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/master/photonlib-java-examples/)). Full code is available at that links.
## Knowledge and Equipment Needed
- Everything required in {ref}`Combining Aiming and Getting in Range <docs/examples/aimandrange:Knowledge and Equipment Needed>`, plus some familiarity with WPILib pose estimation functionality.
## Background
This example builds upon WPILib's [Differential Drive Pose Estimator](https://github.com/wpilibsuite/allwpilib/tree/main/wpilibjExamples/src/main/java/edu/wpi/first/wpilibj/examples/differentialdriveposeestimator). It adds a {code}`PhotonCamera` to gather estimates of the robot's position on the field. This in turn can be used for aligning with vision targets, and increasing accuracy of autonomous routines.
To support simulation, a {code}`SimVisionSystem` is used to drive data into the {code}`PhotonCamera`. The far high goal target from 2020 is modeled.
## Walkthrough
WPILib's {code}`Pose2d` class is used to represent robot positions on the field.
Three different {code}`Pose2d` positions are relevant for this example:
1. Desired Pose: The location some autonomous routine wants the robot to be in.
2. Estimated Pose: The location the software `believes` the robot to be in, based on physics models and sensor feedback.
3. Actual Pose: The locations the robot is actually at. The physics simulation generates this in simulation, but it cannot be directly measured on the real robot.
### Estimating Pose
The {code}`DrivetrainPoseEstimator` class is responsible for generating an estimated robot pose using sensor readings (including PhotonVision).
Please reference the [WPILib documentation](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/state-space/state-space-pose_state-estimators.html) on using the {code}`DifferentialDrivePoseEstimator` class.
For both simulation and on-robot code, we create objects to represent the physical location and size of the vision targets we are calibrated to detect. This example models the down-field high goal vision target from the 2020 and 2021 games.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simposeest/src/main/java/frc/robot/Constants.java
:language: java
:lines: 83-106
:linenos:
:lineno-start: 83
```
To incorporate PhotonVision, we need to create a {code}`PhotonCamera`:
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simposeest/src/main/java/frc/robot/DrivetrainPoseEstimator.java
:language: java
:lines: 46
:linenos:
:lineno-start: 46
```
During periodic execution, we read back camera results. If we see a target in the image, we pass the camera-measured pose of the robot to the {code}`DifferentialDrivePoseEstimator`.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simposeest/src/main/java/frc/robot/DrivetrainPoseEstimator.java
:language: java
:lines: 81-92
:linenos:
:lineno-start: 81
```
That's it!
### Simulating the Camera
First, we create a new {code}`SimVisionSystem` to represent our camera and coprocessor running PhotonVision.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simposeest/src/main/java/frc/robot/DrivetrainSim.java
:language: java
:lines: 76-95
:linenos:
:lineno-start: 76
```
Then, we add our target to the simulated vision system.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simposeest/src/main/java/frc/robot/DrivetrainSim.java
:lines: 97-99
:linenos:
:lineno-start: 97
```
If you have additional targets you want to detect, you can add them in the same way as the first one.
### Updating the Simulated Vision System
Once we have all the properties of our simulated vision system defined, the remaining work is minimal. Periodically, pass in the robot's pose to the simulated vision system.
```{eval-rst}
.. tab-set::
.. tab-item:: Java
:sync: java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simposeest/src/main/java/frc/robot/DrivetrainSim.java
:language: java
:lines: 138-139
:linenos:
:lineno-start: 138
```
The rest is done behind the scenes.

6
docs/source/docs/installation/networking.md
View File

@@ -44,13 +44,17 @@ If you would like to access your Ethernet-connected vision device from a compute
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
PortForwarder.add(5800, "photonvision.local", 5800);
.. code-block:: C++
wpi::PortForwarder::GetInstance().Add(5800, "photonvision.local", 5800);
.. code-block:: Python
# Coming Soon!
```
:::{note}

2
docs/source/docs/installation/sw_install/other-coprocessors.md
View File

@@ -1,7 +1,7 @@
# Other Debian-Based Co-Processor Installation
:::{warning}
Working with unsupported coprocessors requires some level of "know how" of your target system. The install script has only been tested on Debian/Raspberry Pi OS Buster and Ubuntu Bionic. If any issues arise with your specific OS, please open an issue on our [issues page](https://github.com/PhotonVision/photonvision/issues).
Working with unsupported coprocessors requires some level of "know how" of your system. The install script has only been tested on Debian/Raspberry Pi OS Buster and Ubuntu Bionic. If any issues arise with your specific OS, please open an issue on our [issues page](https://github.com/PhotonVision/photonvision/issues).
:::
:::{note}

65
docs/source/docs/integration/advancedStrategies.md
View File

@@ -1,8 +1,12 @@
# Advanced Strategies
Advanced strategies for using vision processing results involve working with the robot's *pose* on the field. A *pose* is a combination an X/Y coordinate, and an angle describing where the robot's front is pointed. It is always considered *relative* to some fixed point on the field.
Advanced strategies for using vision processing results involve working with the robot's *pose* on the field.
WPILib provides a [Pose2d](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/geometry/pose.html) class to describe poses in software.
A *pose* is a combination an X/Y coordinate, and an angle describing where the robot's front is pointed. A pose is always considered *relative* to some fixed point on the field.
WPILib provides a [Pose2d](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/geometry/pose.html) class to describe poses in software.
PhotonVision can supply correcting information to keep estimates of *pose* accurate over a full match.
## Knowledge and Equipment Needed
@@ -12,30 +16,53 @@ WPILib provides a [Pose2d](https://docs.wpilib.org/en/stable/docs/software/advan
\- Sufficient sensors to measure wheel rotation
\- Capable of closed-loop velocity control
- A gyroscope or IMU measuring actual robot heading
- Experience using some path-planning library (WPILib is our recommendation)
- Experience using some path-planning library
## Path Planning in a Target-Centered Reference Frame
## Robot Poses from the Camera
When using 3D mode in PhotonVision, the [SolvePNP Algorithm](https://en.wikipedia.org/wiki/Perspective-n-Point) is used to deduce the *camera's* position in a 3D coordinate system centered on the target itself.
When using 3D mode in PhotonVision, an additional step is run to estimate the 3D position of camera, relative to one or more AprilTags.
A simple algorithm for using this measurement is:
This process does not produce a *unique* solution. There are multiple possible camera positions which might explain the image it observed. Additionally, the camera is rarely mounted in the exact center of a robot.
1. Assume your robot needs to be at a fixed `Pose2D` *relative to the target*.
2. When triggered:
#. Read the most recent vision measurement - this is your *actual* pose.
#. Generate a simple trajectory to the goal position
#. Execute the trajectory
For these reasons, the 3D information must be filtered and transformed before they can describe the robot's pose.
:::{note}
There is not currently an example demonstrating this technique.
:::
PhotonLib provides {ref}`a utility class to assist with this process on the roboRIO <docs/programming/photonlib/robot-pose-estimator:AprilTags and PhotonPoseEstimator>`. Alternatively, {ref}`a "multi-tag" strategy can do this process on the coprocessor. <docs/apriltag-pipelines/multitag:Enabling MultiTag>`.
## Global Pose Estimation
## Field-Relative Pose Estimation
A more complex way to utilize a camera-supplied `Pose2D` is to incorporate it into an estimation of the robot's `Pose2D` in a global field reference frame.
The camera's guess of the robot pose generally should be *fused* with other sensor readings.
When using this strategy, the measurements made by the camera are *fused* with measurements from other sensors, a model of expected robot behavior, and a matrix of weights that describes how trustworthy each sensor is. The result is a *best-guess* at the current pose on the field.
WPILib provides [a set of pose estimation classes](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/state-space/state-space-pose-estimators.html) for doing this work.
In turn, this best-guess position is used to path plan to the known positions on the field, which may or may not have vision targets nearby.
## I have a Pose Estimate, Now What?
See the {ref}`Pose Estimation <docs/examples/simposeest:Knowledge and Equipment Needed>` example for more information.
### Triggering Actions Automatically
A simple way to use a pose estimate is to activate robot functions automatically when in the correct spot on the field.
```{eval-rst}
.. tab-set-code::
.. code-block:: Java
Pose3d robotPose;
boolean launcherSpinCmd;
// ...
if(robotPose.X() < 1.5){
// Near blue alliance wall, start spinning the launcher wheel
launcherSpinCmd = True;
} else {
// Far away, no need to run launcher.
launcherSpinCmd = False;
}
// ...
```
### PathPlanning
A common, but more complex usage of a pose estimate is an input to a path-following algorithm. Specifically, the pose estimate is used to correct for the robot straying off of the pre-defined path.
See the {ref}`Pose Estimation <docs/examples/poseest:Knowledge and Equipment Needed>` example for details on integrating this.

46
docs/source/docs/integration/aprilTagStrategies.md
View File

@@ -1,46 +0,0 @@
# AprilTag Strategies
:::{note}
The same strategies covered in the simple and advanced strategy sections still apply to AprilTags, and we encourage you to read them first. This page will discuss the specific nuances to using AprilTags.
:::
## Simple Strategies
Prior to the introduction of AprilTags, the most common vision strategy for teams was to use the yaw of the detected target in order to turn to the target, and then score. This is still possible with AprilTags as the yaw of the tag is reported. Similarly, getting the distance to the target via trigonometry will also work. This is discussed in greater detail in the previous page.
## Advanced Strategies
AprilTags allows you find the robot pose on the field using data from the tags. A pose is a combination an X/Y coordinate, and an angle describing where the robot’s front is pointed. It is always considered relative to some fixed point on the field.
### Knowledge and Equipment Needed
Knowledge
- How to tune an AprilTag Pipeline (found in the pipeline tuning section)
Equipment
- A Coprocessor running PhotonVision - Accurate camera calibration to support “3D mode” required
- A Drivetrain with wheels and sensors (Sufficient sensors to measure wheel rotation and capable of closed-loop velocity control)
- A gyroscope or IMU measuring actual robot heading
### Global Pose Estimation / Pose Estimation Strategies
:::{note}
See the previous page for more general information. Most of the information is the same except now the camera is supplying a `Pose3D`.
:::
The nature of how AprilTags will be laid out makes it very likely that you will get multiple pose measurements within a single frame from seeing multiple targets. This requires strategies to fuse these observations together and get a "best guess" as to where your robot is. The best way to do this is to use the corners from all visible AprilTags to estimate the robot's pose. This is done by using the `PhotonPoseEstimator` class and the "MULTI_TAG_PNP_ON_COPROCESSOR" strategy. Additional strategies include:
- A camera seeing multiple targets, taking the average of all the returned poses
- A camera seeing one target, with an assumed height off the ground, picking the pose which places it to the assumed height
- A camera seeing one target, and picking a pose most similar to the most recently observed pose
- A camera seeing one target, and picking a pose most similar to one provided externally (ie, from previous loop's odometry)
- A camera seeing one target, and picking the pose with the lowest ambiguity.
PhotonVision supports all of these different strategies via our `PhotonPoseEstimator` class that allows you to select one of the strategies above and get the relevant pose estimation.
### Tuning Pose Estimators
Coming soon!
TODO: Add this back in once simposeest example is added.

1
docs/source/docs/integration/index.md
View File

@@ -6,5 +6,4 @@
background
simpleStrategies
advancedStrategies
aprilTagStrategies
```

26
docs/source/docs/integration/simpleStrategies.md
View File

@@ -1,36 +1,32 @@
# Simple Strategies
Simple strategies for using vision processor outputs involve using the target's position in the 2D image to infer *range* and *angle* to the target.
Simple strategies for using vision processor outputs involve using the target's position in the 2D image to infer *range* and *angle* to a particular AprilTag.
## Knowledge and Equipment Needed
- A Coprocessor running PhotonVision
- A Drivetrain with wheels
- An AprilTag to aim at
## Angle Alignment
The simplest way to use a vision processing result is to first determine how far left or right in the image the vision target should be for your robot to be "aligned" to the target. Then,
The simplest way to align a robot to an AprilTag is to rotate the drivetrain until the tag is centered in the camera image. To do this,
1. Read the current angle to the target from the vision Coprocessor.
2. If too far in one direction, command the drivetrain to rotate in the opposite direction to compensate.
1. Read the current yaw angle to the AprilTag from the vision Coprocessor.
2. If too far off to one side, command the drivetrain to rotate in the opposite direction to compensate.
See the {ref}`Aiming at a Target <docs/examples/aimingatatarget:Knowledge and Equipment Needed>` example for more information.
:::{note}
Sometimes, these strategies have also involved incorporating a gyroscope. This can be necessary due to the high latency of vision processing algorithms. However, advancements in the tools available (including PhotonVision) has made that unnecessary for most applications.
:::
NOTE: This works if the camera is centered on the robot. This is easiest from a software perspective. If the camera is not centered, take a peek at the next example - it shows how to account for an offset.
## Range Alignment
## Adding Range Alignment
By looking at the position of the target in the "vertical" direction in the image, and applying some trigonometry, the distance between the robot and the camera can be deduced.
By looking at the position of the AprilTag in the "vertical" direction in the image, and applying some trigonometry, the distance between the robot and the camera can be deduced.
1. Read the current distance to the target from the vision coprocessor.
1. Read the current pitch angle to the AprilTag from the vision coprocessor.
2. Do math to calculate the distance to the AprilTag.
2. If too far in one direction, command the drivetrain to travel in the opposite direction to compensate.
See the {ref}`Getting in Range of the Target <docs/examples/gettinginrangeofthetarget:Knowledge and Equipment Needed>` example for more information.
## Angle + Range
Since the previous two alignment strategies work on independent axes of the robot, there's no reason you can't do them simultaneously.
This can be done simultaneously while aligning to the desired angle.
See the {ref}`Aim and Range <docs/examples/aimandrange:Knowledge and Equipment Needed>` example for more information.

8
docs/source/docs/programming/photonlib/controlling-led.md
View File

@@ -4,13 +4,17 @@ You can control the vision LEDs of supported hardware via PhotonLib using the `s
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Blink the LEDs.
camera.setLED(VisionLEDMode.kBlink);
.. code-block:: c++
.. code-block:: C++
// Blink the LEDs.
camera.SetLED(photonlib::VisionLEDMode::kBlink);
.. code-block:: Python
# Coming Soon!
```

20
docs/source/docs/programming/photonlib/driver-mode-pipeline-index.md
View File

@@ -9,7 +9,7 @@ You can use the `setDriverMode()`/`SetDriverMode()` (Java and C++ respectively)
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Set driver mode to on.
camera.setDriverMode(true);
@@ -18,6 +18,10 @@ You can use the `setDriverMode()`/`SetDriverMode()` (Java and C++ respectively)
// Set driver mode to on.
camera.SetDriverMode(true);
.. code-block:: Python
# Coming Soon!
```
## Setting the Pipeline Index
@@ -27,7 +31,7 @@ You can use the `setPipelineIndex()`/`SetPipelineIndex()` (Java and C++ respecti
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Change pipeline to 2
camera.setPipelineIndex(2);
@@ -36,6 +40,10 @@ You can use the `setPipelineIndex()`/`SetPipelineIndex()` (Java and C++ respecti
// Change pipeline to 2
camera.SetPipelineIndex(2);
.. code-block:: Python
# Coming Soon!
```
## Getting the Pipeline Latency
@@ -44,15 +52,19 @@ You can also get the pipeline latency from a pipeline result using the `getLaten
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Get the pipeline latency.
double latencySeconds = result.getLatencyMillis() / 1000.0;
.. code-block:: c++
.. code-block:: C++
// Get the pipeline latency.
units::second_t latency = result.GetLatency();
.. code-block:: Python
# Coming Soon!
```
:::{note}

40
docs/source/docs/programming/photonlib/getting-target-data.md
View File

@@ -20,7 +20,7 @@ The `PhotonCamera` class has two constructors: one that takes a `NetworkTable` a
:language: c++
:lines: 42-43
.. code-block:: python
.. code-block:: Python
# Change this to match the name of your camera as shown in the web ui
self.camera = PhotonCamera("your_camera_name_here")
@@ -51,7 +51,7 @@ Use the `getLatestResult()`/`GetLatestResult()` (Java and C++ respectively) to o
:language: c++
:lines: 35-36
.. code-block:: python
.. code-block:: Python
# Query the latest result from PhotonVision
result = self.camera.getLatestResult()
@@ -69,17 +69,17 @@ Each pipeline result has a `hasTargets()`/`HasTargets()` (Java and C++ respectiv
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Check if the latest result has any targets.
boolean hasTargets = result.hasTargets();
.. code-block:: c++
.. code-block:: C++
// Check if the latest result has any targets.
bool hasTargets = result.HasTargets();
.. code-block:: python
.. code-block:: Python
# Check if the latest result has any targets.
hasTargets = result.hasTargets()
@@ -99,17 +99,17 @@ You can get a list of tracked targets using the `getTargets()`/`GetTargets()` (J
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Get a list of currently tracked targets.
List<PhotonTrackedTarget> targets = result.getTargets();
.. code-block:: c++
.. code-block:: C++
// Get a list of currently tracked targets.
wpi::ArrayRef<photonlib::PhotonTrackedTarget> targets = result.GetTargets();
.. code-block:: python
.. code-block:: Python
# Get a list of currently tracked targets.
targets = result.getTargets()
@@ -121,20 +121,20 @@ You can get the {ref}`best target <docs/reflectiveAndShape/contour-filtering:Con
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Get the current best target.
PhotonTrackedTarget target = result.getBestTarget();
.. code-block:: c++
.. code-block:: C++
// Get the current best target.
photonlib::PhotonTrackedTarget target = result.GetBestTarget();
.. code-block:: python
.. code-block:: Python
# TODO - Not currently supported
# Coming Soon!
```
@@ -149,7 +149,7 @@ You can get the {ref}`best target <docs/reflectiveAndShape/contour-filtering:Con
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Get information from target.
double yaw = target.getYaw();
@@ -159,7 +159,7 @@ You can get the {ref}`best target <docs/reflectiveAndShape/contour-filtering:Con
Transform2d pose = target.getCameraToTarget();
List<TargetCorner> corners = target.getCorners();
.. code-block:: c++
.. code-block:: C++
// Get information from target.
double yaw = target.GetYaw();
@@ -169,7 +169,7 @@ You can get the {ref}`best target <docs/reflectiveAndShape/contour-filtering:Con
frc::Transform2d pose = target.GetCameraToTarget();
wpi::SmallVector<std::pair<double, double>, 4> corners = target.GetCorners();
.. code-block:: python
.. code-block:: Python
# Get information from target.
yaw = target.getYaw()
@@ -193,7 +193,7 @@ All of the data above (**except skew**) is available when using AprilTags.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Get information from target.
int targetID = target.getFiducialId();
@@ -201,7 +201,7 @@ All of the data above (**except skew**) is available when using AprilTags.
Transform3d bestCameraToTarget = target.getBestCameraToTarget();
Transform3d alternateCameraToTarget = target.getAlternateCameraToTarget();
.. code-block:: c++
.. code-block:: C++
// Get information from target.
int targetID = target.GetFiducialId();
@@ -209,7 +209,7 @@ All of the data above (**except skew**) is available when using AprilTags.
frc::Transform3d bestCameraToTarget = target.getBestCameraToTarget();
frc::Transform3d alternateCameraToTarget = target.getAlternateCameraToTarget();
.. code-block:: python
.. code-block:: Python
# Get information from target.
targetID = target.getFiducialId()
@@ -227,7 +227,7 @@ Images are stored within the PhotonVision configuration directory. Running the "
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Capture pre-process camera stream image
camera.takeInputSnapshot();
@@ -243,7 +243,7 @@ Images are stored within the PhotonVision configuration directory. Running the "
// Capture post-process camera stream image
camera.TakeOutputSnapshot();
.. code-block:: python
.. code-block:: Python
# Capture pre-process camera stream image
camera.takeInputSnapshot()

35
docs/source/docs/programming/photonlib/robot-pose-estimator.md
View File

@@ -14,16 +14,20 @@ The API documentation can be found in here: [Java](https://github.wpilib.org/all
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// The field from AprilTagFields will be different depending on the game.
AprilTagFieldLayout aprilTagFieldLayout = AprilTagFields.k2024Crescendo.loadAprilTagLayoutField();
.. code-block:: c++
.. code-block:: C++
// The parameter for LoadAPrilTagLayoutField will be different depending on the game.
frc::AprilTagFieldLayout aprilTagFieldLayout = frc::LoadAprilTagLayoutField(frc::AprilTagField::k2024Crescendo);
.. code-block:: Python
# Coming Soon!
```
## Creating a `PhotonPoseEstimator`
@@ -46,7 +50,7 @@ The PhotonPoseEstimator has a constructor that takes an `AprilTagFieldLayout` (s
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
//Forward Camera
cam = new PhotonCamera("testCamera");
@@ -55,7 +59,7 @@ The PhotonPoseEstimator has a constructor that takes an `AprilTagFieldLayout` (s
// Construct PhotonPoseEstimator
PhotonPoseEstimator photonPoseEstimator = new PhotonPoseEstimator(aprilTagFieldLayout, PoseStrategy.CLOSEST_TO_REFERENCE_POSE, cam, robotToCam);
.. code-block:: c++
.. code-block:: C++
// Forward Camera
std::shared_ptr<photonlib::PhotonCamera> cameraOne =
@@ -76,6 +80,22 @@ The PhotonPoseEstimator has a constructor that takes an `AprilTagFieldLayout` (s
photonlib::RobotPoseEstimator estimator(
aprilTags, photonlib::CLOSEST_TO_REFERENCE_POSE, cameras);
.. code-block:: Python
kRobotToCam = wpimath.geometry.Transform3d(
wpimath.geometry.Translation3d(0.5, 0.0, 0.5),
wpimath.geometry.Rotation3d.fromDegrees(0.0, -30.0, 0.0),
)
self.cam = PhotonCamera("YOUR CAMERA NAME")
self.camPoseEst = PhotonPoseEstimator(
loadAprilTagLayoutField(AprilTagField.k2024Crescendo),
PoseStrategy.CLOSEST_TO_REFERENCE_POSE,
self.cam,
kRobotToCam
)
```
## Using a `PhotonPoseEstimator`
@@ -88,7 +108,7 @@ Calling `update()` on your `PhotonPoseEstimator` will return an `EstimatedRobotP
:language: java
:lines: 85-88
.. code-block:: c++
.. code-block:: C++
std::pair<frc::Pose2d, units::millisecond_t> getEstimatedGlobalPose(
frc::Pose3d prevEstimatedRobotPose) {
@@ -102,6 +122,11 @@ Calling `update()` on your `PhotonPoseEstimator` will return an `EstimatedRobotP
return std::make_pair(frc::Pose2d(), 0_ms);
}
}
.. code-block:: Python
```
You should be updating your [drivetrain pose estimator](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html) with the result from the `RobotPoseEstimator` every loop using `addVisionMeasurement()`. TODO: add example note

59
docs/source/docs/programming/photonlib/using-target-data.md
View File

@@ -1,6 +1,6 @@
# Using Target Data
A `PhotonUtils` class with helpful common calculations is included within `PhotonLib` to aid teams in using target data in order to get positional information on the field. This class contains two methods, `calculateDistanceToTargetMeters()`/`CalculateDistanceToTarget()` and `estimateTargetTranslation2d()`/`EstimateTargetTranslation()` (Java and C++ respectively).
A `PhotonUtils` class with helpful common calculations is included within `PhotonLib` to aid teams in using AprilTag data in order to get positional information on the field. This class contains two methods, `calculateDistanceToTargetMeters()`/`CalculateDistanceToTarget()` and `estimateTargetTranslation2d()`/`EstimateTargetTranslation()` (Java and C++ respectively).
## Estimating Field Relative Pose with AprilTags
@@ -8,13 +8,17 @@ A `PhotonUtils` class with helpful common calculations is included within `Photo
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Calculate robot's field relative pose
Pose3d robotPose = PhotonUtils.estimateFieldToRobotAprilTag(target.getBestCameraToTarget(), aprilTagFieldLayout.getTagPose(target.getFiducialId()), cameraToRobot);
.. code-block:: c++
.. code-block:: C++
//TODO
.. code-block:: Python
# Coming Soon!
```
## Estimating Field Relative Pose (Traditional)
@@ -23,18 +27,22 @@ You can get your robot's `Pose2D` on the field using various camera data, target
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Calculate robot's field relative pose
Pose2D robotPose = PhotonUtils.estimateFieldToRobot(
kCameraHeight, kTargetHeight, kCameraPitch, kTargetPitch, Rotation2d.fromDegrees(-target.getYaw()), gyro.getRotation2d(), targetPose, cameraToRobot);
.. code-block:: c++
.. code-block:: C++
// Calculate robot's field relative pose
frc::Pose2D robotPose = photonlib::EstimateFieldToRobot(
kCameraHeight, kTargetHeight, kCameraPitch, kTargetPitch, frc::Rotation2d(units::degree_t(-target.GetYaw())), frc::Rotation2d(units::degree_t(gyro.GetRotation2d)), targetPose, cameraToRobot);
.. code-block:: Python
# Coming Soon!
```
## Calculating Distance to Target
@@ -44,14 +52,18 @@ If your camera is at a fixed height on your robot and the height of the target i
```{eval-rst}
.. tab-set-code::
.. code-block:: Java
.. rli:: https://github.com/PhotonVision/photonvision/raw/a3bcd3ac4f88acd4665371abc3073bdbe5effea8/photonlib-java-examples/src/main/java/org/photonlib/examples/getinrange/Robot.java
:language: java
:lines: 78-94
// TODO
.. code-block:: C++
// TODO
.. code-block:: Python
# Coming Soon!
.. rli:: https://github.com/PhotonVision/photonvision/raw/a3bcd3ac4f88acd4665371abc3073bdbe5effea8/photonlib-cpp-examples/src/main/cpp/examples/getinrange/cpp/Robot.cpp
:language: cpp
:lines: 33-46
```
:::{note}
@@ -64,13 +76,17 @@ The C++ version of PhotonLib uses the Units library. For more information, see [
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
double distanceToTarget = PhotonUtils.getDistanceToPose(robotPose, targetPose);
.. code-block:: c++
.. code-block:: C++
//TODO
.. code-block:: Python
# Coming Soon!
```
## Estimating Camera Translation to Target
@@ -79,17 +95,22 @@ You can get a [translation](https://docs.wpilib.org/en/latest/docs/software/adva
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Calculate a translation from the camera to the target.
Translation2d translation = PhotonUtils.estimateCameraToTargetTranslation(
distanceMeters, Rotation2d.fromDegrees(-target.getYaw()));
.. code-block:: c++
.. code-block:: C++
// Calculate a translation from the camera to the target.
frc::Translation2d translation = photonlib::PhotonUtils::EstimateCameraToTargetTranslationn(
distance, frc::Rotation2d(units::degree_t(-target.GetYaw())));
.. code-block:: Python
# Coming Soon!
```
:::{note}
@@ -102,10 +123,14 @@ We are negating the yaw from the camera from CV (computer vision) conventions to
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
Rotation2d targetYaw = PhotonUtils.getYawToPose(robotPose, targetPose);
.. code-block:: c++
.. code-block:: C++
//TODO
.. code-block:: Python
# Coming Soon!
```

4
docs/source/docs/reflectiveAndShape/3D.md
View File

@@ -1,6 +1,6 @@
# 3D Tuning
In 3D mode, the SolvePNP algorithm is used to compute the position and rotation of the target relative to the robot. This requires your {ref}`camera to be calibrated <docs/calibration/calibration:Calibrating Your Camera>` which can be done through the cameras tab.
In 3D mode, the SolvePNP algorithm is used to compute the position and rotation of the AprilTag or other target relative to the robot. This requires your {ref}`camera to be calibrated <docs/calibration/calibration:Calibrating Your Camera>` which can be done through the cameras tab.
The target model dropdown is used to select the target model used to compute target position. This should match the target your camera will be tracking.
@@ -17,6 +17,6 @@ If solvePNP is working correctly, the target should be displayed as a small rect
</video>
```
## Contour Simplification
## Contour Simplification (Non-Apriltag)
3D mode internally computes a polygon that approximates the target contour being tracked. This polygon is used to detect the extreme corners of the target. The contour simplification slider changes how far from the original contour the approximation is allowed to deviate. Note that the approximate polygon is drawn on the output image for tuning.

2
docs/source/docs/simulation/hardware-in-the-loop-sim.md
View File

@@ -2,7 +2,7 @@
Hardware in the loop simulation is using a physical device, such as a supported co-processor running PhotonVision, to enhance simulation capabilities. This is useful for developing and validating code before the camera is attached to a robot, as well as reducing the work required to use WPILib simulation with PhotonVision.
Before continuing, ensure PhotonVision is installed on your target device. Instructions can be found {ref}`here <docs/installation/index:Installation & Setup>` for all devices.
Before continuing, ensure PhotonVision is installed on your device. Instructions can be found {ref}`here <docs/installation/index:Installation & Setup>` for all devices.
Your coprocessor and computer running simulation will have to be connected to the same network, like a home router. Connecting the coprocessor directly to the computer will not work.

5
docs/source/docs/simulation/index.md
View File

@@ -4,7 +4,8 @@
:maxdepth: 0
:titlesonly: true
simulation
simulation-deprecated
simulation-java
simulation-cpp
simulation-python
hardware-in-the-loop-sim
```

5
docs/source/docs/simulation/simulation-cpp.md Normal file
View File

@@ -0,0 +1,5 @@
# Simulation Support in PhotonLib in C++
## What Is Supported?
Nothing yet.

103
docs/source/docs/simulation/simulation-deprecated.md
View File

@@ -1,103 +0,0 @@
# Simulation Support in PhotonLib (Deprecated)
:::{attention}
This page details the pre-2024 simulation support. For current Java simulation support, see {doc}`/docs/simulation/simulation`.
:::
## What Is Supported?
PhotonLib supports simulation of a camera and coprocessor running PhotonVision moving about a field on a robot.
You can use this to help validate your robot code's behavior in simulation without needing a physical robot.
## Simulation Vision World Model
Sim-specific classes are provided to model sending one frame of a camera image through PhotonVision. Based on what targets are visible, results are published to NetworkTables.
While processing, the given robot `Pose2d` is used to analyze which targets should be in view, and determine where they would have shown up in the camera image.
Targets are considered in view if:
1. Their centroid is within the field of view of the camera.
2. The camera is not in driver mode.
3. The target's in-image pixel size is greater than `minTargetArea`
4. The distance from the camera to the target is less than `maxLEDRange`
:::{warning}
Not all network tables objects are updated in simulation. The interaction through PhotonLib remains the same. Actual camera images are also not simulated.
:::
Latency of processing is not yet modeled.
```{image} diagrams/SimArchitecture-deprecated.drawio.svg
:alt: A diagram comparing the architecture of a real PhotonVision process to a simulated
: one.
```
## Simulated Vision System
A `SimVisionSystem` represents the camera and coprocessor running PhotonVision moving around on the field.
It requires a number of pieces of configuration to accurately simulate your physical setup. Match them to your configuration in PhotonVision, and to your robot's physical dimensions.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/sim/DrivetrainSim.java
:language: java
:lines: 73-93
```
After declaring the system, you should create and add one `SimVisionTarget` per target you are attempting to detect.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/sim/DrivetrainSim.java
:language: java
:lines: 95-111
```
Finally, while running the simulation, process simulated camera frames by providing the robot's pose to the system.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/80e16ece87c735e30755dea271a56a2ce217b588/photonlib-java-examples/simaimandrange/src/main/java/frc/robot/sim/DrivetrainSim.java
:language: java
:lines: 138-139
```
This will cause most NetworkTables fields to update properly, representing any targets that are in view of the robot.
Robot software which uses PhotonLib to interact with a camera running PhotonVision should work the same as though a real camera was hooked up and active.
## Raw-Data Approach
Users may wish to directly provide target information based on an existing detailed simulation.
A `SimPhotonCamera` can be created for this purpose. It provides an interface where the user can supply target data via a list of `PhotonTrackedTarget` objects.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
@Override
public void simulationInit() {
// ...
cam = new SimPhotonCamera("MyCamera");
// ...
}
@Override
public void simulationPeriodic() {
// ...
ArrayList<PhotonTrackedTarget> visibleTgtList = new ArrayList<PhotonTrackedTarget>();
visibleTgtList.add(new PhotonTrackedTarget(yawDegrees, pitchDegrees, area, skew, camToTargetTrans)); // Repeat for each target that you see
cam.submitProcessedFrame(0.0, visibleTgtList);
// ...
}
```
Note that while there is less code and configuration required to get basic data into the simulation, this approach will cause the user to need to implement much more code on their end to calculate the relative positions of the robot and target. If you already have this, the raw interface may be helpful. However, if you don't, you'll likely want to be looking at the Simulated Vision System first.

35
docs/source/docs/simulation/simulation.md → docs/source/docs/simulation/simulation-java.md
View File

@@ -1,14 +1,11 @@
# Simulation Support in PhotonLib
# Simulation Support in PhotonLib in Java
:::{attention}
This page details the current simulation support for Java. For other languages, see {doc}`/docs/simulation/simulation-deprecated`.
:::
## What Is Simulated?
Simulation is a powerful tool for validating robot code without access to a physical robot. Read more about [simulation in WPILib](https://docs.wpilib.org/en/stable/docs/software/wpilib-tools/robot-simulation/introduction.html).
PhotonLib can simulate cameras on the field and generate target data approximating what would be seen in reality. This simulation attempts to include the following:
In Java, PhotonLib can simulate cameras on the field and generate target data approximating what would be seen in reality. This simulation attempts to include the following:
- Camera Properties
- Field of Vision
@@ -58,7 +55,7 @@ A `VisionSystemSim` represents the simulated world for one or more cameras, and
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// A vision system sim labelled as "main" in NetworkTables
VisionSystemSim visionSim = new VisionSystemSim("main");
@@ -71,7 +68,7 @@ Vision targets require a `TargetModel`, which describes the shape of the target.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// A 0.5 x 0.25 meter rectangular target
TargetModel targetModel = new TargetModel(0.5, 0.25);
@@ -82,7 +79,7 @@ These `TargetModel` are paired with a target pose to create a `VisionTargetSim`.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// The pose of where the target is on the field.
// Its rotation determines where "forward" or the target x-axis points.
@@ -104,7 +101,7 @@ For convenience, an `AprilTagFieldLayout` can also be added to automatically cre
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// The layout of AprilTags which we want to add to the vision system
AprilTagFieldLayout tagLayout = AprilTagFieldLayout.loadFromResource(AprilTagFields.k2024Crescendo.m_resourceFile);
@@ -125,7 +122,7 @@ Before adding a simulated camera, we need to define its properties. This is done
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// The simulated camera properties
SimCameraProperties cameraProp = new SimCameraProperties();
@@ -136,7 +133,7 @@ By default, this will create a 960 x 720 resolution camera with a 90 degree diag
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// A 640 x 480 camera with a 100 degree diagonal FOV.
cameraProp.setCalibration(640, 480, Rotation2d.fromDegrees(100));
@@ -154,7 +151,7 @@ These properties are used in a `PhotonCameraSim`, which handles generating captu
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// The PhotonCamera used in the real robot code.
PhotonCamera camera = new PhotonCamera("cameraName");
@@ -168,7 +165,7 @@ The `PhotonCameraSim` can now be added to the `VisionSystemSim`. We have to defi
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Our camera is mounted 0.1 meters forward and 0.5 meters up from the robot pose,
// (Robot pose is considered the center of rotation at the floor level, or Z = 0)
@@ -190,7 +187,7 @@ If the camera is mounted on a mobile mechanism (like a turret) this transform ca
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// The turret the camera is mounted on is rotated 5 degrees
Rotation3d turretRotation = new Rotation3d(0, 0, Math.toRadians(5));
@@ -207,7 +204,7 @@ To update the `VisionSystemSim`, we simply have to pass in the simulated robot p
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Update with the simulated drivetrain pose. This should be called every loop in simulation.
visionSim.update(robotPoseMeters);
@@ -222,14 +219,14 @@ Each `VisionSystemSim` has its own built-in `Field2d` for displaying object pose
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Get the built-in Field2d used by this VisionSystemSim
visionSim.getDebugField();
```
:::{figure} images/SimExampleField.png
*A* `VisionSystemSim`*'s internal* `Field2d` *customized with target images and colors, as seen in the* [swervedriveposeestsim](https://github.com/PhotonVision/photonvision/tree/2a6fa1b6ac81f239c59d724da5339f608897c510/photonlib-java-examples/swervedriveposeestsim) *example.*
*A* `VisionSystemSim`*'s internal* `Field2d` *customized with target images and colors*
:::
A `PhotonCameraSim` can also draw and publish generated camera frames to a MJPEG stream similar to an actual PhotonVision process.
@@ -237,7 +234,7 @@ A `PhotonCameraSim` can also draw and publish generated camera frames to a MJPEG
```{eval-rst}
.. tab-set-code::
.. code-block:: java
.. code-block:: Java
// Enable the raw and processed streams. These are enabled by default.
cameraSim.enableRawStream(true);
@@ -251,5 +248,5 @@ A `PhotonCameraSim` can also draw and publish generated camera frames to a MJPEG
These streams follow the port order mentioned in {ref}`docs/installation/networking:Camera Stream Ports`. For example, a single simulated camera will have its raw stream at `localhost:1181` and processed stream at `localhost:1182`, which can also be found in the CameraServer tab of Shuffleboard like a normal camera stream.
:::{figure} images/SimExampleFrame.png
*A frame from the processed stream of a simulated camera viewing some 2023 AprilTags with the field wireframe enabled, as seen in the* [swervedriveposeestsim example](https://github.com/PhotonVision/photonvision/tree/2a6fa1b6ac81f239c59d724da5339f608897c510/photonlib-java-examples/swervedriveposeestsim).
*A frame from the processed stream of a simulated camera viewing some 2023 AprilTags with the field wireframe enabled*
:::

5
docs/source/docs/simulation/simulation-python.md Normal file
View File

@@ -0,0 +1,5 @@
# Simulation Support in PhotonLib in Python
## What Is Supported?
Nothing Yet