Team2890/PhotonVision
4
0
Fork 0
mirror of https://github.com/PhotonVision/photonvision synced 2026-06-21 01:01:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

Switch from RST to MyST Markdown (#1395)

Browse Source
This commit is contained in:
Mohammad Durrani
2024-08-18 14:05:23 -04:00
committed by GitHub
parent 0493ef9133
commit 4643f86438
124 changed files with 2821 additions and 2757 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docs/source/docs/programming/index.md Normal file
View File

@@ -0,0 +1,11 @@
---
orphan: true
---
# Programming Reference
```{toctree}
:maxdepth: 1
photonlib/index
```

9
docs/source/docs/programming/index.rst
View File

@@ -1,9 +0,0 @@
:orphan:
Programming Reference
=====================
.. toctree::
:maxdepth: 1
photonlib/index

48
docs/source/docs/programming/photonlib/adding-vendordep.md Normal file
View File

@@ -0,0 +1,48 @@
# Installing PhotonLib
## What is PhotonLib?
PhotonLib is the C++ and Java vendor dependency that accompanies PhotonVision. We created this vendor dependency to make it easier for teams to retrieve vision data from their integrated vision system.
PhotonLibPy is a minimal, pure-python implementation of PhotonLib.
## Online Install - Java/C++
Click on the WPI icon on the top right of your VS Code window or hit Ctrl+Shift+P (Cmd+Shift+P on macOS) to bring up the command palette. Type, "Manage Vendor Libraries" and select the "WPILib: Manage Vendor Libraries" option. Then, select the "Install new library (online)" option.
```{image} images/adding-offline-library.png
```
Paste the following URL into the box that pops up:
`https://maven.photonvision.org/repository/internal/org/photonvision/photonlib-json/1.0/photonlib-json-1.0.json`
:::{note}
It is recommended to Build Robot Code at least once when connected to the Internet before heading to an area where Internet connectivity is limited (for example, a competition). This ensures that the relevant files are downloaded to your filesystem.
:::
Refer to [The WPILib docs](https://docs.wpilib.org/en/stable/docs/software/vscode-overview/3rd-party-libraries.html#installing-libraries) for more details on installing vendor libraries.
## Offline Install - Java/C++
Download the latest PhotonLib release from our GitHub releases page (named something like `` photonlib-VERSION.zip` ``), and extract the contents to `$HOME/wpilib/YEAR`. This adds PhotonLib maven artifacts to your local maven repository. PhotonLib will now also appear available in the "install vendor libraries (offline)" menu in WPILib VSCode. Refer to [the WPILib docs](https://docs.wpilib.org/en/stable/docs/software/vscode-overview/3rd-party-libraries.html#installing-libraries) for more details on installing vendor libraries offline.
## Install - Python
Add photonlibpy to `pyproject.toml`.
```toml
# Other pip packages to install
requires = [
"photonlibpy",
]
```
See [The WPILib/RobotPy docs](https://docs.wpilib.org/en/stable/docs/software/python/pyproject_toml.html) for more information on using `pyproject.toml.`
## Install Specific Version - Java/C++
In cases where you want to test a specific version of PhotonLib, make sure you have finished the steps in Online Install - Java/C++ and then manually change the version string in the PhotonLib vendordep json file(at ``/path/to/your/project/vendordep/photonlib.json``) to your desired version.
```{image} images/photonlib-vendordep-json.png
```

46
docs/source/docs/programming/photonlib/adding-vendordep.rst
View File

@@ -1,46 +0,0 @@
Installing PhotonLib
====================
What is PhotonLib?
------------------
PhotonLib is the C++ and Java vendor dependency that accompanies PhotonVision. We created this vendor dependency to make it easier for teams to retrieve vision data from their integrated vision system.
PhotonLibPy is a minimal, pure-python implementation of PhotonLib.
Online Install - Java/C++
-------------------------
Click on the WPI icon on the top right of your VS Code window or hit Ctrl+Shift+P (Cmd+Shift+P on macOS) to bring up the command palette. Type, "Manage Vendor Libraries" and select the "WPILib: Manage Vendor Libraries" option. Then, select the "Install new library (online)" option.
.. image:: images/adding-offline-library.png
Paste the following URL into the box that pops up:
``https://maven.photonvision.org/repository/internal/org/photonvision/photonlib-json/1.0/photonlib-json-1.0.json``
.. note:: It is recommended to Build Robot Code at least once when connected to the Internet before heading to an area where Internet connectivity is limited (for example, a competition). This ensures that the relevant files are downloaded to your filesystem.
Refer to `The WPILib docs <https://docs.wpilib.org/en/stable/docs/software/vscode-overview/3rd-party-libraries.html#installing-libraries>` for more details on installing vendor libraries.
Offline Install - Java/C++
--------------------------
Download the latest photonlib release from our GitHub releases page (named something like `photonlib-VERSION.zip``), and extract the contents to `$HOME/wpilib/YEAR`. This adds PhotonLib maven artifacts to your local maven repository. PhotonLib will now also appear available in the "install vendor libraries (offline)" menu in WPILib VSCode. Refer to `The WPILib docs <https://docs.wpilib.org/en/stable/docs/software/vscode-overview/3rd-party-libraries.html#installing-libraries>` for more details on installing vendor libraries offline.
Install - Python
----------------
Add photonlibpy to `pyproject.toml`.
.. code-block:: toml
# Other pip packages to install
requires = [
"photonlibpy",
]
See `The WPILib/RobotPy docs <https://docs.wpilib.org/en/stable/docs/software/python/pyproject_toml.html>`_ for more information on using `pyproject.toml.`
Install Specific Version - Java/C++
-----------------------------------
In cases where you want to test a specific version of PhotonLib, make sure you have finished the steps in Online Install - Java/C++ and then manually change the version string in the photonlib vendordep json file(at ``/path/to/your/project/vendordep/photonlib.json``) to your desired version.
.. image:: images/photonlib-vendordep-json.png

16
docs/source/docs/programming/photonlib/controlling-led.md Normal file
View File

@@ -0,0 +1,16 @@
# Controlling LEDs
You can control the vision LEDs of supported hardware via PhotonLib using the `setLED()` method on a `PhotonCamera` instance. In Java and C++, an `VisionLEDMode` enum class is provided to choose values from. These values include, `kOff`, `kOn`, `kBlink`, and `kDefault`. `kDefault` uses the default LED value from the selected pipeline.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// Blink the LEDs.
camera.setLED(VisionLEDMode.kBlink);
.. code-block:: c++
// Blink the LEDs.
camera.SetLED(photonlib::VisionLEDMode::kBlink);
```

14
docs/source/docs/programming/photonlib/controlling-led.rst
View File

@@ -1,14 +0,0 @@
Controlling LEDs
=================
You can control the vision LEDs of supported hardware via PhotonLib using the ``setLED()`` method on a ``PhotonCamera`` instance. In Java and C++, an ``VisionLEDMode`` enum class is provided to choose values from. These values include, ``kOff``, ``kOn``, ``kBlink``, and ``kDefault``. ``kDefault`` uses the default LED value from the selected pipeline.
.. tab-set-code::
.. code-block:: java
// Blink the LEDs.
camera.setLED(VisionLEDMode.kBlink);
.. code-block:: c++
// Blink the LEDs.
camera.SetLED(photonlib::VisionLEDMode::kBlink);

60
docs/source/docs/programming/photonlib/driver-mode-pipeline-index.md Normal file
View File

@@ -0,0 +1,60 @@
# Driver Mode and Pipeline Index/Latency
After {ref}`creating a PhotonCamera <docs/programming/photonlib/getting-target-data:Constructing a PhotonCamera>`, one can toggle Driver Mode and change the Pipeline Index of the vision program from robot code.
## Toggle Driver Mode
You can use the `setDriverMode()`/`SetDriverMode()` (Java and C++ respectively) to toggle driver mode from your robot program. Driver mode is an unfiltered / normal view of the camera to be used while driving the robot.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// Set driver mode to on.
camera.setDriverMode(true);
.. code-block:: C++
// Set driver mode to on.
camera.SetDriverMode(true);
```
## Setting the Pipeline Index
You can use the `setPipelineIndex()`/`SetPipelineIndex()` (Java and C++ respectively) to dynamically change the vision pipeline from your robot program.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// Change pipeline to 2
camera.setPipelineIndex(2);
.. code-block:: C++
// Change pipeline to 2
camera.SetPipelineIndex(2);
```
## Getting the Pipeline Latency
You can also get the pipeline latency from a pipeline result using the `getLatencyMillis()`/`GetLatency()` (Java and C++ respectively) methods on a `PhotonPipelineResult`.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// Get the pipeline latency.
double latencySeconds = result.getLatencyMillis() / 1000.0;
.. code-block:: c++
// Get the pipeline latency.
units::second_t latency = result.GetLatency();
```
:::{note}
The C++ version of PhotonLib returns the latency in a unit container. For more information on the Units library, see [here](https://docs.wpilib.org/en/stable/docs/software/basic-programming/cpp-units.html).
:::

53
docs/source/docs/programming/photonlib/driver-mode-pipeline-index.rst
View File

@@ -1,53 +0,0 @@
Driver Mode and Pipeline Index/Latency
======================================
After :ref:`creating a PhotonCamera <docs/programming/photonlib/getting-target-data:Constructing a PhotonCamera>`, one can toggle Driver Mode and change the Pipeline Index of the vision program from robot code.
Toggle Driver Mode
------------------
You can use the ``setDriverMode()``/``SetDriverMode()`` (Java and C++ respectively) to toggle driver mode from your robot program. Driver mode is an unfiltered / normal view of the camera to be used while driving the robot.
.. tab-set-code::
.. code-block:: java
// Set driver mode to on.
camera.setDriverMode(true);
.. code-block:: C++
// Set driver mode to on.
camera.SetDriverMode(true);
Setting the Pipeline Index
--------------------------
You can use the ``setPipelineIndex()``/``SetPipelineIndex()`` (Java and C++ respectively) to dynamically change the vision pipeline from your robot program.
.. tab-set-code::
.. code-block:: java
// Change pipeline to 2
camera.setPipelineIndex(2);
.. code-block:: C++
// Change pipeline to 2
camera.SetPipelineIndex(2);
Getting the Pipeline Latency
----------------------------
You can also get the pipeline latency from a pipeline result using the ``getLatencyMillis()``/``GetLatency()`` (Java and C++ respectively) methods on a ``PhotonPipelineResult``.
.. tab-set-code::
.. code-block:: java
// Get the pipeline latency.
double latencySeconds = result.getLatencyMillis() / 1000.0;
.. code-block:: c++
// Get the pipeline latency.
units::second_t latency = result.GetLatency();
.. note:: The C++ version of PhotonLib returns the latency in a unit container. For more information on the Units library, see `here <https://docs.wpilib.org/en/stable/docs/software/basic-programming/cpp-units.html>`_.

114
docs/source/docs/programming/photonlib/getting-target-data.rst → docs/source/docs/programming/photonlib/getting-target-data.md
View File

@@ -1,16 +1,14 @@
Getting Target Data
===================
# Getting Target Data
Constructing a PhotonCamera
---------------------------
## Constructing a PhotonCamera
What is a PhotonCamera?
^^^^^^^^^^^^^^^^^^^^^^^
``PhotonCamera`` is a class in PhotonLib that allows a user to interact with one camera that is connected to hardware that is running PhotonVision. Through this class, users can retrieve yaw, pitch, roll, robot-relative pose, latency, and a wealth of other information.
### What is a PhotonCamera?
`PhotonCamera` is a class in PhotonLib that allows a user to interact with one camera that is connected to hardware that is running PhotonVision. Through this class, users can retrieve yaw, pitch, roll, robot-relative pose, latency, and a wealth of other information.
The ``PhotonCamera`` class has two constructors: one that takes a ``NetworkTable`` and another that takes in the name of the network table that PhotonVision is broadcasting information over. For ease of use, it is recommended to use the latter. The name of the NetworkTable (for the string constructor) should be the same as the camera's nickname (from the PhotonVision UI).
The `PhotonCamera` class has two constructors: one that takes a `NetworkTable` and another that takes in the name of the network table that PhotonVision is broadcasting information over. For ease of use, it is recommended to use the latter. The name of the NetworkTable (for the string constructor) should be the same as the camera's nickname (from the PhotonVision UI).
```{eval-rst}
.. tab-set-code::
@@ -27,18 +25,21 @@ The ``PhotonCamera`` class has two constructors: one that takes a ``NetworkTable
# Change this to match the name of your camera as shown in the web ui
self.camera = PhotonCamera("your_camera_name_here")
```
.. warning:: Teams must have unique names for all of their cameras regardless of which coprocessor they are attached to.
:::{warning}
Teams must have unique names for all of their cameras regardless of which coprocessor they are attached to.
:::
Getting the Pipeline Result
---------------------------
## Getting the Pipeline Result
What is a Photon Pipeline Result?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A ``PhotonPipelineResult`` is a container that contains all information about currently detected targets from a ``PhotonCamera``. You can retrieve the latest pipeline result using the PhotonCamera instance.
### What is a Photon Pipeline Result?
Use the ``getLatestResult()``/``GetLatestResult()`` (Java and C++ respectively) to obtain the latest pipeline result. An advantage of using this method is that it returns a container with information that is guaranteed to be from the same timestamp. This is important if you are using this data for latency compensation or in an estimator.
A `PhotonPipelineResult` is a container that contains all information about currently detected targets from a `PhotonCamera`. You can retrieve the latest pipeline result using the PhotonCamera instance.
Use the `getLatestResult()`/`GetLatestResult()` (Java and C++ respectively) to obtain the latest pipeline result. An advantage of using this method is that it returns a container with information that is guaranteed to be from the same timestamp. This is important if you are using this data for latency compensation or in an estimator.
```{eval-rst}
.. tab-set-code::
@@ -56,15 +57,17 @@ Use the ``getLatestResult()``/``GetLatestResult()`` (Java and C++ respectively)
result = self.camera.getLatestResult()
```
.. note:: Unlike other vision software solutions, using the latest result guarantees that all information is from the same timestamp. This is achievable because the PhotonVision backend sends a byte-packed string of data which is then deserialized by PhotonLib to get target data. For more information, check out the `PhotonLib source code <https://github.com/PhotonVision/photonvision/tree/master/photon-lib>`_.
:::{note}
Unlike other vision software solutions, using the latest result guarantees that all information is from the same timestamp. This is achievable because the PhotonVision backend sends a byte-packed string of data which is then deserialized by PhotonLib to get target data. For more information, check out the [PhotonLib source code](https://github.com/PhotonVision/photonvision/tree/master/photon-lib).
:::
## Checking for Existence of Targets
Each pipeline result has a `hasTargets()`/`HasTargets()` (Java and C++ respectively) method to inform the user as to whether the result contains any targets.
Checking for Existence of Targets
---------------------------------
Each pipeline result has a ``hasTargets()``/``HasTargets()`` (Java and C++ respectively) method to inform the user as to whether the result contains any targets.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
@@ -80,20 +83,21 @@ Each pipeline result has a ``hasTargets()``/``HasTargets()`` (Java and C++ respe
# Check if the latest result has any targets.
hasTargets = result.hasTargets()
```
.. warning:: In Java/C++, You must *always* check if the result has a target via ``hasTargets()``/``HasTargets()`` before getting targets or else you may get a null pointer exception. Further, you must use the same result in every subsequent call in that loop.
:::{warning}
In Java/C++, You must *always* check if the result has a target via `hasTargets()`/`HasTargets()` before getting targets or else you may get a null pointer exception. Further, you must use the same result in every subsequent call in that loop.
:::
## Getting a List of Targets
Getting a List of Targets
-------------------------
### What is a Photon Tracked Target?
What is a Photon Tracked Target?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A tracked target contains information about each target from a pipeline result. This information includes yaw, pitch, area, and robot relative pose.
You can get a list of tracked targets using the `getTargets()`/`GetTargets()` (Java and C++ respectively) method from a pipeline result.
You can get a list of tracked targets using the ``getTargets()``/``GetTargets()`` (Java and C++ respectively) method from a pipeline result.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
@@ -109,11 +113,13 @@ You can get a list of tracked targets using the ``getTargets()``/``GetTargets()`
# Get a list of currently tracked targets.
targets = result.getTargets()
```
Getting the Best Target
-----------------------
You can get the :ref:`best target <docs/reflectiveAndShape/contour-filtering:Contour Grouping and Sorting>` using ``getBestTarget()``/``GetBestTarget()`` (Java and C++ respectively) method from the pipeline result.
## Getting the Best Target
You can get the {ref}`best target <docs/reflectiveAndShape/contour-filtering:Contour Grouping and Sorting>` using `getBestTarget()`/`GetBestTarget()` (Java and C++ respectively) method from the pipeline result.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
@@ -130,17 +136,18 @@ You can get the :ref:`best target <docs/reflectiveAndShape/contour-filtering:Con
# TODO - Not currently supported
```
Getting Data From A Target
--------------------------
* double ``getYaw()``/``GetYaw()``: The yaw of the target in degrees (positive right).
* double ``getPitch()``/``GetPitch()``: The pitch of the target in degrees (positive up).
* double ``getArea()``/``GetArea()``: The area (how much of the camera feed the bounding box takes up) as a percent (0-100).
* double ``getSkew()``/``GetSkew()``: The skew of the target in degrees (counter-clockwise positive).
* double[] ``getCorners()``/``GetCorners()``: The 4 corners of the minimum bounding box rectangle.
* Transform2d ``getCameraToTarget()``/``GetCameraToTarget()``: The camera to target transform. See `2d transform documentation here <https://docs.wpilib.org/en/latest/docs/software/advanced-controls/geometry/transformations.html#transform2d-and-twist2d>`_.
## Getting Data From A Target
- double `getYaw()`/`GetYaw()`: The yaw of the target in degrees (positive right).
- double `getPitch()`/`GetPitch()`: The pitch of the target in degrees (positive up).
- double `getArea()`/`GetArea()`: The area (how much of the camera feed the bounding box takes up) as a percent (0-100).
- double `getSkew()`/`GetSkew()`: The skew of the target in degrees (counter-clockwise positive).
- double\[\] `getCorners()`/`GetCorners()`: The 4 corners of the minimum bounding box rectangle.
- Transform2d `getCameraToTarget()`/`GetCameraToTarget()`: The camera to target transform. See [2d transform documentation here](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/geometry/transformations.html#transform2d-and-twist2d).
```{eval-rst}
.. tab-set-code::
.. code-block:: java
@@ -171,16 +178,20 @@ Getting Data From A Target
skew = target.getSkew()
pose = target.getCameraToTarget()
corners = target.getDetectedCorners()
```
Getting AprilTag Data From A Target
-----------------------------------
.. note:: All of the data above (**except skew**) is available when using AprilTags.
## Getting AprilTag Data From A Target
* int ``getFiducialId()``/``GetFiducialId()``: The ID of the detected fiducial marker.
* double ``getPoseAmbiguity()``/``GetPoseAmbiguity()``: How ambiguous the pose of the target is (see below).
* Transform3d ``getBestCameraToTarget()``/``GetBestCameraToTarget()``: Get the transform that maps camera space (X = forward, Y = left, Z = up) to object/fiducial tag space (X forward, Y left, Z up) with the lowest reprojection error.
* Transform3d ``getAlternateCameraToTarget()``/``GetAlternateCameraToTarget()``: Get the transform that maps camera space (X = forward, Y = left, Z = up) to object/fiducial tag space (X forward, Y left, Z up) with the highest reprojection error.
:::{note}
All of the data above (**except skew**) is available when using AprilTags.
:::
- int `getFiducialId()`/`GetFiducialId()`: The ID of the detected fiducial marker.
- double `getPoseAmbiguity()`/`GetPoseAmbiguity()`: How ambiguous the pose of the target is (see below).
- Transform3d `getBestCameraToTarget()`/`GetBestCameraToTarget()`: Get the transform that maps camera space (X = forward, Y = left, Z = up) to object/fiducial tag space (X forward, Y left, Z up) with the lowest reprojection error.
- Transform3d `getAlternateCameraToTarget()`/`GetAlternateCameraToTarget()`: Get the transform that maps camera space (X = forward, Y = left, Z = up) to object/fiducial tag space (X forward, Y left, Z up) with the highest reprojection error.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
@@ -205,13 +216,15 @@ Getting AprilTag Data From A Target
poseAmbiguity = target.getPoseAmbiguity()
bestCameraToTarget = target.getBestCameraToTarget()
alternateCameraToTarget = target.getAlternateCameraToTarget()
```
Saving Pictures to File
-----------------------
A ``PhotonCamera`` can save still images from the input or output video streams to file. This is useful for debugging what a camera is seeing while on the field and confirming targets are being identified properly.
## Saving Pictures to File
A `PhotonCamera` can save still images from the input or output video streams to file. This is useful for debugging what a camera is seeing while on the field and confirming targets are being identified properly.
Images are stored within the PhotonVision configuration directory. Running the "Export" operation in the settings tab will download a .zip file which contains the image captures.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
@@ -237,5 +250,8 @@ Images are stored within the PhotonVision configuration directory. Running the "
# Capture post-process camera stream image
camera.takeOutputSnapshot()
```
.. note:: Saving images to file takes a bit of time and uses up disk space, so doing it frequently is not recommended. In general, the camera will save an image every 500ms. Calling these methods faster will not result in additional images. Consider tying image captures to a button press on the driver controller, or an appropriate point in an autonomous routine.
:::{note}
Saving images to file takes a bit of time and uses up disk space, so doing it frequently is not recommended. In general, the camera will save an image every 500ms. Calling these methods faster will not result in additional images. Consider tying image captures to a button press on the driver controller, or an appropriate point in an autonomous routine.
:::

12
docs/source/docs/programming/photonlib/index.md Normal file
View File

@@ -0,0 +1,12 @@
# PhotonLib: Robot Code Interface
```{toctree}
:maxdepth: 1
adding-vendordep
getting-target-data
using-target-data
robot-pose-estimator
driver-mode-pipeline-index
controlling-led
```

12
docs/source/docs/programming/photonlib/index.rst
View File

@@ -1,12 +0,0 @@
PhotonLib: Robot Code Interface
===============================
.. toctree::
:maxdepth: 1
adding-vendordep
getting-target-data
using-target-data
robot-pose-estimator
driver-mode-pipeline-index
controlling-led

117
docs/source/docs/programming/photonlib/robot-pose-estimator.md Normal file
View File

@@ -0,0 +1,117 @@
# AprilTags and PhotonPoseEstimator
:::{note}
For more information on how to methods to get AprilTag data, look {ref}`here <docs/programming/photonlib/getting-target-data:Getting AprilTag Data From A Target>`.
:::
PhotonLib includes a `PhotonPoseEstimator` class, which allows you to combine the pose data from all tags in view in order to get a field relative pose. The `PhotonPoseEstimator` class works with one camera per object instance, but more than one instance may be created.
## Creating an `AprilTagFieldLayout`
`AprilTagFieldLayout` is used to represent a layout of AprilTags within a space (field, shop at home, classroom, etc.). WPILib provides a JSON that describes the layout of AprilTags on the field which you can then use in the AprilTagFieldLayout constructor. You can also specify a custom layout.
The API documentation can be found in here: [Java](https://github.wpilib.org/allwpilib/docs/release/java/edu/wpi/first/apriltag/AprilTagFieldLayout.html) and [C++](https://github.wpilib.org/allwpilib/docs/release/cpp/classfrc_1_1_april_tag_field_layout.html).
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// The field from AprilTagFields will be different depending on the game.
AprilTagFieldLayout aprilTagFieldLayout = AprilTagFields.k2024Crescendo.loadAprilTagLayoutField();
.. code-block:: c++
// The parameter for LoadAPrilTagLayoutField will be different depending on the game.
frc::AprilTagFieldLayout aprilTagFieldLayout = frc::LoadAprilTagLayoutField(frc::AprilTagField::k2024Crescendo);
```
## Creating a `PhotonPoseEstimator`
The PhotonPoseEstimator has a constructor that takes an `AprilTagFieldLayout` (see above), `PoseStrategy`, `PhotonCamera`, and `Transform3d`. `PoseStrategy` has six possible values:
- MULTI_TAG_PNP_ON_COPROCESSOR
- Calculates a new robot position estimate by combining all visible tag corners. Recommended for all teams as it will be the most accurate.
- Must configure the AprilTagFieldLayout properly in the UI, please see {ref}`here <docs/apriltag-pipelines/multitag:multitag localization>` for more information.
- LOWEST_AMBIGUITY
- Choose the Pose with the lowest ambiguity.
- CLOSEST_TO_CAMERA_HEIGHT
- Choose the Pose which is closest to the camera height.
- CLOSEST_TO_REFERENCE_POSE
- Choose the Pose which is closest to the pose from setReferencePose().
- CLOSEST_TO_LAST_POSE
- Choose the Pose which is closest to the last pose calculated.
- AVERAGE_BEST_TARGETS
- Choose the Pose which is the average of all the poses from each tag.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
//Forward Camera
cam = new PhotonCamera("testCamera");
Transform3d robotToCam = new Transform3d(new Translation3d(0.5, 0.0, 0.5), new Rotation3d(0,0,0)); //Cam mounted facing forward, half a meter forward of center, half a meter up from center.
// Construct PhotonPoseEstimator
PhotonPoseEstimator photonPoseEstimator = new PhotonPoseEstimator(aprilTagFieldLayout, PoseStrategy.CLOSEST_TO_REFERENCE_POSE, cam, robotToCam);
.. code-block:: c++
// Forward Camera
std::shared_ptr<photonlib::PhotonCamera> cameraOne =
std::make_shared<photonlib::PhotonCamera>("testCamera");
// Camera is mounted facing forward, half a meter forward of center, half a
// meter up from center.
frc::Transform3d robotToCam =
frc::Transform3d(frc::Translation3d(0.5_m, 0_m, 0.5_m),
frc::Rotation3d(0_rad, 0_rad, 0_rad));
// ... Add other cameras here
// Assemble the list of cameras & mount locations
std::vector<
std::pair<std::shared_ptr<photonlib::PhotonCamera>, frc::Transform3d>>
cameras;
cameras.push_back(std::make_pair(cameraOne, robotToCam));
photonlib::RobotPoseEstimator estimator(
aprilTags, photonlib::CLOSEST_TO_REFERENCE_POSE, cameras);
```
## Using a `PhotonPoseEstimator`
Calling `update()` on your `PhotonPoseEstimator` will return an `EstimatedRobotPose`, which includes a `Pose3d` of the latest estimated pose (using the selected strategy) along with a `double` of the timestamp when the robot pose was estimated. You should be updating your [drivetrain pose estimator](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html) with the result from the `PhotonPoseEstimator` every loop using `addVisionMeasurement()`.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/357d8a518a93f7a1f8084a79449249e613b605a7/photonlib-java-examples/apriltagExample/src/main/java/frc/robot/PhotonCameraWrapper.java
:language: java
:lines: 85-88
.. code-block:: c++
std::pair<frc::Pose2d, units::millisecond_t> getEstimatedGlobalPose(
frc::Pose3d prevEstimatedRobotPose) {
robotPoseEstimator.SetReferencePose(prevEstimatedRobotPose);
units::millisecond_t currentTime = frc::Timer::GetFPGATimestamp();
auto result = robotPoseEstimator.Update();
if (result.second) {
return std::make_pair<>(result.first.ToPose2d(),
currentTime - result.second);
} else {
return std::make_pair(frc::Pose2d(), 0_ms);
}
}
```
You should be updating your [drivetrain pose estimator](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html) with the result from the `RobotPoseEstimator` every loop using `addVisionMeasurement()`. TODO: add example note
## Additional `PhotonPoseEstimator` Methods
### `setReferencePose(Pose3d referencePose)`
Updates the stored reference pose when using the CLOSEST_TO_REFERENCE_POSE strategy.
### `setLastPose(Pose3d lastPose)`
Update the stored last pose. Useful for setting the initial estimate when using the CLOSEST_TO_LAST_POSE strategy.

113
docs/source/docs/programming/photonlib/robot-pose-estimator.rst
View File

@@ -1,113 +0,0 @@
AprilTags and PhotonPoseEstimator
=================================
.. note:: For more information on how to methods to get AprilTag data, look :ref:`here <docs/programming/photonlib/getting-target-data:Getting AprilTag Data From A Target>`.
PhotonLib includes a ``PhotonPoseEstimator`` class, which allows you to combine the pose data from all tags in view in order to get a field relative pose. The ``PhotonPoseEstimator`` class works with one camera per object instance, but more than one instance may be created.
Creating an ``AprilTagFieldLayout``
-----------------------------------
``AprilTagFieldLayout`` is used to represent a layout of AprilTags within a space (field, shop at home, classroom, etc.). WPILib provides a JSON that describes the layout of AprilTags on the field which you can then use in the AprilTagFieldLayout constructor. You can also specify a custom layout.
The API documentation can be found in here: `Java <https://github.wpilib.org/allwpilib/docs/release/java/edu/wpi/first/apriltag/AprilTagFieldLayout.html>`_ and `C++ <https://github.wpilib.org/allwpilib/docs/release/cpp/classfrc_1_1_april_tag_field_layout.html>`_.
.. tab-set-code::
.. code-block:: java
// The field from AprilTagFields will be different depending on the game.
AprilTagFieldLayout aprilTagFieldLayout = AprilTagFields.k2024Crescendo.loadAprilTagLayoutField();
.. code-block:: c++
// The parameter for LoadAPrilTagLayoutField will be different depending on the game.
frc::AprilTagFieldLayout aprilTagFieldLayout = frc::LoadAprilTagLayoutField(frc::AprilTagField::k2024Crescendo);
Creating a ``PhotonPoseEstimator``
----------------------------------
The PhotonPoseEstimator has a constructor that takes an ``AprilTagFieldLayout`` (see above), ``PoseStrategy``, ``PhotonCamera``, and ``Transform3d``. ``PoseStrategy`` has six possible values:
* MULTI_TAG_PNP_ON_COPROCESSOR
* Calculates a new robot position estimate by combining all visible tag corners. Recommended for all teams as it will be the most accurate.
* Must configure the AprilTagFieldLayout properly in the UI, please see :ref:`here <docs/apriltag-pipelines/multitag:multitag localization>` for more information.
* LOWEST_AMBIGUITY
* Choose the Pose with the lowest ambiguity.
* CLOSEST_TO_CAMERA_HEIGHT
* Choose the Pose which is closest to the camera height.
* CLOSEST_TO_REFERENCE_POSE
* Choose the Pose which is closest to the pose from setReferencePose().
* CLOSEST_TO_LAST_POSE
* Choose the Pose which is closest to the last pose calculated.
* AVERAGE_BEST_TARGETS
* Choose the Pose which is the average of all the poses from each tag.
.. tab-set-code::
.. code-block:: java
//Forward Camera
cam = new PhotonCamera("testCamera");
Transform3d robotToCam = new Transform3d(new Translation3d(0.5, 0.0, 0.5), new Rotation3d(0,0,0)); //Cam mounted facing forward, half a meter forward of center, half a meter up from center.
// Construct PhotonPoseEstimator
PhotonPoseEstimator photonPoseEstimator = new PhotonPoseEstimator(aprilTagFieldLayout, PoseStrategy.CLOSEST_TO_REFERENCE_POSE, cam, robotToCam);
.. code-block:: c++
// Forward Camera
std::shared_ptr<photonlib::PhotonCamera> cameraOne =
std::make_shared<photonlib::PhotonCamera>("testCamera");
// Camera is mounted facing forward, half a meter forward of center, half a
// meter up from center.
frc::Transform3d robotToCam =
frc::Transform3d(frc::Translation3d(0.5_m, 0_m, 0.5_m),
frc::Rotation3d(0_rad, 0_rad, 0_rad));
// ... Add other cameras here
// Assemble the list of cameras & mount locations
std::vector<
std::pair<std::shared_ptr<photonlib::PhotonCamera>, frc::Transform3d>>
cameras;
cameras.push_back(std::make_pair(cameraOne, robotToCam));
photonlib::RobotPoseEstimator estimator(
aprilTags, photonlib::CLOSEST_TO_REFERENCE_POSE, cameras);
Using a ``PhotonPoseEstimator``
-------------------------------
Calling ``update()`` on your ``PhotonPoseEstimator`` will return an ``EstimatedRobotPose``, which includes a ``Pose3d`` of the latest estimated pose (using the selected strategy) along with a ``double`` of the timestamp when the robot pose was estimated. You should be updating your `drivetrain pose estimator <https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html>`_ with the result from the ``PhotonPoseEstimator`` every loop using ``addVisionMeasurement()``.
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/357d8a518a93f7a1f8084a79449249e613b605a7/photonlib-java-examples/apriltagExample/src/main/java/frc/robot/PhotonCameraWrapper.java
:language: java
:lines: 85-88
.. code-block:: c++
std::pair<frc::Pose2d, units::millisecond_t> getEstimatedGlobalPose(
frc::Pose3d prevEstimatedRobotPose) {
robotPoseEstimator.SetReferencePose(prevEstimatedRobotPose);
units::millisecond_t currentTime = frc::Timer::GetFPGATimestamp();
auto result = robotPoseEstimator.Update();
if (result.second) {
return std::make_pair<>(result.first.ToPose2d(),
currentTime - result.second);
} else {
return std::make_pair(frc::Pose2d(), 0_ms);
}
}
You should be updating your `drivetrain pose estimator <https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html>`_ with the result from the ``RobotPoseEstimator`` every loop using ``addVisionMeasurement()``. TODO: add example note
Additional ``PhotonPoseEstimator`` Methods
------------------------------------------
``setReferencePose(Pose3d referencePose)``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Updates the stored reference pose when using the CLOSEST_TO_REFERENCE_POSE strategy.
``setLastPose(Pose3d lastPose)``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Update the stored last pose. Useful for setting the initial estimate when using the CLOSEST_TO_LAST_POSE strategy.

111
docs/source/docs/programming/photonlib/using-target-data.md Normal file
View File

@@ -0,0 +1,111 @@
# Using Target Data
A `PhotonUtils` class with helpful common calculations is included within `PhotonLib` to aid teams in using target data in order to get positional information on the field. This class contains two methods, `calculateDistanceToTargetMeters()`/`CalculateDistanceToTarget()` and `estimateTargetTranslation2d()`/`EstimateTargetTranslation()` (Java and C++ respectively).
## Estimating Field Relative Pose with AprilTags
`estimateFieldToRobotAprilTag(Transform3d cameraToTarget, Pose3d fieldRelativeTagPose, Transform3d cameraToRobot)` returns your robot's `Pose3d` on the field using the pose of the AprilTag relative to the camera, pose of the AprilTag relative to the field, and the transform from the camera to the origin of the robot.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// Calculate robot's field relative pose
Pose3d robotPose = PhotonUtils.estimateFieldToRobotAprilTag(target.getBestCameraToTarget(), aprilTagFieldLayout.getTagPose(target.getFiducialId()), cameraToRobot);
.. code-block:: c++
//TODO
```
## Estimating Field Relative Pose (Traditional)
You can get your robot's `Pose2D` on the field using various camera data, target yaw, gyro angle, target pose, and camera position. This method estimates the target's relative position using `estimateCameraToTargetTranslation` (which uses pitch and yaw to estimate range and heading), and the robot's gyro to estimate the rotation of the target.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// Calculate robot's field relative pose
Pose2D robotPose = PhotonUtils.estimateFieldToRobot(
kCameraHeight, kTargetHeight, kCameraPitch, kTargetPitch, Rotation2d.fromDegrees(-target.getYaw()), gyro.getRotation2d(), targetPose, cameraToRobot);
.. code-block:: c++
// Calculate robot's field relative pose
frc::Pose2D robotPose = photonlib::EstimateFieldToRobot(
kCameraHeight, kTargetHeight, kCameraPitch, kTargetPitch, frc::Rotation2d(units::degree_t(-target.GetYaw())), frc::Rotation2d(units::degree_t(gyro.GetRotation2d)), targetPose, cameraToRobot);
```
## Calculating Distance to Target
If your camera is at a fixed height on your robot and the height of the target is fixed, you can calculate the distance to the target based on your camera's pitch and the pitch to the target.
```{eval-rst}
.. tab-set-code::
.. rli:: https://github.com/PhotonVision/photonvision/raw/a3bcd3ac4f88acd4665371abc3073bdbe5effea8/photonlib-java-examples/src/main/java/org/photonlib/examples/getinrange/Robot.java
:language: java
:lines: 78-94
.. rli:: https://github.com/PhotonVision/photonvision/raw/a3bcd3ac4f88acd4665371abc3073bdbe5effea8/photonlib-cpp-examples/src/main/cpp/examples/getinrange/cpp/Robot.cpp
:language: cpp
:lines: 33-46
```
:::{note}
The C++ version of PhotonLib uses the Units library. For more information, see [here](https://docs.wpilib.org/en/stable/docs/software/basic-programming/cpp-units.html).
:::
## Calculating Distance Between Two Poses
`getDistanceToPose(Pose2d robotPose, Pose2d targetPose)` allows you to calculate the distance between two poses. This is useful when using AprilTags, given that there may not be an AprilTag directly on the target.
```{eval-rst}
.. tab-set-code::
.. code-block:: java
double distanceToTarget = PhotonUtils.getDistanceToPose(robotPose, targetPose);
.. code-block:: c++
//TODO
```
## Estimating Camera Translation to Target
You can get a [translation](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/geometry/pose.html#translation) to the target based on the distance to the target (calculated above) and angle to the target (yaw).
```{eval-rst}
.. tab-set-code::
.. code-block:: java
// Calculate a translation from the camera to the target.
Translation2d translation = PhotonUtils.estimateCameraToTargetTranslation(
distanceMeters, Rotation2d.fromDegrees(-target.getYaw()));
.. code-block:: c++
// Calculate a translation from the camera to the target.
frc::Translation2d translation = photonlib::PhotonUtils::EstimateCameraToTargetTranslationn(
distance, frc::Rotation2d(units::degree_t(-target.GetYaw())));
```
:::{note}
We are negating the yaw from the camera from CV (computer vision) conventions to standard mathematical conventions. In standard mathematical conventions, as you turn counter-clockwise, angles become more positive.
:::
## Getting the Yaw To a Pose
`getYawToPose(Pose2d robotPose, Pose2d targetPose)` returns the `Rotation2d` between your robot and a target. This is useful when turning towards an arbitrary target on the field (ex. the center of the hub in 2022).
```{eval-rst}
.. tab-set-code::
.. code-block:: java
Rotation2d targetYaw = PhotonUtils.getYawToPose(robotPose, targetPose);
.. code-block:: c++
//TODO
```

97
docs/source/docs/programming/photonlib/using-target-data.rst
View File

@@ -1,97 +0,0 @@
Using Target Data
=================
A ``PhotonUtils`` class with helpful common calculations is included within ``PhotonLib`` to aid teams in using target data in order to get positional information on the field. This class contains two methods, ``calculateDistanceToTargetMeters()``/``CalculateDistanceToTarget()`` and ``estimateTargetTranslation2d()``/``EstimateTargetTranslation()`` (Java and C++ respectively).
Estimating Field Relative Pose with AprilTags
---------------------------------------------
``estimateFieldToRobotAprilTag(Transform3d cameraToTarget, Pose3d fieldRelativeTagPose, Transform3d cameraToRobot)`` returns your robot's ``Pose3d`` on the field using the pose of the AprilTag relative to the camera, pose of the AprilTag relative to the field, and the transform from the camera to the origin of the robot.
.. tab-set-code::
.. code-block:: java
// Calculate robot's field relative pose
Pose3d robotPose = PhotonUtils.estimateFieldToRobotAprilTag(target.getBestCameraToTarget(), aprilTagFieldLayout.getTagPose(target.getFiducialId()), cameraToRobot);
.. code-block:: c++
//TODO
Estimating Field Relative Pose (Traditional)
--------------------------------------------
You can get your robot's ``Pose2D`` on the field using various camera data, target yaw, gyro angle, target pose, and camera position. This method estimates the target's relative position using ``estimateCameraToTargetTranslation`` (which uses pitch and yaw to estimate range and heading), and the robot's gyro to estimate the rotation of the target.
.. tab-set-code::
.. code-block:: java
// Calculate robot's field relative pose
Pose2D robotPose = PhotonUtils.estimateFieldToRobot(
kCameraHeight, kTargetHeight, kCameraPitch, kTargetPitch, Rotation2d.fromDegrees(-target.getYaw()), gyro.getRotation2d(), targetPose, cameraToRobot);
.. code-block:: c++
// Calculate robot's field relative pose
frc::Pose2D robotPose = photonlib::EstimateFieldToRobot(
kCameraHeight, kTargetHeight, kCameraPitch, kTargetPitch, frc::Rotation2d(units::degree_t(-target.GetYaw())), frc::Rotation2d(units::degree_t(gyro.GetRotation2d)), targetPose, cameraToRobot);
Calculating Distance to Target
------------------------------
If your camera is at a fixed height on your robot and the height of the target is fixed, you can calculate the distance to the target based on your camera's pitch and the pitch to the target.
.. tab-set-code::
.. rli:: https://github.com/PhotonVision/photonvision/raw/a3bcd3ac4f88acd4665371abc3073bdbe5effea8/photonlib-java-examples/src/main/java/org/photonlib/examples/getinrange/Robot.java
:language: java
:lines: 78-94
.. rli:: https://github.com/PhotonVision/photonvision/raw/a3bcd3ac4f88acd4665371abc3073bdbe5effea8/photonlib-cpp-examples/src/main/cpp/examples/getinrange/cpp/Robot.cpp
:language: cpp
:lines: 33-46
.. note:: The C++ version of PhotonLib uses the Units library. For more information, see `here <https://docs.wpilib.org/en/stable/docs/software/basic-programming/cpp-units.html>`_.
Calculating Distance Between Two Poses
--------------------------------------
``getDistanceToPose(Pose2d robotPose, Pose2d targetPose)`` allows you to calculate the distance between two poses. This is useful when using AprilTags, given that there may not be an AprilTag directly on the target.
.. tab-set-code::
.. code-block:: java
double distanceToTarget = PhotonUtils.getDistanceToPose(robotPose, targetPose);
.. code-block:: c++
//TODO
Estimating Camera Translation to Target
---------------------------------------
You can get a `translation <https://docs.wpilib.org/en/latest/docs/software/advanced-controls/geometry/pose.html#translation>`_ to the target based on the distance to the target (calculated above) and angle to the target (yaw).
.. tab-set-code::
.. code-block:: java
// Calculate a translation from the camera to the target.
Translation2d translation = PhotonUtils.estimateCameraToTargetTranslation(
distanceMeters, Rotation2d.fromDegrees(-target.getYaw()));
.. code-block:: c++
// Calculate a translation from the camera to the target.
frc::Translation2d translation = photonlib::PhotonUtils::EstimateCameraToTargetTranslationn(
distance, frc::Rotation2d(units::degree_t(-target.GetYaw())));
.. note:: We are negating the yaw from the camera from CV (computer vision) conventions to standard mathematical conventions. In standard mathematical conventions, as you turn counter-clockwise, angles become more positive.
Getting the Yaw To a Pose
-------------------------
``getYawToPose(Pose2d robotPose, Pose2d targetPose)`` returns the ``Rotation2d`` between your robot and a target. This is useful when turning towards an arbitrary target on the field (ex. the center of the hub in 2022).
.. tab-set-code::
.. code-block:: java
Rotation2d targetYaw = PhotonUtils.getYawToPose(robotPose, targetPose);
.. code-block:: c++
//TODO