Merge branch 'main' into py-docs

This commit is contained in:
Sam Freund
2025-05-08 18:29:37 -05:00
committed by GitHub
230 changed files with 6791 additions and 6964 deletions

View File

@@ -23,30 +23,41 @@ Ensure that your camera is calibrated and 3D mode is enabled. Navigate to the Ou
By default, enabling multi-target will disable calculating camera-to-target transforms for each observed AprilTag target to increase performance; the X/Y/angle numbers shown in the target table of the UI are instead calculated using the tag's expected location (per the field layout JSON) and the field-to-camera transform calculated using MultiTag. If you additionally want the individual camera-to-target transform calculated using SolvePNP for each target, enable "Always Do Single-Target Estimation".
:::
This multi-target pose estimate can be accessed using PhotonLib. We suggest using {ref}`the PhotonPoseEstimator class <docs/programming/photonlib/robot-pose-estimator:AprilTags and PhotonPoseEstimator>` with the `MULTI_TAG_PNP_ON_COPROCESSOR` strategy to simplify code, but the transform can be directly accessed using `getMultiTagResult`/`MultiTagResult()` (Java/C++).
This multi-target pose estimate can be accessed using PhotonLib. We suggest using {ref}`the PhotonPoseEstimator class <docs/programming/photonlib/robot-pose-estimator:AprilTags and PhotonPoseEstimator>` with the `MULTI_TAG_PNP_ON_COPROCESSOR` strategy to simplify code, but the transform can be directly accessed using `getMultiTagResult`/`MultiTagResult()`/`multitagResult` (Java/C++/Python).
```{eval-rst}
.. tab-set-code::
.. code-block:: Java
var result = camera.getLatestResult();
if (result.getMultiTagResult().estimatedPose.isPresent) {
Transform3d fieldToCamera = result.getMultiTagResult().estimatedPose.best;
var results = camera.getAllUnreadResults();
for (var result : results) {
var multiTagResult = result.getMultiTagResult();
if (multiTagResult.isPresent()) {
var fieldToCamera = multiTagResult.get().estimatedPose.best;
}
}
.. code-block:: C++
auto result = camera.GetLatestResult();
if (result.MultiTagResult().result.isPresent) {
frc::Transform3d fieldToCamera = result.MultiTagResult().result.best;
auto results = camera.GetAllUnreadResults();
for (auto &result : results)
{
auto multiTagResult = result.MultiTagResult();
if (multiTagResult.has_value()) {
frc::Transform3d fieldToCamera = multiTagResult->estimatedPose.best;
}
}
.. code-block:: Python
# Coming Soon!
results = camera.getAllUnreadResults()
for result in results:
multitagResult = result.multitagResult
if multitagResult is not None:
fieldToCamera = multitagResult.estimatedPose.best
```
:::{note}

View File

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 34 KiB

View File

Before

Width:  |  Height:  |  Size: 86 KiB

After

Width:  |  Height:  |  Size: 86 KiB

View File

@@ -0,0 +1,8 @@
# Camera-Specifc Configuration
```{toctree}
:maxdepth: 2
arducam-cameras
picamconfig
```

View File

@@ -23,6 +23,7 @@ Windows may report "There is a problem with this drive". This should be ignored.
Locate `config.txt` in the folder, and open it with your favorite text editor.
```{image} images/bootConfigTxt.png
```
Within the file, find this block of text:

View File

@@ -12,17 +12,7 @@ This section contains the build instructions from the source code available at [
**Node JS:**
The UI is written in Node JS. To compile the UI, Node 18.20.4 to Node 20.0.0 is required. To install Node JS follow the instructions for your platform [on the official Node JS website](https://nodejs.org/en/download/). However, modify this line
```bash
nvm install 20
```
so that it instead reads
```javascript
nvm install 18.20.4
```
The UI is written in Node JS. To compile the UI, Node 22.15.0 is required. To install Node JS follow the instructions for your platform [on the official Node JS website](https://nodejs.org/en/download/).
## Compiling Instructions
@@ -283,3 +273,9 @@ Using the [GitHub CLI](https://cli.github.com/), we can download artifacts from
```
~/photonvision$ gh run download 11759699679 -n jar-Linux
```
#### MacOS Builds
MacOS builds are not published to releases as MacOS is not an officially
supported platform. However, MacOS builds are still available from the MacOS
build action, which can be found [here](https://github.com/PhotonVision/photonvision/actions/workflows/build.yml).

View File

@@ -11,7 +11,7 @@ PhotonVision has a myriad of advantages over similar solutions, including:
### Affordable
Compared to alternatives, PhotonVision is much cheaper to use (at the cost of your coprocessor and camera) compared to alternatives that cost \$400. This allows your team to save money while still being competitive.
PhotonVision offers a more affordable solution to vision, with costs being from your coprocessor(s) and camera(s). Teams may choose to run multiple cameras from one coprocessor. This makes it a great solution for teams with limited budgets.
### Easy to Use User Interface

View File

@@ -1,6 +1,6 @@
# Combining Aiming and Getting in Range
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/main/photonlib-java-examples/aimandrange)/[C++](https://github.com/PhotonVision/photonvision/tree/main/photonlib-cpp-examples/aimandrange)).
The following example is from the PhotonLib example repository ([Java](https://github.com/PhotonVision/photonvision/tree/main/photonlib-java-examples/aimandrange)/[C++](https://github.com/PhotonVision/photonvision/tree/main/photonlib-cpp-examples/aimandrange)/[Python](https://github.com/PhotonVision/photonvision/tree/main/photonlib-python-examples/aimandrange))
## Knowledge and Equipment Needed
@@ -10,7 +10,7 @@ The following example is from the PhotonLib example repository ([Java](https://g
Now that you know how to aim toward the AprilTag, let's also drive the correct distance from the AprilTag.
To do this, we'll use the *pitch* of the target in the camera image and trigonometry to figure out how far away the robot is from the AprilTag. Then, like before, we'll use the P term of a PID controller to drive the robot to the correct distance.
To do this, we'll use the _pitch_ of the target in the camera image and trigonometry to figure out how far away the robot is from the AprilTag. Then, like before, we'll use the P term of a PID controller to drive the robot to the correct distance.
```{eval-rst}
.. tab-set::
@@ -43,8 +43,8 @@ To do this, we'll use the *pitch* of the target in the camera image and trigonom
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/abe95dfaa055bbe3609f72cfcaaba0f96ee7978c/photonlib-python-examples/aimandrange/robot.py
:language: python
:lines: 44-95
:lines: 52-91
:linenos:
:lineno-start: 44
:lineno-start: 52
```

View File

@@ -4,6 +4,5 @@
:maxdepth: 2
selecting-hardware
picamconfig
customhardware
```

View File

@@ -47,9 +47,17 @@ Before beginning, it is necessary to install the [rknn-toolkit2](https://github.
PhotonVision currently ONLY supports 640x640 Ultralytics YOLOv5, YOLOv8, and YOLOv11 models trained and converted to `.rknn` format for RK3588 CPUs! Other models require different post-processing code and will NOT work. The model conversion process is also highly particular. Proceed with care.
:::
:::{warning}
Non-quantized models are not supported! If you have the option, make sure quantization is enabled when exporting to .rknn format. This will represent the weights and activations of the model as 8-bit integers, instead of 32-bit floats which PhotonVision doesn't support. Quantized models are also much faster.
:::
In the settings, under `Device Control`, there's an option to upload a new object detection model. Naming convention
should be `name-verticalResolution-horizontalResolution-yolovXXX`. The
`name` should only include alphanumeric characters, periods, and underscores. Additionally, the labels
file ought to have the same name as the RKNN file, with `-labels` appended to the end. For
example, if the RKNN file is named `Algae_1.03.2025-640-640-yolov5s.rknn`, the labels file should be
named `Algae_1.03.2025-640-640-yolov5s-labels.txt`.
:::{note}
Currently there is no way to delete custom models in the GUI, though this is a planned feature. To do this, you have to SSH into the coprocessor and delete the files manually from `/opt/photonvision/photonvision_config/models`.
:::

View File

@@ -8,24 +8,38 @@ PhotonLibPy is a minimal, pure-python implementation of PhotonLib.
## Online Install - Java/C++
Click on the WPI icon on the top right of your VS Code window or hit Ctrl+Shift+P (Cmd+Shift+P on macOS) to bring up the command palette. Type, "Manage Vendor Libraries" and select the "WPILib: Manage Vendor Libraries" option. Then, select the "Install new library (online)" option.
Click on the WPILib logo in the activity bar to access the Vendor Dependencies interface.
```{image} images/adding-offline-library.png
```{image} images/wpilib-vendor-dependencies.png
:scale: 50%
:align: center
:alt: WPILib Vendor Dependencies
```
Paste the following URL into the box that pops up:
Select the install button for the "PhotonLib" dependency.
`https://maven.photonvision.org/repository/internal/org/photonvision/photonlib-json/1.0/photonlib-json-1.0.json`
```{image} images/photonlib-install.png
:scale: 50%
:align: center
:alt: PhotonLib Install Button
```
:::{note}
It is recommended to Build Robot Code at least once when connected to the Internet before heading to an area where Internet connectivity is limited (for example, a competition). This ensures that the relevant files are downloaded to your filesystem.
The Dependency Manager will automatically build your program when it loses focus. This allows you to use the changed dependencies.
:::
When an update is available for PhotonLib, a "To Latest" button will become available. This will update the vendordep to the latest version of PhotonLib.
```{image} images/photonlib-to-latest.png
:align: center
:alt: PhotonLib Update Button
```
Refer to [The WPILib docs](https://docs.wpilib.org/en/stable/docs/software/vscode-overview/3rd-party-libraries.html#installing-libraries) for more details on installing vendor libraries.
## Offline Install - Java/C++
Download the latest PhotonLib release from our GitHub releases page (named something like `` photonlib-VERSION.zip` ``), and extract the contents to `$HOME/wpilib/YEAR`. This adds PhotonLib maven artifacts to your local maven repository. PhotonLib will now also appear available in the "install vendor libraries (offline)" menu in WPILib VSCode. Refer to [the WPILib docs](https://docs.wpilib.org/en/stable/docs/software/vscode-overview/3rd-party-libraries.html#installing-libraries) for more details on installing vendor libraries offline.
Download the latest PhotonLib release from our [GitHub releases page](https://github.com/PhotonVision/photonvision/releases) (named in the format `photonlib-VERSION.zip`), and extract the contents to `~/wpilib/YYYY/vendordeps` (where YYYY is the year and ~ is `C:\Users\Public` on Windows). This adds PhotonLib maven artifacts to your local maven repository. PhotonLib will now also appear available in the "install vendor libraries (offline)" menu in WPILib VSCode. Refer to [the WPILib docs](https://docs.wpilib.org/en/stable/docs/software/vscode-overview/3rd-party-libraries.html#how-does-it-work) for more details on installing vendor libraries offline.
## Install - Python

View File

@@ -86,7 +86,7 @@ Each pipeline result has a `hasTargets()`/`HasTargets()` (Java and C++ respectiv
```
:::{warning}
In Java/C++, You must *always* check if the result has a target via `hasTargets()`/`HasTargets()` before getting targets or else you may get a null pointer exception. Further, you must use the same result in every subsequent call in that loop.
In Java/C++, You must _always_ check if the result has a target via `hasTargets()`/`HasTargets()` before getting targets or else you may get a null pointer exception. Further, you must use the same result in every subsequent call in that loop.
:::
## Getting a List of Targets
@@ -140,7 +140,7 @@ You can get the {ref}`best target <docs/reflectiveAndShape/contour-filtering:Con
## Getting Data From A Target
- double `getYaw()`/`GetYaw()`: The yaw of the target in degrees (positive right).
- double `getYaw()`/`GetYaw()`: The yaw of the target in degrees (positive left).
- double `getPitch()`/`GetPitch()`: The pitch of the target in degrees (positive up).
- double `getArea()`/`GetArea()`: The area (how much of the camera feed the bounding box takes up) as a percent (0-100).
- double `getSkew()`/`GetSkew()`: The skew of the target in degrees (counter-clockwise positive).

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

View File

@@ -4,30 +4,46 @@
For more information on how to methods to get AprilTag data, look {ref}`here <docs/programming/photonlib/getting-target-data:Getting AprilTag Data From A Target>`.
:::
PhotonLib includes a `PhotonPoseEstimator` class, which allows you to combine the pose data from all tags in view in order to get a field relative pose. The `PhotonPoseEstimator` class works with one camera per object instance, but more than one instance may be created.
PhotonLib includes a `PhotonPoseEstimator` class, which allows you to combine the pose data from all tags in view in order to get a field relative pose. For each camera, a separate instance of the `PhotonPoseEstimator` class should be created.
## Creating an `AprilTagFieldLayout`
`AprilTagFieldLayout` is used to represent a layout of AprilTags within a space (field, shop at home, classroom, etc.). WPILib provides a JSON that describes the layout of AprilTags on the field which you can then use in the AprilTagFieldLayout constructor. You can also specify a custom layout.
The API documentation can be found in here: [Java](https://github.wpilib.org/allwpilib/docs/release/java/edu/wpi/first/apriltag/AprilTagFieldLayout.html) and [C++](https://github.wpilib.org/allwpilib/docs/release/cpp/classfrc_1_1_april_tag_field_layout.html).
The API documentation can be found in here: [Java](https://github.wpilib.org/allwpilib/docs/release/java/edu/wpi/first/apriltag/AprilTagFieldLayout.html), [C++](https://github.wpilib.org/allwpilib/docs/release/cpp/classfrc_1_1_april_tag_field_layout.html), and [Python](https://robotpy.readthedocs.io/projects/apriltag/en/stable/robotpy_apriltag/AprilTagFieldLayout.html#robotpy_apriltag.AprilTagFieldLayout).
```{eval-rst}
.. tab-set-code::
.. code-block:: Java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-java-examples/poseest/src/main/java/frc/robot/Constants.java
:language: java
:lines: 48-49
// The field from AprilTagFields will be different depending on the game.
AprilTagFieldLayout aprilTagFieldLayout = AprilTagFieldLayout.loadField(AprilTagFields.kDefaultField);
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-cpp-examples/poseest/src/main/include/Constants.h
:language: c++
:lines: 46-47
.. code-block:: C++
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 46
```
// The parameter for LoadAPrilTagLayoutField will be different depending on the game.
frc::AprilTagFieldLayout aprilTagFieldLayout = frc::LoadAprilTagLayoutField(frc::AprilTagField::kDefaultField);
## Defining the Robot to Camera `Transform3d`
.. code-block:: Python
Another necessary argument for creating a `PhotonPoseEstimator` is the `Transform3d` representing the robot-relative location and orientation of the camera. A `Transform3d` contains a `Translation3d` and a `Rotation3d`. The `Translation3d` is created in meters and the `Rotation3d` is created with radians. For more information on the coordinate system, please see the {ref}`Coordinate Systems <docs/apriltag-pipelines/coordinate-systems:Coordinate Systems>` documentation.
# Coming Soon!
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-java-examples/poseest/src/main/java/frc/robot/Constants.java
:language: java
:lines: 44-45
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-cpp-examples/poseest/src/main/include/Constants.h
:language: c++
:lines: 43-45
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 33-36
```
## Creating a `PhotonPoseEstimator`
@@ -35,127 +51,116 @@ The API documentation can be found in here: [Java](https://github.wpilib.org/all
The PhotonPoseEstimator has a constructor that takes an `AprilTagFieldLayout` (see above), `PoseStrategy`, `PhotonCamera`, and `Transform3d`. `PoseStrategy` has nine possible values:
- MULTI_TAG_PNP_ON_COPROCESSOR
- Calculates a new robot position estimate by combining all visible tag corners. Recommended for all teams as it will be the most accurate.
- Must configure the AprilTagFieldLayout properly in the UI, please see {ref}`here <docs/apriltag-pipelines/multitag:multitag localization>` for more information.
- Calculates a new robot position estimate by combining all visible tag corners. Recommended for all teams as it will be the most accurate.
- Must configure the AprilTagFieldLayout properly in the UI, please see {ref}`here <docs/apriltag-pipelines/multitag:multitag localization>` for more information.
- LOWEST_AMBIGUITY
- Choose the Pose with the lowest ambiguity.
- Choose the Pose with the lowest ambiguity.
- CLOSEST_TO_CAMERA_HEIGHT
- Choose the Pose which is closest to the camera height.
- Choose the Pose which is closest to the camera height.
- CLOSEST_TO_REFERENCE_POSE
- Choose the Pose which is closest to the pose from setReferencePose().
- Choose the Pose which is closest to the pose from setReferencePose().
- CLOSEST_TO_LAST_POSE
- Choose the Pose which is closest to the last pose calculated.
- Choose the Pose which is closest to the last pose calculated.
- AVERAGE_BEST_TARGETS
- Choose the Pose which is the average of all the poses from each tag.
- Choose the Pose which is the average of all the poses from each tag.
- MULTI_TAG_PNP_ON_RIO
- A slower, older version of MULTI_TAG_PNP_ON_COPROCESSOR, not recommended for use.
- A slower, older version of MULTI_TAG_PNP_ON_COPROCESSOR, not recommended for use.
- PNP_DISTANCE_TRIG_SOLVE
- Use distance data from best visible tag to compute a Pose. This runs on the RoboRIO in order
to access the robot's yaw heading, and MUST have addHeadingData called every frame so heading
data is up-to-date. Based on a reference implementation by [FRC Team 6328 Mechanical Advantage](https://www.chiefdelphi.com/t/frc-6328-mechanical-advantage-2025-build-thread/477314/98).
- Use distance data from best visible tag to compute a Pose. This runs on the RoboRIO in order
to access the robot's yaw heading, and MUST have addHeadingData called every frame so heading
data is up-to-date. Based on a reference implementation by [FRC Team 6328 Mechanical Advantage](https://www.chiefdelphi.com/t/frc-6328-mechanical-advantage-2025-build-thread/477314/98).
- CONSTRAINED_SOLVEPNP
- Solve a constrained version of the Perspective-n-Point problem with the robot's drivebase
flat on the floor. This computation takes place on the RoboRIO, and should not take more than 2ms.
This also requires addHeadingData to be called every frame so heading data is up to date.
If Multi-Tag PNP is enabled on the coprocessor, it will be used to provide an initial seed to
the optimization algorithm -- otherwise, the multi-tag fallback strategy will be used as the
seed.
- Solve a constrained version of the Perspective-n-Point problem with the robot's drivebase
flat on the floor. This computation takes place on the RoboRIO, and should not take more than 2ms.
This also requires addHeadingData to be called every frame so heading data is up to date.
If Multi-Tag PNP is enabled on the coprocessor, it will be used to provide an initial seed to
the optimization algorithm -- otherwise, the multi-tag fallback strategy will be used as the
seed.
```{eval-rst}
.. tab-set-code::
.. code-block:: Java
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-java-examples/poseest/src/main/java/frc/robot/Vision.java
:language: java
:lines: 65-66
//Forward Camera
cam = new PhotonCamera("testCamera");
Transform3d robotToCam = new Transform3d(new Translation3d(0.5, 0.0, 0.5), new Rotation3d(0,0,0)); //Cam mounted facing forward, half a meter forward of center, half a meter up from center.
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-cpp-examples/poseest/src/main/include/Vision.h
:language: c++
:lines: 150-153
// Construct PhotonPoseEstimator
PhotonPoseEstimator photonPoseEstimator = new PhotonPoseEstimator(aprilTagFieldLayout, PoseStrategy.CLOSEST_TO_REFERENCE_POSE, cam, robotToCam);
.. code-block:: C++
// Forward Camera
std::shared_ptr<photonlib::PhotonCamera> cameraOne =
std::make_shared<photonlib::PhotonCamera>("testCamera");
// Camera is mounted facing forward, half a meter forward of center, half a
// meter up from center.
frc::Transform3d robotToCam =
frc::Transform3d(frc::Translation3d(0.5_m, 0_m, 0.5_m),
frc::Rotation3d(0_rad, 0_rad, 0_rad));
// ... Add other cameras here
// Assemble the list of cameras & mount locations
std::vector<
std::pair<std::shared_ptr<photonlib::PhotonCamera>, frc::Transform3d>>
cameras;
cameras.push_back(std::make_pair(cameraOne, robotToCam));
photonlib::RobotPoseEstimator estimator(
aprilTags, photonlib::CLOSEST_TO_REFERENCE_POSE, cameras);
.. code-block:: Python
kRobotToCam = wpimath.geometry.Transform3d(
wpimath.geometry.Translation3d(0.5, 0.0, 0.5),
wpimath.geometry.Rotation3d.fromDegrees(0.0, -30.0, 0.0),
)
self.cam = PhotonCamera("YOUR CAMERA NAME")
self.camPoseEst = PhotonPoseEstimator(
loadAprilTagLayoutField(AprilTagField.kDefaultField),
PoseStrategy.CLOSEST_TO_REFERENCE_POSE,
self.cam,
kRobotToCam
)
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 45-50
```
:::{note}
Python still takes a `PhotonCamera` in the constructor, so you must create the camera as shown in the next section and then return and use it to create the `PhotonPoseEstimator`.
:::
## Using a `PhotonPoseEstimator`
Calling `update()` on your `PhotonPoseEstimator` will return an `EstimatedRobotPose`, which includes a `Pose3d` of the latest estimated pose (using the selected strategy) along with a `double` of the timestamp when the robot pose was estimated. You should be updating your [drivetrain pose estimator](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html) with the result from the `PhotonPoseEstimator` every loop using `addVisionMeasurement()`.
The final prerequisite to using your `PhotonPoseEstimator` is creating a `PhotonCamera`. To do this, you must set the name of your camera in Photon Client. From there you can define the camera in code.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/357d8a518a93f7a1f8084a79449249e613b605a7/photonlib-java-examples/apriltagExample/src/main/java/frc/robot/PhotonCameraWrapper.java
:language: java
:lines: 85-88
.. code-block:: C++
std::pair<frc::Pose2d, units::millisecond_t> getEstimatedGlobalPose(
frc::Pose3d prevEstimatedRobotPose) {
robotPoseEstimator.SetReferencePose(prevEstimatedRobotPose);
units::millisecond_t currentTime = frc::Timer::GetFPGATimestamp();
auto result = robotPoseEstimator.Update();
if (result.second) {
return std::make_pair<>(result.first.ToPose2d(),
currentTime - result.second);
} else {
return std::make_pair(frc::Pose2d(), 0_ms);
}
}
.. code-block:: Python
# Coming Soon!
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-java-examples/poseest/src/main/java/frc/robot/Vision.java
:language: java
:lines: 63
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-cpp-examples/aimattarget/src/main/include/Robot.h
:language: c++
:lines: 55
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 44
```
You should be updating your [drivetrain pose estimator](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html) with the result from the `RobotPoseEstimator` every loop using `addVisionMeasurement()`. TODO: add example note
Calling `update()` on your `PhotonPoseEstimator` will return an `EstimatedRobotPose`, which includes a `Pose3d` of the latest estimated pose (using the selected strategy) along with a `double` of the timestamp when the robot pose was estimated.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-java-examples/poseest/src/main/java/frc/robot/Vision.java
:language: java
:lines: 93-116
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-cpp-examples/poseest/src/main/include/Vision.h
:language: c++
:lines: 80-100
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 53
```
You should be updating your [drivetrain pose estimator](https://docs.wpilib.org/en/latest/docs/software/advanced-controls/state-space/state-space-pose-estimators.html) with the result from the `PhotonPoseEstimator` every loop using `addVisionMeasurement()`.
```{eval-rst}
.. tab-set-code::
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-java-examples/poseest/src/main/java/frc/robot/Robot.java
:language: java
:lines: 49
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-cpp-examples/poseest/src/main/include/Robot.h
:language: c++
:lines: 54-57
.. rli:: https://raw.githubusercontent.com/PhotonVision/photonvision/refs/heads/main/photonlib-python-examples/poseest/robot.py
:language: python
:lines: 54-57
```
## Complete Examples
The complete examples for the `PhotonPoseEstimator` can be found in the following locations:
- [Java](https://github.com/PhotonVision/photonvision/tree/main/photonlib-java-examples/poseest)
- [C++](https://github.com/PhotonVision/photonvision/tree/main/photonlib-cpp-examples/poseest)
- [Python](https://github.com/PhotonVision/photonvision/tree/main/photonlib-python-examples/poseest)
## Additional `PhotonPoseEstimator` Methods
### `setReferencePose(Pose3d referencePose)`
For more information on the `PhotonPoseEstimator` class, please see the API documentation.
Updates the stored reference pose when using the CLOSEST_TO_REFERENCE_POSE strategy.
### `setLastPose(Pose3d lastPose)`
Update the stored last pose. Useful for setting the initial estimate when using the CLOSEST_TO_LAST_POSE strategy.
### `addHeadingData(double timestampSeconds, Rotation2d heading)`
Adds robot heading data to be stored in buffer. Must be called periodically with a proper timestamp for the PNP_DISTANCE_TRIG_SOLVE and CONSTRAINED_SOLVEPNP strategies
- [Java Documentation](https://javadocs.photonvision.org/release/org/photonvision/PhotonPoseEstimator.html)
- [C++ Documentation](https://cppdocs.photonvision.org/release/classphoton_1_1_photon_pose_estimator.html)
- [Python Documentation](https://pydocs.photonvision.org/release/reference/photonPoseEstimator/)

View File

@@ -0,0 +1,41 @@
# Camera Matching
## Activating and Deactivating Cameras
When you first plug in a camera, it will be detected and added to the list of cameras with the "Unassigned" status, as shown below. You can press the "Activate" button to enable PhotonVision to use the camera.
```{image} images/camera-matching/unassigned-camera.png
:scale: 50%
```
If a camera has been activated in the past, it will be listed as "Deactivated" in the camera list. You can press the "Activate" button to enable PhotonVision to use the camera.
```{image} images/camera-matching/deactivated-camera.png
:scale: 50%
```
Once a camera is activated, it will be listed as "Active" in the camera list. You can press the "Deactivate" button to stop PhotonVision from using the camera.
```{image} images/camera-matching/activated-camera.png
:scale: 50%
```
## Deleting Cameras
If you want to remove a camera from the list, you can press the delete button. This will clear all settings for that particular camera, including the calibration data and any other settings you have configured. It is recommended to make a backup of the camera's settings before deleting it, as this action cannot be undone.
## Matching Cameras
When you plug in a camera, PhotonVision will attempt to match it to a previously configured camera based on the physical USB port it is connected to. If you plug another camera into that port, the cameras will have a "Camera Mismatch" status, indicating that the camera is not recognized as the one that was previously configured.
Additionally, pressing on the Details button will show you the details of the camera mismatch, allowing you to compare the current camera with the previously configured camera.
```{image} images/camera-matching/camera-mismatch-details.png
:scale: 50%
```
```{note}
Camera matching is based on the USB ports on the device. If you unplug a camera and plug it into a different port, PhotonVision will attempt to use settings from the camera that was previously configured in that port, causing unexpected behavior.
```
To resolve the camera mismatch, you should ensure each camera is plugged into the same port that you configured it in.

View File

@@ -34,7 +34,7 @@ Innomaker and Arducam are common manufacturers of hardware designed specifically
- Driver Camera
- OV9281
- OV9782
- Pi Camera Module V1 {ref}`(More setup info)<docs/hardware/picamconfig:Pi Camera Configuration>`
- Pi Camera Module V1 {ref}`(More setup info)<docs/camera-specific-configuration/picamconfig:Pi Camera Configuration>`
Feel free to get started with any color webcam you have sitting around.

Binary file not shown.

After

Width:  |  Height:  |  Size: 287 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 119 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 212 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 83 KiB

View File

@@ -7,7 +7,7 @@ common-setups
quick-install
wiring
networking
arducam-cameras
camera-matching
camera-calibration
quick-configure
```

View File

@@ -54,13 +54,13 @@ Only use a static IP when connected to the **robot radio**, and never when testi
5. Change your IP to Static.
6. Set your coprocessor's IP address to “10.TE.AM.11”. More information on IP format can be found [here](https://docs.wpilib.org/en/stable/docs/networking/networking-introduction/ip-configurations.html#on-the-field-static-configuration).
7. Click the “Save” button.
8. Set your roboRIO to the following static IP address: “10.TE.AM.2”. This can be done via the [roboRIO web dashboard](https://docs.wpilib.org/en/stable/docs/software/roborio-info/roborio-web-dashboard.html#roborio-web-dashboard).
Power-cycle your robot and then you will now be access the PhotonVision dashboard at `10.TE.AM.11:5800`.
```{image} images/static.png
:alt: Correctly set static IP
```
The "team number" field will accept (in addition to a team number) an IP address or hostname. This is useful for testing PhotonVision on the same computer as a simulated robot program;
you can set the team number to "localhost", and PhotonVision will send data to the network tables in the simulated robot.

View File

@@ -78,7 +78,11 @@ This diagram shows how to use the recommended regulator to power a coprocessor.
Pigtails can be purchased from many sources we recommend [(USB C)](https://ctr-electronics.com/products/usb-type-c-wire-breakout?_pos=19&_sid=bf06b6a6b&_ss=r) [(Micro USB)](https://ctr-electronics.com/products/usb-micro-power-wire-breakout?pr_prod_strat=e5_desc&pr_rec_id=10bf36ce7&pr_rec_pid=7863771070637&pr_ref_pid=7863771103405&pr_seq=uniform)
## Coprocessor with Passive POE (Pi with SnakeEyes and Limelight)
## Limelight
Follow the wiring instructions located in the [Limelight Documentation](https://docs.limelightvision.io/) for your Limelight model.
## Coprocessor with Passive POE (Pi with SnakeEyes)
1. Plug the [passive POE injector](https://www.revrobotics.com/rev-11-1210/) into the coprocessor and wire it to PDP/PDH (NOT the VRM).
2. Add a breaker to relevant slot in your PDP/PDH

View File

@@ -2,7 +2,7 @@
## Pi Cameras
If you haven't yet, please refer to {ref}`the Pi CSI Camera Configuration page <docs/hardware/picamconfig:Pi Camera Configuration>` for information on updating {code}`config.txt` for your use case. If you've tried that, and things still aren't working, restart PhotonVision using the restart button in the settings tab, and press tilde (\`) in the web UI once connection is restored. This should show the most recent boot log.
If you haven't yet, please refer to {ref}`the Pi CSI Camera Configuration page <docs/camera-specific-configuration/picamconfig:Pi Camera Configuration>` for information on updating {code}`config.txt` for your use case. If you've tried that, and things still aren't working, restart PhotonVision using the restart button in the settings tab, and press tilde (\`) in the web UI once connection is restored. This should show the most recent boot log.
| | Expected output | Bad |
| ------------------------------- | ----------------------------------------------------- | ---------------------------------- |
@@ -12,7 +12,7 @@ If you haven't yet, please refer to {ref}`the Pi CSI Camera Configuration page <
If the driver isn't loaded, you may be using a non-official Pi image, or an image not new enough. Try updating to the most recent image available (one released for 2023) -- if that doesn't resolve the problem, {ref}`contact us<index:Contact Us>` with your settings ZIP file and Pi version/camera version/config.txt file used.
If the camera is not detected, the most likely cause is either a config.txt file incorrectly set-up, or a ribbon cable attached backwards. Review the {ref}`picam configuration page <docs/hardware/picamconfig:pi camera configuration>`, and verify the ribbon cable is properly oriented at both ends, and that it is \_fully\_ inserted into the FFC connector. Then, {ref}`contact us<index:Contact Us>` with your settings ZIP file and Pi version/camera version/config.txt file used.
If the camera is not detected, the most likely cause is either a config.txt file incorrectly set-up, or a ribbon cable attached backwards. Review the {ref}`picam configuration page <docs/camera-specific-configuration/picamconfig:pi camera configuration>`, and verify the ribbon cable is properly oriented at both ends, and that it is _fully_ inserted into the FFC connector. Then, {ref}`contact us<index:Contact Us>` with your settings ZIP file and Pi version/camera version/config.txt file used.
## USB cameras
@@ -27,7 +27,7 @@ USB cameras supported by CSCore require no libcamera driver initialization to wo
On Linux devices (including Raspberry Pi), PhotonVision uses WPILib's CSCore to interact with video devices, which internally uses Video4Linux (v4l2). CSCore, and therefore Photon, requires that cameras attached have good v4l drivers for proper functionality. These should be built into the Linux kernel, and do not need to be installed manually. Valid picamera setup (from /boot/config.txt) can also be determined using these steps. The list-devices command will show all valid video devices detected, and list-formats the list of "video modes" each camera can be in.
- For picams: edit the config.txt file as described in the {ref}`picam configuration page <docs/hardware/picamconfig:pi camera configuration>`
- For picams: edit the config.txt file as described in the {ref}`picam configuration page <docs/camera-specific-configuration/picamconfig:pi camera configuration>`
- SSH into your Pi: {code}`ssh pi@photonvision.local` and enter the username "pi" & password "raspberry"
- run {code}`v4l2-ctl --list-devices` and {code}`v4l2-ctl --list-formats`

View File

@@ -7,4 +7,5 @@ common-errors
logging
camera-troubleshooting
networking-troubleshooting
unix-commands
```

View File

@@ -20,3 +20,21 @@ Logs are stored inside the {code}`photonvision_config/logs` directory. Exporting
Your browser does not support the video tag.
</video>
```
Robot mode transitions are also recorded in program logs. These transition messages look something like the two shown below, and show the contents of the [HAL Control Word](https://github.wpilib.org/allwpilib/docs/release/java/edu/wpi/first/hal/ControlWord.html) that the robot was in previously, and what it is now in. This includes:
- Enabled state
- Robot state (autonomous vs teleoperated)
- If the robot e-stop is active
If the robot is connected to the FMS at an event, we will additionally print out:
- Event name
- Match type and number
- Driver station position
```
[2025-04-19 19:52:08] [NetworkTables - NTDriverStation] [INFO] ROBOT TRANSITIONED MODES! From NtControlWord[m_enabled=true, m_autonomous=false, m_test=false, m_emergencyStop=false, m_fmsAttached=true, m_dsAttached=true] to NtControlWord[m_enabled=true, m_autonomous=false, m_test=true, m_emergencyStop=false, m_fmsAttached=true, m_dsAttached=true]
[2025-04-19 19:52:09] [NetworkTables - NTDriverStation] [INFO] ROBOT TRANSITIONED MODES! From NtControlWord[m_enabled=true, m_autonomous=false, m_test=true, m_emergencyStop=false, m_fmsAttached=true, m_dsAttached=true] to NtControlWord[m_enabled=false, m_autonomous=false, m_test=false, m_emergencyStop=false, m_fmsAttached=false, m_dsAttached=false]
[2025-04-19 19:52:19] [NetworkTables - NTDriverStation] [INFO] ROBOT TRANSITIONED MODES! From NtControlWord[m_enabled=false, m_autonomous=false, m_test=false, m_emergencyStop=false, m_fmsAttached=false, m_dsAttached=false] to NtControlWord[m_enabled=true, m_autonomous=true, m_test=false, m_emergencyStop=false, m_fmsAttached=true, m_dsAttached=true]
```

View File

@@ -0,0 +1,132 @@
# Useful Unix Commands
## Networking
### SSH
[SSH (Secure Shell)](https://www.mankier.com/1/ssh) is used to securely connect from a local to a remote system (ex. from a laptop to a coprocessor). Unlike other commands on this page, ssh is not Unix specific and can be done on Windows and MacOS from their respective terminals.
:::{note}
You may see a warning similar to `The authenticity of host 'xxx' can't be established...` or `WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!`, in most cases this can be safely ignored if you have confirmed that you are connecting to the correct host over a secure connection, and the fingerprint will change when your operating system is reinstalled or PhotonVision's coprocessor image is re-flashed. This can also occur if you have multiple coprocessors with the same hostname on your network. You can read more about it [here](https://superuser.com/questions/421997/what-is-a-ssh-key-fingerprint-and-how-is-it-generated)
:::
Example:
```
ssh pi@hostname
```
For PhotonVision, the username will be `pi` and the password will be `raspberry`.
### ip
Run [ip address](https://www.mankier.com/8/ip) with your coprocessor connected to a monitor in order to see its IP address and other network configuration information.
Your output might look something like this:
```
2: end1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether de:9a:8f:7d:31:aa brd ff:ff:ff:ff:ff:ff
inet 10.88.47.12/24 brd 10.88.47.255 scope global dynamic noprefixroute end1
valid_lft 27367sec preferred_lft 27367sec
```
In this example, the numbers following `inet` (10.88.47.12) are your IP address.
### ping
[ping](https://www.mankier.com/8/ping) is a command-line utility used to test the reachability of a host on an IP network. It also measures the round-trip time for messages sent from the originating host to a destination computer. It can be used to determine if a network interface is available, which can be helpful when debugging.
## File Transfer
All files under `/opt/photonvision` are owned by the root user. This means that if you want to modify them, the commands to do so must be ran as sudo.
### SCP
[SCP (Secure Copy)](https://www.mankier.com/1/scp) is used to securely transfer files between local and remote systems.
Example:
```
scp [file] pi@hostname:/path/to/destination
```
### SFTP
[SFTP (SSH File Transfer Protocol)](https://www.mankier.com/1/sftp#) is another option for transferring files between local and remote systems.
### Filezilla
[Filezilla](https://filezilla-project.org/) is a GUI alternative to SCP and SFTP. It is available for Windows, MacOS, and Linux.
## Miscellaneous
### v4l2-ctl
[v4l2-ctl](https://www.mankier.com/1/v4l2-ctl) is a command-line tool for controlling video devices.
List available video devices (used to verify the device recognized a connected camera):
```
v4l2-ctl --list-devices
```
List supported formats and resolutions for a specific video device:
```
v4l2-ctl --list-formats-ext --device /path/to/video_device
```
List all video device's controls and their values:
```
v4l2-ctl --list-ctrls --device path/to/video_device
```
:::{note}
This command is especially useful in helping to debug when certain camera controls, like exposure, aren't behaving as expected. If you see an error in the logs similar to `WARNING 30: failed to set property [property name] (UsbCameraImpl.cpp:646)`, that means that PhotonVision is trying to use a control that doesn't exist or has a different name on your hardware. If you encounter this issue, please [file an issue](https://github.com/PhotonVision/photonvision/issues) with the necessary logs and output of the `v4l2-ctl --list-ctrls` command.
:::
### systemctl
[systemctl](https://www.mankier.com/1/systemctl) is a command that controls the `systemd` system and service manager.
Start PhotonVision:
```
systemctl start photonvision
```
Stop PhotonVision:
```
systemctl stop photonvision
```
Restart PhotonVision:
```
systemctl restart photonvision
```
Check the status of PhotonVision:
```
systemctl status photonvision
```
### journalctl
[journalctl](https://www.mankier.com/1/journalctl) is a command that queries the systemd journal, which is a logging system used by many Linux distributions.
View the PhotonVision logs:
```
journalctl -u photonvision
```
View the PhotonVision logs in real-time:
```
journalctl -u photonvision -f
```

View File

@@ -91,6 +91,7 @@ docs/description
docs/quick-start/index
docs/hardware/index
docs/advanced-installation/index
docs/camera-specific-configuration/index
```
```{toctree}