mirror of
https://github.com/PhotonVision/photonvision
synced 2026-06-20 00:51:41 +00:00
Switch from RST to MyST Markdown (#1395)
This commit is contained in:
@@ -1,43 +1,41 @@
|
||||
Advanced Strategies
|
||||
===================
|
||||
# Advanced Strategies
|
||||
|
||||
Advanced strategies for using vision processing results involve working with the robot's *pose* on the field. A *pose* is a combination an X/Y coordinate, and an angle describing where the robot's front is pointed. It is always considered *relative* to some fixed point on the field.
|
||||
|
||||
WPILib provides a `Pose2d <https://docs.wpilib.org/en/stable/docs/software/advanced-controls/geometry/pose.html>`_ class to describe poses in software.
|
||||
WPILib provides a [Pose2d](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/geometry/pose.html) class to describe poses in software.
|
||||
|
||||
Knowledge and Equipment Needed
|
||||
------------------------------
|
||||
## Knowledge and Equipment Needed
|
||||
|
||||
- A Coprocessor running PhotonVision
|
||||
- Accurate camera calibration to support "3D mode" required
|
||||
\- Accurate camera calibration to support "3D mode" required
|
||||
- A Drivetrain with wheels and sensors
|
||||
- Sufficient sensors to measure wheel rotation
|
||||
- Capable of closed-loop velocity control
|
||||
\- Sufficient sensors to measure wheel rotation
|
||||
\- Capable of closed-loop velocity control
|
||||
- A gyroscope or IMU measuring actual robot heading
|
||||
- Experience using some path-planning library (WPILib is our recommendation)
|
||||
|
||||
Path Planning in a Target-Centered Reference Frame
|
||||
--------------------------------------------------
|
||||
## Path Planning in a Target-Centered Reference Frame
|
||||
|
||||
When using 3D mode in PhotonVision, the `SolvePNP Algorithm <https://en.wikipedia.org/wiki/Perspective-n-Point>`_ is used to deduce the *camera\'s* position in a 3D coordinate system centered on the target itself.
|
||||
When using 3D mode in PhotonVision, the [SolvePNP Algorithm](https://en.wikipedia.org/wiki/Perspective-n-Point) is used to deduce the *camera's* position in a 3D coordinate system centered on the target itself.
|
||||
|
||||
A simple algorithm for using this measurement is:
|
||||
|
||||
#. Assume your robot needs to be at a fixed ``Pose2D`` *relative to the target*.
|
||||
#. When triggered:
|
||||
1. Assume your robot needs to be at a fixed `Pose2D` *relative to the target*.
|
||||
2. When triggered:
|
||||
#. Read the most recent vision measurement - this is your *actual* pose.
|
||||
#. Generate a simple trajectory to the goal position
|
||||
#. Execute the trajectory
|
||||
|
||||
.. note:: There is not currently an example demonstrating this technique.
|
||||
:::{note}
|
||||
There is not currently an example demonstrating this technique.
|
||||
:::
|
||||
|
||||
Global Pose Estimation
|
||||
----------------------
|
||||
## Global Pose Estimation
|
||||
|
||||
A more complex way to utilize a camera-supplied ``Pose2D`` is to incorporate it into an estimation of the robot's ``Pose2D`` in a global field reference frame.
|
||||
A more complex way to utilize a camera-supplied `Pose2D` is to incorporate it into an estimation of the robot's `Pose2D` in a global field reference frame.
|
||||
|
||||
When using this strategy, the measurements made by the camera are *fused* with measurements from other sensors, a model of expected robot behavior, and a matrix of weights that describes how trustworthy each sensor is. The result is a *best-guess* at the current pose on the field.
|
||||
|
||||
In turn, this best-guess position is used to path plan to the known positions on the field, which may or may not have vision targets nearby.
|
||||
|
||||
See the :ref:`Pose Estimation <docs/examples/simposeest:Knowledge and Equipment Needed>` example for more information.
|
||||
See the {ref}`Pose Estimation <docs/examples/simposeest:Knowledge and Equipment Needed>` example for more information.
|
||||
46
docs/source/docs/integration/aprilTagStrategies.md
Normal file
46
docs/source/docs/integration/aprilTagStrategies.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# AprilTag Strategies
|
||||
|
||||
:::{note}
|
||||
The same strategies covered in the simple and advanced strategy sections still apply to AprilTags, and we encourage you to read them first. This page will discuss the specific nuances to using AprilTags.
|
||||
:::
|
||||
|
||||
## Simple Strategies
|
||||
|
||||
Prior to the introduction of AprilTags, the most common vision strategy for teams was to use the yaw of the detected target in order to turn to the target, and then score. This is still possible with AprilTags as the yaw of the tag is reported. Similarly, getting the distance to the target via trigonometry will also work. This is discussed in greater detail in the previous page.
|
||||
|
||||
## Advanced Strategies
|
||||
|
||||
AprilTags allows you find the robot pose on the field using data from the tags. A pose is a combination an X/Y coordinate, and an angle describing where the robot’s front is pointed. It is always considered relative to some fixed point on the field.
|
||||
|
||||
### Knowledge and Equipment Needed
|
||||
|
||||
Knowledge
|
||||
|
||||
- How to tune an AprilTag Pipeline (found in the pipeline tuning section)
|
||||
|
||||
Equipment
|
||||
|
||||
- A Coprocessor running PhotonVision - Accurate camera calibration to support “3D mode” required
|
||||
- A Drivetrain with wheels and sensors (Sufficient sensors to measure wheel rotation and capable of closed-loop velocity control)
|
||||
- A gyroscope or IMU measuring actual robot heading
|
||||
|
||||
### Global Pose Estimation / Pose Estimation Strategies
|
||||
|
||||
:::{note}
|
||||
See the previous page for more general information. Most of the information is the same except now the camera is supplying a `Pose3D`.
|
||||
:::
|
||||
|
||||
The nature of how AprilTags will be laid out makes it very likely that you will get multiple pose measurements within a single frame from seeing multiple targets. This requires strategies to fuse these observations together and get a "best guess" as to where your robot is. The best way to do this is to use the corners from all visible AprilTags to estimate the robot's pose. This is done by using the `PhotonPoseEstimator` class and the "MULTI_TAG_PNP_ON_COPROCESSOR" strategy. Additional strategies include:
|
||||
|
||||
- A camera seeing multiple targets, taking the average of all the returned poses
|
||||
- A camera seeing one target, with an assumed height off the ground, picking the pose which places it to the assumed height
|
||||
- A camera seeing one target, and picking a pose most similar to the most recently observed pose
|
||||
- A camera seeing one target, and picking a pose most similar to one provided externally (ie, from previous loop's odometry)
|
||||
- A camera seeing one target, and picking the pose with the lowest ambiguity.
|
||||
|
||||
PhotonVision supports all of these different strategies via our `PhotonPoseEstimator` class that allows you to select one of the strategies above and get the relevant pose estimation.
|
||||
|
||||
### Tuning Pose Estimators
|
||||
|
||||
Coming soon!
|
||||
TODO: Add this back in once simposeest example is added.
|
||||
@@ -1,48 +0,0 @@
|
||||
AprilTag Strategies
|
||||
====================
|
||||
|
||||
.. note:: The same strategies covered in the simple and advanced strategy sections still apply to AprilTags, and we encourage you to read them first. This page will discuss the specific nuances to using AprilTags.
|
||||
|
||||
Simple Strategies
|
||||
-----------------
|
||||
|
||||
Prior to the introduction of AprilTags, the most common vision strategy for teams was to use the yaw of the detected target in order to turn to the target, and then score. This is still possible with AprilTags as the yaw of the tag is reported. Similarly, getting the distance to the target via trigonometry will also work. This is discussed in greater detail in the previous page.
|
||||
|
||||
Advanced Strategies
|
||||
-------------------
|
||||
AprilTags allows you find the robot pose on the field using data from the tags. A pose is a combination an X/Y coordinate, and an angle describing where the robot’s front is pointed. It is always considered relative to some fixed point on the field.
|
||||
|
||||
Knowledge and Equipment Needed
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Knowledge
|
||||
|
||||
* How to tune an AprilTag Pipeline (found in the pipeline tuning section)
|
||||
|
||||
Equipment
|
||||
|
||||
* A Coprocessor running PhotonVision - Accurate camera calibration to support “3D mode” required
|
||||
|
||||
* A Drivetrain with wheels and sensors (Sufficient sensors to measure wheel rotation and capable of closed-loop velocity control)
|
||||
|
||||
* A gyroscope or IMU measuring actual robot heading
|
||||
|
||||
Global Pose Estimation / Pose Estimation Strategies
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. note:: See the previous page for more general information. Most of the information is the same except now the camera is supplying a ``Pose3D``.
|
||||
|
||||
The nature of how AprilTags will be laid out makes it very likely that you will get multiple pose measurements within a single frame from seeing multiple targets. This requires strategies to fuse these observations together and get a "best guess" as to where your robot is. The best way to do this is to use the corners from all visible AprilTags to estimate the robot's pose. This is done by using the ``PhotonPoseEstimator`` class and the "MULTI_TAG_PNP_ON_COPROCESSOR" strategy. Additional strategies include:
|
||||
|
||||
* A camera seeing multiple targets, taking the average of all the returned poses
|
||||
* A camera seeing one target, with an assumed height off the ground, picking the pose which places it to the assumed height
|
||||
* A camera seeing one target, and picking a pose most similar to the most recently observed pose
|
||||
* A camera seeing one target, and picking a pose most similar to one provided externally (ie, from previous loop's odometry)
|
||||
* A camera seeing one target, and picking the pose with the lowest ambiguity.
|
||||
|
||||
PhotonVision supports all of these different strategies via our ``PhotonPoseEstimator`` class that allows you to select one of the strategies above and get the relevant pose estimation.
|
||||
|
||||
Tuning Pose Estimators
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Coming soon!
|
||||
TODO: Add this back in once simposeest example is added.
|
||||
@@ -1,8 +1,6 @@
|
||||
Vision - Robot Integration Background
|
||||
=====================================
|
||||
# Vision - Robot Integration Background
|
||||
|
||||
Vision Processing's Purpose
|
||||
---------------------------
|
||||
## Vision Processing's Purpose
|
||||
|
||||
Each year, the FRC game requires a fundamental operation: **Align the Robot to a Goal**.
|
||||
|
||||
@@ -14,8 +12,8 @@ Software strategies can be used to help augment the ability of a human operator,
|
||||
|
||||
There are many valid strategies for doing this transformation. Picking a strategy is a balancing act between:
|
||||
|
||||
1. Available team resources (time, programming skills, previous experience)
|
||||
2. Precision of alignment required
|
||||
3. Team willingness to take on risk
|
||||
> 1. Available team resources (time, programming skills, previous experience)
|
||||
> 2. Precision of alignment required
|
||||
> 3. Team willingness to take on risk
|
||||
|
||||
Simple strategies are low-risk - they require comparatively little effort to implement and tune, but have hard limits on the complexity of motion they can control on the robot. Advanced methods allow for more complex and precise movement, but take more effort to implement and tune. For this reason, it is more risky to attempt to use them.
|
||||
10
docs/source/docs/integration/index.md
Normal file
10
docs/source/docs/integration/index.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# Robot Integration
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 2
|
||||
|
||||
background
|
||||
simpleStrategies
|
||||
advancedStrategies
|
||||
aprilTagStrategies
|
||||
```
|
||||
@@ -1,10 +0,0 @@
|
||||
Robot Integration
|
||||
=================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
background
|
||||
simpleStrategies
|
||||
advancedStrategies
|
||||
aprilTagStrategies
|
||||
@@ -1,40 +1,36 @@
|
||||
Simple Strategies
|
||||
=================
|
||||
# Simple Strategies
|
||||
|
||||
Simple strategies for using vision processor outputs involve using the target's position in the 2D image to infer *range* and *angle* to the target.
|
||||
|
||||
Knowledge and Equipment Needed
|
||||
------------------------------
|
||||
## Knowledge and Equipment Needed
|
||||
|
||||
- A Coprocessor running PhotonVision
|
||||
- A Drivetrain with wheels
|
||||
|
||||
Angle Alignment
|
||||
---------------
|
||||
## Angle Alignment
|
||||
|
||||
The simplest way to use a vision processing result is to first determine how far left or right in the image the vision target should be for your robot to be "aligned" to the target. Then,
|
||||
|
||||
1. Read the current angle to the target from the vision Coprocessor.
|
||||
2. If too far in one direction, command the drivetrain to rotate in the opposite direction to compensate.
|
||||
|
||||
See the :ref:`Aiming at a Target <docs/examples/aimingatatarget:Knowledge and Equipment Needed>` example for more information.
|
||||
See the {ref}`Aiming at a Target <docs/examples/aimingatatarget:Knowledge and Equipment Needed>` example for more information.
|
||||
|
||||
.. note:: Sometimes, these strategies have also involved incorporating a gyroscope. This can be necessary due to the high latency of vision processing algorithms. However, advancements in the tools available (including PhotonVision) has made that unnecessary for most applications.
|
||||
:::{note}
|
||||
Sometimes, these strategies have also involved incorporating a gyroscope. This can be necessary due to the high latency of vision processing algorithms. However, advancements in the tools available (including PhotonVision) has made that unnecessary for most applications.
|
||||
:::
|
||||
|
||||
Range Alignment
|
||||
---------------
|
||||
## Range Alignment
|
||||
|
||||
By looking at the position of the target in the "vertical" direction in the image, and applying some trigonometry, the distance between the robot and the camera can be deduced.
|
||||
|
||||
1. Read the current distance to the target from the vision coprocessor.
|
||||
2. If too far in one direction, command the drivetrain to travel in the opposite direction to compensate.
|
||||
|
||||
See the :ref:`Getting in Range of the Target <docs/examples/gettinginrangeofthetarget:Knowledge and Equipment Needed>` example for more information.
|
||||
See the {ref}`Getting in Range of the Target <docs/examples/gettinginrangeofthetarget:Knowledge and Equipment Needed>` example for more information.
|
||||
|
||||
|
||||
Angle + Range
|
||||
-------------
|
||||
## Angle + Range
|
||||
|
||||
Since the previous two alignment strategies work on independent axes of the robot, there's no reason you can't do them simultaneously.
|
||||
|
||||
See the :ref:`Aim and Range <docs/examples/aimandrange:Knowledge and Equipment Needed>` example for more information.
|
||||
See the {ref}`Aim and Range <docs/examples/aimandrange:Knowledge and Equipment Needed>` example for more information.
|
||||
Reference in New Issue
Block a user