Compare commits

...

16 Commits

Author SHA1 Message Date
Devon Doyle
5a87e4c738 Add calibration info tooltip (#2350) 2026-02-03 05:41:24 +00:00
Alan Everett
284e818e74 Fix dimmable LED off-state, topic names, PWM flicker, brightness update outside kDefault, indefinite blinking, and blinking reliability (#2337) 2026-02-02 21:11:34 -08:00
Gold856
f4b30da6b3 Dynamically import echarts and three.js (#2349) 2026-02-02 08:06:48 -08:00
Sam Freund
798b01c3a6 Copy old configs before testing (#2348)
## Description

Presently, any tests using our old configs happen in place. This is
problematic, as it changes the files themselves. This means that anyone
running the tests will cause unintentional modifications, which stand a
chance of being committed and merged into main. We want these old
database files to remain untouched, thus we copy them to a temp
directory prior to running our tests.

## Meta

Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_, including events
that led to this PR
- [ ] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with all settings going back to the previous seasons's last release
(seasons end after champs ends)
- [ ] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [ ] If this PR addresses a bug, a regression test for it is added
2026-02-02 03:30:37 +00:00
Sam Freund
23392f8d46 Add missing early-return to legacy ML model load (#2347)
## Description

Previously, when attempting to load a model that did not follow the old
naming pattern that was in the models directory, we caused PV to crash
due to a missing early return (reproducable on this branch with the
extra bonus ML model and missing label file):

```
    [2026-02-01 17:35:15] [Config - NeuralNetworkModelManager] [ERROR] Model properties are null. This could mean the config for model /home/matth/photonvision/test-resources/old_configs/2025.3.1-old-nnmm/models/iCauseProblems.rknn was unable to be found in the database. Trying legacy...
    [2026-02-01 17:35:15] [Config - NeuralNetworkModelManager] [ERROR] Failed to translate legacy model filename to properties: /home/matth/photonvision/test-resources/old_configs/2025.3.1-old-nnmm/models/iCauseProblems.rknn: /home/matth/photonvision/test-resources/old_configs/2025.3.1-old-nnmm/models/iCauseProblems-labels.txt

SQLConfigTest > testLoadNewNNMM() FAILED
    java.lang.NullPointerException: Cannot invoke "org.photonvision.common.configuration.NeuralNetworkModelsSettings$ModelProperties.toString()" because "properties" is null
        at org.photonvision.common.configuration.NeuralNetworkModelManager.loadModel(NeuralNetworkModelManager.java:342)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
```

This PR fixes it by adding the missing early-return, and adding unit
tests to make sure we handle this.

## Meta

Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_, including events
that led to this PR
- [ ] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with all settings going back to the previous seasons's last release
(seasons end after champs ends)
- [ ] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [x] If this PR addresses a bug, a regression test for it is added

---------

Co-authored-by: Matt M <matthew.morley.ca@gmail.com>
2026-02-02 02:09:13 +00:00
Wave Robotics - 2826
09e6d45e77 Allow configuring maximum target count (#2338) 2026-02-01 16:19:13 -08:00
Craig Schardt
77457219c7 Remove unused commands from the custom hardware configuration documentation (#2343)
## Description

#2255 introduced a new, cross-platform method for monitoring hardware
and removed the custom shell commands that had been used previously.
This PR updates the documentation to reflect the removal of those
commands from hardwareConfig.json.

## Meta

Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_, including events
that led to this PR
- [ ] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with all settings going back to the previous seasons's last release
(seasons end after champs ends)
- [ ] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [ ] If this PR addresses a bug, a regression test for it is added

---------

Co-authored-by: Gold856 <117957790+Gold856@users.noreply.github.com>
2026-01-29 23:40:09 -06:00
Bruce Kahn
da88867c60 Doc: Add a warning about versions 2.0.2 and later of the Raspberry Pi Imager not working properly (#2340)
## Description

FRC Team 6413 ran into problems trying to write the 2026.1.1 image using
the Raspberry Pi Imager tool. After some investigation we discovered a
bug was introduced in v2.0.2 of the imager tool which prevents it from
writing the PhotonVision images to any SD card on any Windows 10 or
Windows 11 computer.

Attempting to write a custom image (.img and .img.xz) results in the
following error dialog appearing as soon as the write action is
confirmed:

<img width="684" height="483" alt="photonvisionBurnFailure202"
src="https://github.com/user-attachments/assets/5c802ef0-75f9-4056-ac1f-760dcd3605bf"
/>

Versions 2.0.0 and earlier will successfully write the image files to
the selected SD card.

## Meta

Merge checklist:
- [X] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [X] The description documents the _what_ and _why_, including events
that led to this PR
- [ ] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with all settings going back to the previous seasons's last release
(seasons end after champs ends)
- [ ] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [ ] If this PR addresses a bug, a regression test for it is added
2026-01-28 19:45:30 -05:00
Matt Morley
a39844328d Add 2025.3.1 backwards-compat to ML models (#2331) 2026-01-28 12:54:35 -08:00
John Fogarty
1b5f4fa802 Update RubikPi Collab Notebook with working Device (#2339)
The RubikPi collab notebook included a device for creating a quantized
model that was deprecated by the Qualcomm team. I've included a separate
cell with a command to check available devices and updated the command
to a known working state.
2026-01-28 07:11:25 +00:00
Cooper Bouchard
7cc22e52ea Add FRC 2026 game piece detection models (#2332)
Added a model for detecting Rebuilt's FUEL game piece for versions of
the Orange Pi 5 and the Rubik Pi 3, which lowers the bar for less
experienced FRC teams to use object detection on their robots. Also
updated the documentation of object detection to reference the new
models.

The original model weights were trained by FRC team 2826 and were
published on this [Chief Delphi
thread](https://www.chiefdelphi.com/t/introducing-wave-robotics-yolov11-model-for-rebuilt/512701).

The dataset used for RKNN quantization is the [Fuel V3
dataset](https://universe.roboflow.com/frcroboraiders/frc-2026-fuel-sbrdk/dataset/3)
provided by
[frcroboraiders](https://universe.roboflow.com/frcroboraiders) on
Roboflow Universe.
2026-01-27 23:47:54 -06:00
Chris Gerth
49629afe9b not 2024 anymore (#2328)
Update apriltag field language in the docs to be year agnostic
2026-01-27 04:42:22 +00:00
Gold856
ae74b171aa Check POST request status before displaying success (#2336) 2026-01-26 20:11:25 -08:00
Gold856
cfd5773e7c Update README (#2333) 2026-01-26 06:25:59 +00:00
Gold856
c348f0e3ba Update PR template (#2334) 2026-01-26 05:56:17 +00:00
Matt Morley
4139566514 Add OV9281 AE startup quirk (#1814)
## Description

In https://github.com/PhotonVision/photonvision/issues/1771, we
discovered that the OV9281 on Linux seems to sometimes ignore our
exposure requests on first boot if we're in manual mode. Cycling the
camera's auto exposure with pauses in-between seems to fix it, which is
what this PR does.

Tested by [Team 2881 on
Discord](https://discord.com/channels/725836368059826228/725846784131203222/1465098110765105456);
thanks for helping us test this fix out!

## Meta

Closes https://github.com/PhotonVision/photonvision/issues/1771

Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_
- [ ] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with settings back to v2024.3.1
- [ ] If this PR addresses a bug, a regression test for it is added

---------

Co-authored-by: Gold856 <117957790+Gold856@users.noreply.github.com>
2026-01-26 00:11:14 +00:00
84 changed files with 1057 additions and 608 deletions

View File

@@ -1,18 +1,18 @@
## Description
<!-- What changed? Why? (the code + comments should speak for itself on the "how") -->
What changed? Why? (the code + comments should speak for itself on the "how")
<!-- Fun screenshots or a cool video or something are super helpful as well. If this touches platform-specific behavior, this is where test evidence should be collected. -->
Include fun testing screenshots or a cool video, to collect test evidence in a place where we can later reference it. Including proof this change was tested makes reviewing easier, helps us make sure we tested all our edge cases, and helps provide context for the future.
<!-- Any issues this pull request closes or pull requests this supersedes should be linked with `Closes #issuenumber`. -->
Any issues this pull request closes or pull requests this supersedes should be linked with `Closes #issuenumber`.
## Meta
Merge checklist:
- [ ] Pull Request title is [short, imperative summary](https://cbea.ms/git-commit/) of proposed changes
- [ ] The description documents the _what_ and _why_
- [ ] The description documents the _what_ and _why_, including events that led to this PR
- [ ] If this PR changes behavior or adds a feature, user documentation is updated
- [ ] If this PR touches photon-serde, all messages have been regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible with settings back to v2025.3.2
- [ ] If this PR touches configuration, this is backwards compatible with all settings going back to the previous seasons's last release (seasons end after champs ends)
- [ ] If this PR touches pipeline settings or anything related to data exchange, the frontend typing is updated
- [ ] If this PR addresses a bug, a regression test for it is added

View File

@@ -10,7 +10,7 @@ concurrency:
cancel-in-progress: true
env:
IMAGE_VERSION: v2026.1.1
IMAGE_VERSION: v2026.1.2
jobs:

View File

@@ -8,18 +8,18 @@ The latest release of platform-specific jars and images is found [here](https://
If you are interested in contributing code or documentation to the project, please [read our getting started page for contributors](https://docs.photonvision.org/en/latest/docs/contributing/index.html) and **[join the Discord](https://discord.gg/wYxTwym) to introduce yourself!** We hope to provide a welcoming community to anyone who is interested in helping.
## Authors
<a href="https://github.com/PhotonVision/photonvision/graphs/contributors">
<img src="https://contrib.rocks/image?repo=PhotonVision/photonvision" />
</a>
## Documentation
- Our main documentation page: [docs.photonvision.org](https://docs.photonvision.org)
- Photon UI demo: [demo.photonvision.org](https://demo.photonvision.org)
- Javadocs: [javadocs.photonvision.org](https://javadocs.photonvision.org)
- C++ Doxygen [cppdocs.photonvision.org](https://cppdocs.photonvision.org)
- C++ Doxygen: [cppdocs.photonvision.org](https://cppdocs.photonvision.org)
## Authors
<a href="https://github.com/PhotonVision/photonvision/graphs/contributors">
<img src="https://contrib.rocks/image?repo=PhotonVision/photonvision" />
</a>
## Building
@@ -32,7 +32,6 @@ You can run one of the many built in examples straight from the command line, to
Note that these are case sensitive!
* `-PArchOverride=foobar`: builds for a target system other than your current architecture. [Valid overrides](https://github.com/wpilibsuite/wpilib-tool-plugin/blob/main/src/main/java/edu/wpi/first/tools/NativePlatforms.java) are:
* winx32
* winx64
* winarm64
* macx64
@@ -46,33 +45,34 @@ Note that these are case sensitive!
- `-Pprofile`: enables JVM profiling
- `-PwithSanitizers`: On Linux, enables `-fsanitize=address,undefined,leak`
If you're cross-compiling, you'll need the wpilib toolchain installed. This can be done via Gradle: for example `./gradlew installArm64Toolchain` or `./gradlew installRoboRioToolchain`
If you're cross-compiling, you'll need the WPILib toolchain installed. This must be done via Gradle: for example `./gradlew installArm64Toolchain` or `./gradlew installRoboRioToolchain`
## Out-of-Source Dependencies
PhotonVision uses the following additional out-of-source repositories for building code.
- Base system images for Raspberry Pi & Orange Pi: https://github.com/PhotonVision/photon-image-modifier
- Base system images for supported coprocessors: https://github.com/PhotonVision/photon-image-modifier
- C++ driver for Raspberry Pi CSI cameras: https://github.com/PhotonVision/photon-libcamera-gl-driver
- JNI code for [mrcal](https://mrcal.secretsauce.net/): https://github.com/PhotonVision/mrcal-java
- Custom build of OpenCV with GStreamer/Protobuf/other custom flags: https://github.com/PhotonVision/thirdparty-opencv
- JNI code for aruco-nano: https://github.com/PhotonVision/aruconano-jni
- JNI code for RKNN: https://github.com/PhotonVision/rknn_jni
- JNI code for Rubik Pi NPU: https://github.com/PhotonVision/rubik_jni
## Acknowledgments
PhotonVision was forked from [Chameleon Vision](https://github.com/Chameleon-Vision/chameleon-vision/). Thank you to everyone who worked on the original project.
* [WPILib](https://github.com/wpilibsuite) - Specifically [cscore](https://github.com/wpilibsuite/allwpilib/tree/main/cscore), [CameraServer](https://github.com/wpilibsuite/allwpilib/tree/main/cameraserver), [NTCore](https://github.com/wpilibsuite/allwpilib/tree/main/ntcore), and [OpenCV](https://github.com/wpilibsuite/thirdparty-opencv).
* [Apache Commons](https://commons.apache.org/) - Specifically [Commons Math](https://commons.apache.org/proper/commons-math/), and [Commons Lang](https://commons.apache.org/proper/commons-lang/)
* [WPILib](https://github.com/wpilibsuite) - Specifically [allwpilib](https://github.com/wpilibsuite/allwpilib) and [their build of OpenCV](https://github.com/wpilibsuite/thirdparty-opencv).
* [Apache Commons](https://commons.apache.org/) - Specifically [Commons IO](https://commons.apache.org/proper/commons-io/), and [Commons CLI](https://commons.apache.org/proper/commons-cli/)
* [diozero](https://www.diozero.com/)
* [EJML](https://github.com/lessthanoptimal/ejml)
* [Javalin](https://javalin.io/)
* [JSON](https://json.org)
* [FasterXML](https://github.com/FasterXML) - Specifically [jackson](https://github.com/FasterXML/jackson)
* [MessagePack for Java](https://github.com/msgpack/msgpack-java)
* [OSHI](https://github.com/oshi/oshi)
* [QuickBuffers](https://github.com/HebiRobotics/QuickBuffers)
* [SQLite JDBC](https://github.com/xerial/sqlite-jdbc)
* [ZT ZIP](https://github.com/zeroturnaround/zt-zip)
## License

View File

@@ -3,6 +3,6 @@
"supportURL" : "https://limelightvision.io",
"ledPins" : [ 13, 18 ],
"ledsCanDim" : true,
"ledPWMFrequency" : 30000,
"ledPWMFrequency" : 1000,
"vendorFOV" : 75.76079874010732
}

View File

@@ -18,7 +18,7 @@ The {code}`Drivetrain` class includes functionality to fuse multiple sensor read
Please reference the [WPILib documentation](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/state-space/state-space-pose_state-estimators.html) on using the {code}`SwerveDrivePoseEstimator` class.
We use the 2024 game's AprilTag Locations:
We use the current game's AprilTag Locations:
```{eval-rst}
.. tab-set-code::

View File

@@ -73,27 +73,21 @@ If you were using custom LED commands from 2025 or earlier and still need custom
## Hardware Interaction Commands
For Non-Raspberry-Pi hardware, users must provide valid hardware-specific commands for some parts of the UI interaction (including performance metrics, and executing system restarts).
For non-Linux hardware, users must provide the hardware-specific command for executing system restarts.
Leaving a command blank will disable the associated functionality.
Leaving this command blank will disable the restart functionality.
```{eval-rst}
.. tab-set-code::
.. code-block:: json
{
"cpuTempCommand" : "",
"cpuMemoryCommand" : "",
"cpuUtilCommand" : "",
"gpuMemoryCommand" : "",
"gpuTempCommand" : "",
"ramUtilCommand" : "",
"restartHardwareCommand" : "",
}
```
:::{note}
These settings have no effect if PhotonVision detects it is running on a Raspberry Pi. See [the MetricsBase class](https://github.com/PhotonVision/photonvision/blob/dbd631da61b7c86b70fa6574c2565ad57d80a91a/photon-core/src/main/java/org/photonvision/common/hardware/metrics/MetricsBase.java) for the commands utilized.
This setting has no effect if PhotonVision detects it is running on Linux. On Linux, the restart is accomplished by executing `reboot now` in a shell.
:::
## Known Camera FOV
@@ -150,13 +144,7 @@ Here is a complete example `hardwareConfig.json`:
"setGPIOCommand" : "setGPIO {p} {s}",
"setPWMCommand" : "setPWM {p} {v}",
"setPWMFrequencyCommand" : "setPWMFrequency {p} {f}",
"releaseGPIOCommand" : "releseGPIO {p}",
"cpuTempCommand" : "",
"cpuMemoryCommand" : "",
"cpuUtilCommand" : "",
"gpuMemoryCommand" : "",
"gpuTempCommand" : "",
"ramUtilCommand" : "",
"releaseGPIOCommand" : "releaseGPIO {p}",
"restartHardwareCommand" : "",
"vendorFOV" : 72.5
}

View File

@@ -4,7 +4,7 @@
PhotonVision supports object detection using neural network accelerator hardware, commonly known as an NPU. The two coprocessors currently supported are the {ref}`Orange Pi 5 <docs/objectDetection/opi:Orange Pi 5 (and variants) Object Detection>` and the {ref}`Rubik Pi 3 <docs/objectDetection/rubik:Rubik Pi 3 Object Detection>`.
PhotonVision currently ships with a model trained on the [COCO dataset](https://cocodataset.org/) by [Ultralytics](https://github.com/ultralytics/ultralytics) (this model is licensed under [AGPLv3](https://www.gnu.org/licenses/agpl-3.0.en.html)). This model is meant to be used for testing and other miscellaneous purposes. It is not meant to be used in competition. For the 2025 post-season, PhotonVision also ships with a pretrained ALGAE model. A model to detect coral is available in the PhotonVision discord, but will not be distributed with PhotonVision.
PhotonVision currently ships with a model trained on the [COCO dataset](https://cocodataset.org/) by [Ultralytics](https://github.com/ultralytics/ultralytics) (this model is licensed under [AGPLv3](https://www.gnu.org/licenses/agpl-3.0.en.html)). This model is meant to be used for testing and other miscellaneous purposes. It is not meant to be used in competition. For the 2026 season, PhotonVision ships with a model to detect FUEL, this is also licensed under AGPL.
## Tracking Objects

View File

@@ -28,6 +28,10 @@ Unless otherwise noted in release notes or if updating from the prior years vers
Use the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) to flash the image onto the coprocessors microSD card. Select the downloaded `.img.xz` file, select your microSD card, and flash.
:::{warning}
Avoid using Raspberry Pi Imager version 2.0.2 or later. Those versions fail to write the image to an SD card. Versions 2.0.0 and earlier write images successfully. [GitHub issue 1489](https://github.com/raspberrypi/rpi-imager/issues/1489) was created for this problem.
:::
:::{warning}
Balena Etcher has been recommended in the past, but should no longer be used due to instability and lack of ongoing support from developers.
:::

View File

@@ -44,7 +44,7 @@ let renderer: WebGLRenderer | undefined;
let controls: TrackballControls | undefined;
let previousTargets: Object3D[] = [];
const drawTargets = (targets: PhotonTarget[]) => {
const drawTargets = async (targets: PhotonTarget[]) => {
// Check here, since if we check in watchEffect this never gets called
if (!scene || !camera || !renderer || !controls) {
return;
@@ -89,7 +89,11 @@ const drawTargets = (targets: PhotonTarget[]) => {
if (calibrationCoeffs) {
// And show camera frustum
const calibCamera = createPerspectiveCamera(calibrationCoeffs.resolution, calibrationCoeffs.cameraIntrinsics, 10);
const calibCamera = await createPerspectiveCamera(
calibrationCoeffs.resolution,
calibrationCoeffs.cameraIntrinsics,
10
);
const helper = new CameraHelper(calibCamera);
const helperGroup = new Group();
helperGroup.add(helper);

View File

@@ -65,7 +65,7 @@ const createChessboard = (obs: BoardObservation, cal: CameraCalibrationResult):
let previousTargets: Object3D[] = [];
let baseAspect: number | undefined;
const drawCalibration = (cal: CameraCalibrationResult | null) => {
const drawCalibration = async (cal: CameraCalibrationResult | null) => {
// Check here, since if we check in watchEffect this never gets called
if (!cal || !scene || !camera || !renderer || !controls) {
return;
@@ -95,7 +95,7 @@ const drawCalibration = (cal: CameraCalibrationResult | null) => {
});
// And show camera frustum
const calibCamera = createPerspectiveCamera(props.resolution, cal.cameraIntrinsics);
const calibCamera = await createPerspectiveCamera(props.resolution, cal.cameraIntrinsics);
const helper = new CameraHelper(calibCamera);
// Flip to +Z forward

View File

@@ -14,6 +14,7 @@ import { getResolutionString, resolutionsAreEqual } from "@/lib/PhotonUtils";
import CameraCalibrationInfoCard from "@/components/cameras/CameraCalibrationInfoCard.vue";
import { useSettingsStore } from "@/stores/settings/GeneralSettingsStore";
import { useTheme } from "vuetify";
import TooltippedLabel from "@/components/common/pv-tooltipped-label.vue";
const PromptRegular = import("@/assets/fonts/PromptRegular");
const jspdf = import("jspdf");
@@ -243,7 +244,14 @@ const setSelectedVideoFormat = (format: VideoFormat) => {
<v-card class="mb-3 rounded-12" color="surface" dark>
<v-card-title>Camera Calibration</v-card-title>
<v-card-text v-if="!isCalibrating" class="pb-0">
<v-card-subtitle class="pa-0 pb-3 text-white">Current Calibrations</v-card-subtitle>
<div class="pb-3">
<tooltipped-label
label="Curent Calibrations"
icon="mdi-information"
location="top"
tooltip="Click on a resolution to view detailed calibration information and import/export a calibration."
/>
</div>
<v-table fixed-header height="100%" density="compact">
<thead>
<tr>
@@ -282,22 +290,10 @@ const setSelectedVideoFormat = (format: VideoFormat) => {
</v-card-text>
<v-card-text class="pt-0">
<div v-if="useCameraSettingsStore().isConnected" class="d-flex flex-column">
<v-card-subtitle v-if="!isCalibrating" class="pl-0 pb-3 pt-3 text-white"
<v-card-subtitle v-if="!isCalibrating" class="pl-0 pb-3 pt-4 opacity-100"
>Configure New Calibration</v-card-subtitle
>
<v-form ref="form" v-model="settingsValid">
<v-alert
closable
density="compact"
:variant="theme.global.name.value === 'LightTheme' ? 'elevated' : 'tonal'"
:color="useSettingsStore().general.mrCalWorking ? 'buttonPassive' : 'error'"
:icon="useSettingsStore().general.mrCalWorking ? 'mdi-check' : 'mdi-close'"
:text="
useSettingsStore().general.mrCalWorking
? 'Mrcal was successfully loaded and will be used!'
: 'MrCal failed to load, check journalctl logs for details.'
"
/>
<pv-select
v-model="uniqueVideoResolutionString"
label="Resolution"
@@ -470,7 +466,20 @@ const setSelectedVideoFormat = (format: VideoFormat) => {
"
/>
</div>
<div v-if="isCalibrating" class="d-flex justify-center align-center pt-10px pb-5">
<v-alert
closable
density="compact"
class="mb-5"
:variant="theme.global.name.value === 'LightTheme' ? 'elevated' : 'tonal'"
:color="useSettingsStore().general.mrCalWorking ? 'buttonPassive' : 'error'"
:icon="useSettingsStore().general.mrCalWorking ? 'mdi-check' : 'mdi-close'"
:text="
useSettingsStore().general.mrCalWorking
? 'Mrcal was successfully loaded and will be used!'
: 'MrCal failed to load, check journalctl logs for details.'
"
/>
<div v-if="isCalibrating" class="d-flex justify-center align-center pb-5">
<v-chip
:variant="theme.global.name.value === 'LightTheme' ? 'elevated' : 'tonal'"
label

View File

@@ -3,6 +3,7 @@ import PvSelect from "@/components/common/pv-select.vue";
import { useCameraSettingsStore } from "@/stores/settings/CameraSettingsStore";
import { type ActivePipelineSettings, PipelineType, RobotOffsetPointMode } from "@/types/PipelineTypes";
import PvSwitch from "@/components/common/pv-switch.vue";
import PvSlider from "@/components/common/pv-slider.vue";
import { computed } from "vue";
import { RobotOffsetType } from "@/types/SettingTypes";
import { useStateStore } from "@/stores/StateStore";
@@ -58,14 +59,17 @@ const interactiveCols = computed(() =>
<template>
<div>
<pv-switch
v-model="useCameraSettingsStore().currentPipelineSettings.outputShowMultipleTargets"
label="Show Multiple Targets"
tooltip="If enabled, up to five targets will be displayed and sent via PhotonLib, instead of just one"
:disabled="isTagPipeline"
<pv-slider
v-model="useCameraSettingsStore().currentPipelineSettings.outputMaximumTargets"
label="Maximum Targets"
tooltip="The maximum number of targets to display and send."
:hidden="isTagPipeline"
:min="1"
:max="127"
:step="1"
:switch-cols="interactiveCols"
@update:modelValue="
(value) => useCameraSettingsStore().changeCurrentPipelineSetting({ outputShowMultipleTargets: value }, false)
(value) => useCameraSettingsStore().changeCurrentPipelineSetting({ outputMaximumTargets: value }, false)
"
/>
<pv-switch

View File

@@ -13,13 +13,15 @@ import { metricsHistorySnapshot } from "@/stores/settings/GeneralSettingsStore";
const theme = useTheme();
const restartProgram = () => {
axiosPost("/utils/restartProgram", "restart PhotonVision");
forceReloadPage();
const restartProgram = async () => {
if (await axiosPost("/utils/restartProgram", "restart PhotonVision")) {
forceReloadPage();
}
};
const restartDevice = () => {
axiosPost("/utils/restartDevice", "restart the device");
forceReloadPage();
const restartDevice = async () => {
if (await axiosPost("/utils/restartDevice", "restart the device")) {
forceReloadPage();
}
};
const address = inject<string>("backendHost");
@@ -38,28 +40,30 @@ const handleOfflineUpdate = async () => {
color: "secondary",
timeout: -1
});
await axiosPost("/utils/offlineUpdate", "upload new software", formData, {
headers: { "Content-Type": "multipart/form-data" },
onUploadProgress: ({ progress }) => {
const uploadPercentage = (progress || 0) * 100.0;
if (uploadPercentage < 99.5) {
useStateStore().showSnackbarMessage({
message: "New Software Upload in Progress",
color: "secondary",
timeout: -1,
progressBar: uploadPercentage,
progressBarColor: "primary"
});
} else {
useStateStore().showSnackbarMessage({
message: "Installing uploaded software...",
color: "secondary",
timeout: -1
});
if (
await axiosPost("/utils/offlineUpdate", "upload new software", formData, {
headers: { "Content-Type": "multipart/form-data" },
onUploadProgress: ({ progress }) => {
const uploadPercentage = (progress || 0) * 100.0;
if (uploadPercentage < 99.5) {
useStateStore().showSnackbarMessage({
message: "New Software Upload in Progress",
color: "secondary",
timeout: -1,
progressBar: uploadPercentage,
progressBarColor: "primary"
});
}
}
}
});
forceReloadPage();
})
) {
useStateStore().showSnackbarMessage({
message: "Installing uploaded software...",
color: "secondary",
timeout: -1
});
forceReloadPage();
}
};
const exportLogFile = ref();
@@ -116,9 +120,10 @@ const handleSettingsImport = () => {
};
const showFactoryReset = ref(false);
const nukePhotonConfigDirectory = () => {
axiosPost("/utils/nukeConfigDirectory", "delete the config directory");
forceReloadPage();
const nukePhotonConfigDirectory = async () => {
if (await axiosPost("/utils/nukeConfigDirectory", "delete the config directory")) {
forceReloadPage();
}
};
interface MetricItem {
@@ -371,21 +376,33 @@ watch(metricsHistorySnapshot, () => {
<span>CPU Usage</span>
<span>{{ Math.round(cpuUsageData.at(-1)?.value ?? 0) }}%</span>
</div>
<MetricsChart id="chart" :data="cpuUsageData" type="percentage" :min="0" :max="100" color="blue" />
<Suspense>
<!-- Allows us to import echarts when it's actually needed -->
<MetricsChart id="chart" :data="cpuUsageData" type="percentage" :min="0" :max="100" color="blue" />
<template #fallback> Loading... </template>
</Suspense>
</v-card-text>
<v-card-text class="pt-0 flex-0-0 pb-2">
<div class="d-flex justify-space-between pb-3 pt-3">
<span>CPU Memory Usage</span>
<span>{{ Math.round(cpuMemoryUsageData.at(-1)?.value ?? 0) }}%</span>
</div>
<MetricsChart id="chart" :data="cpuMemoryUsageData" type="percentage" :min="0" :max="100" color="purple" />
<Suspense>
<!-- Allows us to import echarts when it's actually needed -->
<MetricsChart id="chart" :data="cpuMemoryUsageData" type="percentage" :min="0" :max="100" color="purple" />
<template #fallback> Loading... </template>
</Suspense>
</v-card-text>
<v-card-text class="pt-0 flex-0-0 pb-2">
<div class="d-flex justify-space-between pb-3 pt-3">
<span>CPU Temperature</span>
<span>{{ cpuTempData.at(-1)?.value == -1 ? "--- " : Math.round(cpuTempData.at(-1)?.value ?? 0) }}°C</span>
</div>
<MetricsChart id="chart" :data="cpuTempData" type="temperature" color="red" />
<Suspense>
<!-- Allows us to import echarts when it's actually needed -->
<MetricsChart id="chart" :data="cpuTempData" type="temperature" color="red" />
<template #fallback> Loading... </template>
</Suspense>
</v-card-text>
<v-card-text class="pt-0 flex-0-0">
<div class="d-flex justify-space-between pb-3 pt-3">
@@ -399,7 +416,11 @@ watch(metricsHistorySnapshot, () => {
>{{ networkUsageData.at(-1)?.value == -1 ? "---" : networkUsageData.at(-1)?.value.toFixed(3) }} Mb/s</span
>
</div>
<MetricsChart id="chart" :data="networkUsageData" type="mb" :min="0" :max="10" color="green" />
<Suspense>
<!-- Allows us to import echarts when it's actually needed -->
<MetricsChart id="chart" :data="networkUsageData" type="mb" :min="0" :max="10" color="green" />
<template #fallback> Loading... </template>
</Suspense>
</v-card-text>
</v-card>
</v-col>

View File

@@ -1,5 +1,4 @@
<script setup lang="ts">
import * as echarts from "echarts";
import { onMounted, ref, onBeforeUnmount, watch } from "vue";
import { useTheme } from "vuetify";
@@ -198,7 +197,8 @@ const props = defineProps<{
color?: string;
}>();
onMounted(() => {
onMounted(async () => {
const echarts = await import("echarts");
chart = echarts.init(chartRef.value);
chart.setOption(getOptions(props.data));

View File

@@ -26,7 +26,7 @@ const importWidth = ref<number | null>(null);
const importVersion = ref<string | null>(null);
// TODO gray out the button when model is uploading
const handleImport = () => {
const handleImport = async () => {
if (importModelFile.value === null) return;
const formData = new FormData();
@@ -43,25 +43,27 @@ const handleImport = () => {
timeout: -1
});
axiosPost("/objectdetection/import", "import an object detection model", formData, {
headers: { "Content-Type": "multipart/form-data" },
onUploadProgress: ({ progress }) => {
const uploadPercentage = (progress || 0) * 100.0;
if (uploadPercentage < 99.5) {
useStateStore().showSnackbarMessage({
message: "Object Detection Model Upload in Process, " + uploadPercentage.toFixed(2) + "% complete",
color: "secondary",
timeout: -1
});
} else {
useStateStore().showSnackbarMessage({
message: "Processing uploaded Object Detection Model...",
color: "secondary",
timeout: -1
});
if (
await axiosPost("/objectdetection/import", "import an object detection model", formData, {
headers: { "Content-Type": "multipart/form-data" },
onUploadProgress: ({ progress }) => {
const uploadPercentage = (progress || 0) * 100.0;
if (uploadPercentage < 99.5) {
useStateStore().showSnackbarMessage({
message: "Object Detection Model Upload in Process, " + uploadPercentage.toFixed(2) + "% complete",
color: "secondary",
timeout: -1
});
}
}
}
});
})
) {
useStateStore().showSnackbarMessage({
message: "Processing uploaded Object Detection Model...",
color: "secondary",
timeout: -1
});
}
showImportDialog.value = false;
@@ -121,33 +123,35 @@ const nukeModels = () => {
const showBulkImportDialog = ref(false);
const importFile = ref<File | null>(null);
const handleBulkImport = () => {
const handleBulkImport = async () => {
if (importFile.value === null) return;
const formData = new FormData();
formData.append("data", importFile.value);
axiosPost("/objectdetection/bulkimport", "import object detection models", formData, {
headers: { "Content-Type": "multipart/form-data" },
onUploadProgress: ({ progress }) => {
const uploadPercentage = (progress || 0) * 100.0;
if (uploadPercentage < 99.5) {
useStateStore().showSnackbarMessage({
message: "Object Detection Models Upload in Progress",
color: "secondary",
timeout: -1,
progressBar: uploadPercentage,
progressBarColor: "primary"
});
} else {
useStateStore().showSnackbarMessage({
message: "Importing New Object Detection Models...",
color: "secondary",
timeout: -1
});
if (
await axiosPost("/objectdetection/bulkimport", "import object detection models", formData, {
headers: { "Content-Type": "multipart/form-data" },
onUploadProgress: ({ progress }) => {
const uploadPercentage = (progress || 0) * 100.0;
if (uploadPercentage < 99.5) {
useStateStore().showSnackbarMessage({
message: "Object Detection Models Upload in Progress",
color: "secondary",
timeout: -1,
progressBar: uploadPercentage,
progressBarColor: "primary"
});
}
}
}
});
})
) {
useStateStore().showSnackbarMessage({
message: "Importing New Object Detection Models...",
color: "secondary",
timeout: -1
});
}
showImportDialog.value = false;
importFile.value = null;
};

View File

@@ -71,33 +71,33 @@ export const parseJsonFile = async <T extends Record<string, any>>(file: File):
* @param description A brief description of the request for users, e.g., "import object detection models".
* @param data Payload to be sent in the POST request
* @param config Optional axios request configuration
* @returns A promise that resolves when the POST request is complete
* @returns A promise that resolves to true if the POST request is successful, or false if an error occurs.
*/
export const axiosPost = (url: string, description: string, data?: any, config?: any): Promise<void> => {
return axios
.post(url, data, config)
.then(() => {
useStateStore().showSnackbarMessage({
message: "Successfully dispatched the request to " + description + ". Waiting for backend to respond",
color: "success"
});
})
.catch((error) => {
if (error.response) {
useStateStore().showSnackbarMessage({
message: "The backend is unable to fulfill the request to " + description + ".",
color: "error"
});
} else if (error.request) {
useStateStore().showSnackbarMessage({
message: "Error while trying to process the request to " + description + "! The backend didn't respond.",
color: "error"
});
} else {
useStateStore().showSnackbarMessage({
message: "An error occurred while trying to process the request to " + description + ".",
color: "error"
});
}
export const axiosPost = async (url: string, description: string, data?: any, config?: any): Promise<boolean> => {
try {
await axios.post(url, data, config);
useStateStore().showSnackbarMessage({
message: "Successfully dispatched the request to " + description + ". Waiting for backend to respond",
color: "success"
});
return true;
} catch (error: any) {
if (error.response) {
useStateStore().showSnackbarMessage({
message: "The backend is unable to fulfill the request to " + description + ".",
color: "error"
});
} else if (error.request) {
useStateStore().showSnackbarMessage({
message: "Error while trying to process the request to " + description + "! The backend didn't respond.",
color: "error"
});
} else {
useStateStore().showSnackbarMessage({
message: "An error occurred while trying to process the request to " + description + ".",
color: "error"
});
}
return false;
}
};

View File

@@ -1,5 +1,5 @@
import type { JsonMatOfDouble, Resolution } from "@/types/SettingTypes";
const { PerspectiveCamera } = await import("three");
const three = import("three");
/**
* Convert a camera intrinsics matrix and image resolution to a Three.js PerspectiveCamera. This assumes no skew and square pixels (same focal length in x and y), which is a sane assumption for most FRC cameras
@@ -8,11 +8,12 @@ const { PerspectiveCamera } = await import("three");
* @param intrinsicsCore camera intrinsics from the backend, row-major
* @returns a Three.js PerspectiveCamera matching the provided intrinsics
*/
export const createPerspectiveCamera = (
export const createPerspectiveCamera = async (
resolution: Resolution,
intrinsicsCore: JsonMatOfDouble,
frustumMax: number = 1
) => {
const { PerspectiveCamera } = await three;
const imageWidth = resolution.width;
const imageHeight = resolution.height;
const focalLengthY = intrinsicsCore.data[4];

View File

@@ -66,7 +66,7 @@ export interface PipelineSettings {
hsvHue: WebsocketNumberPair | [number, number];
ledMode: boolean;
hueInverted: boolean;
outputShowMultipleTargets: boolean;
outputMaximumTargets: number;
contourSortMode: number;
cameraExposureRaw: number;
cameraMinExposureRaw: number;
@@ -108,7 +108,7 @@ export type ConfigurablePipelineSettings = Partial<
// Omitted settings are changed for all pipeline types
export const DefaultPipelineSettings: Omit<
PipelineSettings,
"cameraGain" | "targetModel" | "ledMode" | "outputShowMultipleTargets" | "cameraExposureRaw" | "pipelineType"
"cameraGain" | "targetModel" | "ledMode" | "cameraExposureRaw" | "pipelineType"
> = {
offsetRobotOffsetMode: RobotOffsetPointMode.None,
streamingFrameDivisor: 0,
@@ -137,6 +137,7 @@ export const DefaultPipelineSettings: Omit<
offsetDualPointB: { x: 0, y: 0 },
hsvHue: { first: 50, second: 180 },
hueInverted: false,
outputMaximumTargets: 20,
contourSortMode: 0,
offsetSinglePoint: { x: 0, y: 0 },
cameraBrightness: 50,
@@ -166,7 +167,7 @@ export const DefaultReflectivePipelineSettings: ReflectivePipelineSettings = {
cameraGain: 20,
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
ledMode: true,
outputShowMultipleTargets: false,
outputMaximumTargets: 20,
cameraExposureRaw: 6,
pipelineType: PipelineType.Reflective,
@@ -197,7 +198,7 @@ export const DefaultColoredShapePipelineSettings: ColoredShapePipelineSettings =
cameraGain: 75,
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
ledMode: true,
outputShowMultipleTargets: false,
outputMaximumTargets: 20,
cameraExposureRaw: 20,
pipelineType: PipelineType.ColoredShape,
@@ -237,10 +238,9 @@ export const DefaultAprilTagPipelineSettings: AprilTagPipelineSettings = {
cameraGain: 75,
targetModel: TargetModel.AprilTag6p5in_36h11,
ledMode: false,
outputShowMultipleTargets: true,
outputMaximumTargets: 127,
cameraExposureRaw: 20,
pipelineType: PipelineType.AprilTag,
hammingDist: 0,
numIterations: 40,
decimate: 1,
@@ -278,13 +278,12 @@ export type ConfigurableArucoPipelineSettings = Partial<Omit<ArucoPipelineSettin
export const DefaultArucoPipelineSettings: ArucoPipelineSettings = {
...DefaultPipelineSettings,
cameraGain: 75,
outputShowMultipleTargets: true,
outputMaximumTargets: 127,
targetModel: TargetModel.AprilTag6p5in_36h11,
cameraExposureRaw: -1,
cameraAutoExposure: true,
ledMode: false,
pipelineType: PipelineType.Aruco,
tagFamily: AprilTagFamily.Family36h11,
threshWinSizes: { first: 11, second: 91 },
threshStepSize: 40,
@@ -316,7 +315,7 @@ export const DefaultObjectDetectionPipelineSettings: ObjectDetectionPipelineSett
cameraGain: 20,
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
ledMode: true,
outputShowMultipleTargets: false,
outputMaximumTargets: 20,
cameraExposureRaw: 6,
confidence: 0.9,
nms: 0.45,
@@ -335,7 +334,7 @@ export const DefaultCalibration3dPipelineSettings: Calibration3dPipelineSettings
cameraGain: 20,
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
ledMode: true,
outputShowMultipleTargets: false,
outputMaximumTargets: 1,
cameraExposureRaw: 6,
drawAllSnapshots: false
};

View File

@@ -39,7 +39,7 @@ import org.photonvision.vision.processes.VisionSource;
import org.zeroturnaround.zip.ZipUtil;
public class ConfigManager {
private static ConfigManager INSTANCE;
static ConfigManager INSTANCE;
public static final String HW_CFG_FNAME = "hardwareConfig.json";
public static final String HW_SET_FNAME = "hardwareSettings.json";

View File

@@ -218,7 +218,7 @@ class LegacyConfigProvider extends ConfigProvider {
hardwareSettings,
networkConfig,
atfl,
new NeuralNetworkPropertyManager(),
new NeuralNetworkModelsSettings(),
cameraConfigurations);
}

View File

@@ -34,7 +34,7 @@ import java.util.Optional;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.stream.Stream;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
import org.photonvision.common.hardware.Platform;
import org.photonvision.common.logging.LogGroup;
import org.photonvision.common.logging.Logger;
@@ -50,20 +50,20 @@ import org.photonvision.vision.objects.RubikModel;
* extracted to the filesystem, it will not be extracted again.
*
* <p>Each model must have a corresponding {@link ModelProperties} entry in {@link
* NeuralNetworkPropertyManager}.
* NeuralNetworkModelsSettings}.
*/
public class NeuralNetworkModelManager {
/** Singleton instance of the NeuralNetworkModelManager */
private static NeuralNetworkModelManager INSTANCE;
private final List<Family> supportedBackends = new ArrayList<>();
final List<Family> supportedBackends = new ArrayList<>();
/**
* This function stores the properties of the shipped object detection models. It is stored as a
* function so that it can be dynamic, to adjust for the models directory.
*/
private NeuralNetworkPropertyManager getShippedProperties(File modelsDirectory) {
NeuralNetworkPropertyManager nnProps = new NeuralNetworkPropertyManager();
private NeuralNetworkModelsSettings getShippedProperties(File modelsDirectory) {
NeuralNetworkModelsSettings nnProps = new NeuralNetworkModelsSettings();
LinkedList<String> cocoLabels =
new LinkedList<String>(
@@ -149,16 +149,6 @@ public class NeuralNetworkModelManager {
"hair drier", // Typo in official COCO documentation
"toothbrush"));
nnProps.addModelProperties(
new ModelProperties(
Path.of(modelsDirectory.getAbsolutePath(), "algaeV1-640-640-yolov8n.rknn"),
"Algae v8n",
new LinkedList<String>(List.of("Algae")),
640,
480,
Family.RKNN,
Version.YOLOV8));
nnProps.addModelProperties(
new ModelProperties(
Path.of(modelsDirectory.getAbsolutePath(), "yolov8nCOCO.rknn"),
@@ -171,13 +161,13 @@ public class NeuralNetworkModelManager {
nnProps.addModelProperties(
new ModelProperties(
Path.of(modelsDirectory.getAbsolutePath(), "algae-coral-yolov8s.tflite"),
"Algae Coral v8s",
new LinkedList<String>(List.of("Algae", "Coral")),
Path.of(modelsDirectory.getAbsolutePath(), "fuelV1-yolo11n.rknn"),
"Fuel v11n",
new LinkedList<String>(List.of("Fuel")),
640,
640,
Family.RUBIK,
Version.YOLOV8));
Family.RKNN,
Version.YOLOV11));
nnProps.addModelProperties(
new ModelProperties(
@@ -189,6 +179,16 @@ public class NeuralNetworkModelManager {
Family.RUBIK,
Version.YOLOV8));
nnProps.addModelProperties(
new ModelProperties(
Path.of(modelsDirectory.getAbsolutePath(), "fuelV1-yolo11n.tflite"),
"Fuel v11n",
new LinkedList<String>(List.of("Fuel")),
640,
640,
Family.RUBIK,
Version.YOLOV11));
return nnProps;
}
@@ -272,7 +272,7 @@ public class NeuralNetworkModelManager {
*
* <p>The first model in the list is the default model.
*/
private Map<Family, ArrayList<Model>> models;
Map<Family, ArrayList<Model>> models;
/**
* Retrieves the model with the specified name, assuming it is available under a supported
@@ -321,13 +321,27 @@ public class NeuralNetworkModelManager {
ConfigManager.getInstance().getConfig().neuralNetworkPropertyManager().getModel(path);
if (properties == null) {
logger.error(
logger.warn(
"Model properties are null. This could mean the config for model "
+ path
+ " was unable to be found in the database.");
return;
+ " was unable to be found in the database. Trying legacy...");
try {
properties = ModelProperties.createFromFilename(path.getFileName().toString());
// At this point this property is not serialized or known to our configuration. add to
// NeuralNetworkModelsSettings
ConfigManager.getInstance()
.getConfig()
.neuralNetworkPropertyManager()
.addModelProperties(properties);
} catch (IllegalArgumentException | IOException e) {
logger.error("Failed to translate legacy model filename to properties: " + path, e);
return;
}
}
logger.debug(properties.toString());
if (!supportedBackends.contains(properties.family())) {
logger.warn(
"Model "
@@ -412,7 +426,7 @@ public class NeuralNetworkModelManager {
File modelsDirectory = ConfigManager.getInstance().getModelsDirectory();
// Filter shippedProprties by supportedBackends
NeuralNetworkPropertyManager supportedProperties = new NeuralNetworkPropertyManager();
NeuralNetworkModelsSettings supportedProperties = new NeuralNetworkModelsSettings();
for (ModelProperties model : getShippedProperties(modelsDirectory).getModels()) {
if (supportedBackends.contains(model.family())) {
supportedProperties.addModelProperties(model);

View File

@@ -0,0 +1,277 @@
/*
* Copyright (C) Photon Vision.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package org.photonvision.common.configuration;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
public class NeuralNetworkModelsSettings {
/*
* The properties of the model. This is used to determine which model to load.
* The only families currently supported are RKNN and Rubik (custom .tflite)
*/
public record ModelProperties(
@JsonProperty("modelPath") Path modelPath,
@JsonProperty("nickname") String nickname,
@JsonProperty("labels") List<String> labels,
@JsonProperty("resolutionWidth") int resolutionWidth,
@JsonProperty("resolutionHeight") int resolutionHeight,
@JsonProperty("family") Family family,
@JsonProperty("version") Version version) {
@JsonCreator
public ModelProperties {}
ModelProperties(ModelProperties other) {
this(
other.modelPath,
other.nickname,
other.labels, // note this does not clone the underlying list
other.resolutionWidth,
other.resolutionHeight,
other.family,
other.version);
}
// In v2025.3.1, this was single string for the model path. but the first argument
// is now nickname
public ModelProperties(@JsonProperty("nickname") String filename)
throws IllegalArgumentException, IOException {
this(createFromFilename(filename));
}
// ============= Migration code from v2025.3.1 ===========
private static Pattern modelPattern =
Pattern.compile("^([a-zA-Z0-9._]+)-(\\d+)-(\\d+)-(yolov(?:5|8|11)[nsmlx]*)\\.rknn$");
static ModelProperties createFromFilename(String modelFileName)
throws IllegalArgumentException, IOException {
// Used to point to default models directory
var model =
ConfigManager.getInstance().getModelsDirectory().toPath().resolve(modelFileName).toFile();
// Get the model extension and check if it is supported
String modelExtension = model.getName().substring(model.getName().lastIndexOf('.'));
if (!modelExtension.equals(".rknn")) {
throw new IllegalArgumentException("Model " + modelFileName + " is not a supported format");
}
var backend =
Arrays.stream(NeuralNetworkModelManager.Family.values())
.filter(b -> b.extension().equals(modelExtension))
.findFirst();
if (!backend.isPresent()) {
throw new IllegalArgumentException("Model " + modelFileName + " cannot find backend");
}
String labelFile = model.getAbsolutePath().replace(backend.get().extension(), "-labels.txt");
List<String> labels = Files.readAllLines(Paths.get(labelFile));
String[] parts = parseRKNNName(modelFileName);
var version = getModelVersion(parts[3]);
int width = Integer.parseInt(parts[1]);
int height = Integer.parseInt(parts[2]);
return new ModelProperties(
model.toPath(),
model.getName(),
labels,
// all files used to be 640x640
width,
height,
Family.RKNN,
version);
}
/**
* Determines the model version based on the model's filename.
*
* <p>"yolov5" -> "YOLO_V5"
*
* <p>"yolov8" -> "YOLO_V8"
*
* <p>"yolov11" -> "YOLO_V11"
*
* @param modelName The model's filename
* @return The model version
*/
private static Version getModelVersion(String modelName) throws IllegalArgumentException {
if (modelName.contains("yolov5")) {
return Version.YOLOV5;
} else if (modelName.contains("yolov8")) {
return Version.YOLOV8;
} else if (modelName.contains("yolov11")) {
return Version.YOLOV11;
} else {
throw new IllegalArgumentException("Unknown model version for model " + modelName);
}
}
/**
* Parse RKNN name and return the name, width, height, and model type.
*
* <p>This is static as it is not dependent on the state of the class.
*
* @param modelName the name of the model
* @throws IllegalArgumentException if the model name does not follow the naming convention
* @return an array containing the name, width, height, and model type
*/
public static String[] parseRKNNName(String modelName) {
Matcher modelMatcher = modelPattern.matcher(modelName);
if (!modelMatcher.matches()) {
throw new IllegalArgumentException(
"Model name must follow the naming convention of name-widthResolution-heightResolution-modelType.rknn");
}
return new String[] {
modelMatcher.group(1), modelMatcher.group(2), modelMatcher.group(3), modelMatcher.group(4)
};
}
}
// The path to the model is used as the key in the map because it is unique to
// the model, and should not change
@JsonProperty("modelPathToProperties")
private HashMap<Path, ModelProperties> modelPathToProperties =
new HashMap<Path, ModelProperties>();
/**
* Constructor for the NeuralNetworkProperties class.
*
* <p>This object holds a LinkedList of {@link ModelProperties} objects
*/
public NeuralNetworkModelsSettings() {}
/**
* Constructor for the NeuralNetworkProperties class.
*
* <p>This object holds a LinkedList of {@link ModelProperties} objects.
*
* @param modelPropertiesList When the class is constructed, it will hold the provided list
*/
public NeuralNetworkModelsSettings(HashMap<Path, ModelProperties> modelPropertiesList) {}
@Override
public String toString() {
String toReturn = "";
toReturn += "NeuralNetworkProperties [";
toReturn += modelPathToProperties.toString() + "]";
return toReturn;
}
/**
* Add a model to the list of models.
*
* @param modelProperties
*/
public void addModelProperties(ModelProperties modelProperties) {
modelPathToProperties.put(modelProperties.modelPath, modelProperties);
}
/**
* Add two Neural Network Properties together.
*
* <p>Any properties that are the same will be overwritten by the second
*
* @param nnProps
* @return itself, so it can be chained and used fluently
*/
public NeuralNetworkModelsSettings sum(NeuralNetworkModelsSettings nnProps) {
modelPathToProperties.putAll(nnProps.modelPathToProperties);
return this;
}
/**
* Remove a model from the list of models.
*
* @param modelPath
* @return True if the model was removed, false if it was not found
*/
public boolean removeModel(Path modelPath) {
return modelPathToProperties.remove(modelPath) != null;
}
/**
* Get the model properties for a given model path.
*
* @param modelPath
* @return {@link ModelProperties} object
*/
public ModelProperties getModel(Path modelPath) {
return modelPathToProperties.get(modelPath);
}
/**
* Get all models
*
* @return A list of all models
*/
@JsonIgnore
public ModelProperties[] getModels() {
return modelPathToProperties.values().toArray(new ModelProperties[0]);
}
/**
* Change the nickname of a {@link ModelProperties} object.
*
* @param modelPath
* @param newName
* @return True if the model was found and renamed, false if it was not found
*/
public boolean renameModel(Path modelPath, String newName) {
ModelProperties temp = modelPathToProperties.get(modelPath);
if (temp != null) {
modelPathToProperties.remove(modelPath);
modelPathToProperties.put(
modelPath,
new ModelProperties(
temp.modelPath,
newName,
temp.labels,
temp.resolutionWidth,
temp.resolutionHeight,
temp.family,
temp.version));
return true;
}
return false;
}
public boolean clear() {
modelPathToProperties.clear();
return true;
}
}

View File

@@ -1,164 +0,0 @@
/*
* Copyright (C) Photon Vision.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package org.photonvision.common.configuration;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.LinkedList;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
public class NeuralNetworkPropertyManager {
/*
* The properties of the model. This is used to determine which model to load.
* The only families currently supported are RKNN and Rubik (custom .tflite)
*/
public record ModelProperties(
@JsonProperty("modelPath") Path modelPath,
@JsonProperty("nickname") String nickname,
@JsonProperty("labels") LinkedList<String> labels,
@JsonProperty("resolutionWidth") int resolutionWidth,
@JsonProperty("resolutionHeight") int resolutionHeight,
@JsonProperty("family") Family family,
@JsonProperty("version") Version version) {
@JsonCreator
public ModelProperties {
// Record constructor is automatically annotated with @JsonCreator
}
}
// The path to the model is used as the key in the map because it is unique to
// the model, and should not change
@JsonProperty("modelPathToProperties")
private HashMap<Path, ModelProperties> modelPathToProperties =
new HashMap<Path, ModelProperties>();
/**
* Constructor for the NeuralNetworkProperties class.
*
* <p>This object holds a LinkedList of {@link ModelProperties} objects
*/
public NeuralNetworkPropertyManager() {}
/**
* Constructor for the NeuralNetworkProperties class.
*
* <p>This object holds a LinkedList of {@link ModelProperties} objects.
*
* @param modelPropertiesList When the class is constructed, it will hold the provided list
*/
public NeuralNetworkPropertyManager(HashMap<Path, ModelProperties> modelPropertiesList) {}
@Override
public String toString() {
String toReturn = "";
toReturn += "NeuralNetworkProperties [";
toReturn += modelPathToProperties.toString() + "]";
return toReturn;
}
/**
* Add a model to the list of models.
*
* @param modelProperties
*/
public void addModelProperties(ModelProperties modelProperties) {
modelPathToProperties.put(modelProperties.modelPath, modelProperties);
}
/**
* Add two Neural Network Properties together.
*
* <p>Any properties that are the same will be overwritten by the second
*
* @param nnProps
* @return itself, so it can be chained and used fluently
*/
public NeuralNetworkPropertyManager sum(NeuralNetworkPropertyManager nnProps) {
modelPathToProperties.putAll(nnProps.modelPathToProperties);
return this;
}
/**
* Remove a model from the list of models.
*
* @param modelPath
* @return True if the model was removed, false if it was not found
*/
public boolean removeModel(Path modelPath) {
return modelPathToProperties.remove(modelPath) != null;
}
/**
* Get the model properties for a given model path.
*
* @param modelPath
* @return {@link ModelProperties} object
*/
public ModelProperties getModel(Path modelPath) {
return modelPathToProperties.get(modelPath);
}
/**
* Get all models
*
* @return A list of all models
*/
@JsonIgnore
public ModelProperties[] getModels() {
return modelPathToProperties.values().toArray(new ModelProperties[0]);
}
/**
* Change the nickname of a {@link ModelProperties} object.
*
* @param modelPath
* @param newName
* @return True if the model was found and renamed, false if it was not found
*/
public boolean renameModel(Path modelPath, String newName) {
ModelProperties temp = modelPathToProperties.get(modelPath);
if (temp != null) {
modelPathToProperties.remove(modelPath);
modelPathToProperties.put(
modelPath,
new ModelProperties(
temp.modelPath,
newName,
temp.labels,
temp.resolutionWidth,
temp.resolutionHeight,
temp.family,
temp.version));
return true;
}
return false;
}
public boolean clear() {
modelPathToProperties.clear();
return true;
}
}

View File

@@ -28,7 +28,7 @@ public class PhotonConfiguration {
private final HardwareSettings hardwareSettings;
private NetworkConfig networkConfig;
private AprilTagFieldLayout atfl;
private NeuralNetworkPropertyManager neuralNetworkProperties;
private NeuralNetworkModelsSettings neuralNetworkProperties;
private HashMap<String, CameraConfiguration> cameraConfigurations;
public PhotonConfiguration(
@@ -36,7 +36,7 @@ public class PhotonConfiguration {
HardwareSettings hardwareSettings,
NetworkConfig networkConfig,
AprilTagFieldLayout atfl,
NeuralNetworkPropertyManager neuralNetworkProperties) {
NeuralNetworkModelsSettings neuralNetworkProperties) {
this(
hardwareConfig,
hardwareSettings,
@@ -51,7 +51,7 @@ public class PhotonConfiguration {
HardwareSettings hardwareSettings,
NetworkConfig networkConfig,
AprilTagFieldLayout atfl,
NeuralNetworkPropertyManager neuralNetworkProperties,
NeuralNetworkModelsSettings neuralNetworkProperties,
HashMap<String, CameraConfiguration> cameraConfigurations) {
this.hardwareConfig = hardwareConfig;
this.hardwareSettings = hardwareSettings;
@@ -67,7 +67,7 @@ public class PhotonConfiguration {
new HardwareSettings(),
new NetworkConfig(),
new AprilTagFieldLayout(List.of(), 0, 0),
new NeuralNetworkPropertyManager());
new NeuralNetworkModelsSettings());
}
public HardwareConfig getHardwareConfig() {
@@ -86,7 +86,7 @@ public class PhotonConfiguration {
return atfl;
}
public NeuralNetworkPropertyManager neuralNetworkPropertyManager() {
public NeuralNetworkModelsSettings neuralNetworkPropertyManager() {
return neuralNetworkProperties;
}
@@ -98,7 +98,7 @@ public class PhotonConfiguration {
this.networkConfig = networkConfig;
}
public void setNeuralNetworkProperties(NeuralNetworkPropertyManager neuralNetworkProperties) {
public void setNeuralNetworkProperties(NeuralNetworkModelsSettings neuralNetworkProperties) {
this.neuralNetworkProperties = neuralNetworkProperties;
}
@@ -132,6 +132,16 @@ public class PhotonConfiguration {
@Override
public String toString() {
StringBuilder cameraConfigurationsString = new StringBuilder();
cameraConfigurations.forEach(
(key, value) -> {
cameraConfigurationsString
.append("\n ")
.append(key)
.append(" -> ")
.append(value.toString());
});
return "PhotonConfiguration [\n hardwareConfig="
+ hardwareConfig
+ "\n hardwareSettings="
@@ -142,8 +152,8 @@ public class PhotonConfiguration {
+ atfl
+ "\n neuralNetworkProperties="
+ neuralNetworkProperties
+ "\n cameraConfigurations="
+ cameraConfigurations
+ "\n]";
+ "\n cameraConfigurations={"
+ cameraConfigurationsString
+ "}\n]";
}
}

View File

@@ -269,7 +269,7 @@ public class SqlConfigProvider extends ConfigProvider {
} else {
logger.debug("No " + ref.getSimpleName() + " in database");
}
// either the config entry is empty or Jackson threw and exception
// either the config entry is empty or Jackson threw an exception
try {
configObj = factory.get();
logger.info("Loaded default " + ref.getSimpleName());
@@ -313,8 +313,8 @@ public class SqlConfigProvider extends ConfigProvider {
loadConfigOrDefault(
conn,
GlobalKeys.NEURAL_NETWORK_PROPERTIES,
NeuralNetworkPropertyManager.class,
NeuralNetworkPropertyManager::new);
NeuralNetworkModelsSettings.class,
NeuralNetworkModelsSettings::new);
var atfl =
loadConfigOrDefault(
conn, GlobalKeys.ATFL_CONFIG_FILE, AprilTagFieldLayout.class, this::atflDefault);
@@ -612,52 +612,65 @@ public class SqlConfigProvider extends ConfigProvider {
// Iterate over every row/"camera" in the table
while (result.next()) {
List<String> dummyList = new ArrayList<>();
String uniqueName = "";
try {
List<String> dummyList = new ArrayList<>();
var uniqueName = result.getString(Columns.CAM_UNIQUE_NAME);
uniqueName = result.getString(Columns.CAM_UNIQUE_NAME);
// A horrifying hack to keep backward compat with otherpaths
// We -really- need to delete this -stupid- otherpaths column. I hate it.
var configStr = result.getString(Columns.CAM_CONFIG_JSON);
CameraConfiguration config = JacksonUtils.deserialize(configStr, CameraConfiguration.class);
// A horrifying hack to keep backward compat with otherpaths
// We -really- need to delete this -stupid- otherpaths column. I hate it.
var configStr = result.getString(Columns.CAM_CONFIG_JSON);
CameraConfiguration config =
JacksonUtils.deserialize(configStr, CameraConfiguration.class);
if (config.matchedCameraInfo == null) {
logger.info("Legacy CameraConfiguration detected - upgrading");
if (config.matchedCameraInfo == null) {
logger.info("Legacy CameraConfiguration detected - upgrading");
// manually create the matchedCameraInfo ourselves. Need to upgrade:
// baseName, path, otherPaths, cameraType, usbvid/pid -> matchedCameraInfo
config.matchedCameraInfo =
JacksonUtils.deserialize(configStr, LegacyCameraConfigStruct.class).matchedCameraInfo;
// manually create the matchedCameraInfo ourselves. Need to upgrade:
// baseName, path, otherPaths, cameraType, usbvid/pid -> matchedCameraInfo
config.matchedCameraInfo =
JacksonUtils.deserialize(configStr, LegacyCameraConfigStruct.class)
.matchedCameraInfo;
// Except that otherPaths used to be its own column. so hack that in here as well
var otherPaths =
// Except that otherPaths used to be its own column. so hack that in here as well
var otherPaths =
JacksonUtils.deserialize(
result.getString(Columns.CAM_OTHERPATHS_JSON), String[].class);
if (config.matchedCameraInfo instanceof UsbCameraInfo usbInfo) {
usbInfo.otherPaths = otherPaths;
}
}
var driverMode =
JacksonUtils.deserialize(
result.getString(Columns.CAM_OTHERPATHS_JSON), String[].class);
if (config.matchedCameraInfo instanceof UsbCameraInfo usbInfo) {
usbInfo.otherPaths = otherPaths;
result.getString(Columns.CAM_DRIVERMODE_JSON), DriverModePipelineSettings.class);
List<?> pipelineSettings =
JacksonUtils.deserialize(
result.getString(Columns.CAM_PIPELINE_JSONS), dummyList.getClass());
List<CVPipelineSettings> loadedSettings = new ArrayList<>();
for (var setting : pipelineSettings) {
if (setting instanceof String str) {
try {
loadedSettings.add(JacksonUtils.deserialize(str, CVPipelineSettings.class));
} catch (IOException e) {
logger.error(
"Could not deserialize pipeline setting for camera " + config.nickname, e);
}
}
}
config.pipelineSettings = loadedSettings;
config.driveModeSettings = driverMode;
loadedConfigurations.put(uniqueName, config);
} catch (IOException e) {
logger.error(
"Could not deserialize camera configuration " + uniqueName + " from database!", e);
}
var driverMode =
JacksonUtils.deserialize(
result.getString(Columns.CAM_DRIVERMODE_JSON), DriverModePipelineSettings.class);
List<?> pipelineSettings =
JacksonUtils.deserialize(
result.getString(Columns.CAM_PIPELINE_JSONS), dummyList.getClass());
List<CVPipelineSettings> loadedSettings = new ArrayList<>();
for (var setting : pipelineSettings) {
if (setting instanceof String str) {
loadedSettings.add(JacksonUtils.deserialize(str, CVPipelineSettings.class));
}
}
config.pipelineSettings = loadedSettings;
config.driveModeSettings = driverMode;
loadedConfigurations.put(uniqueName, config);
}
} catch (SQLException | IOException e) {
logger.error("Err loading cameras: ", e);
} catch (SQLException e) {
logger.error("Err querying database to load cameras: ", e);
} finally {
try {
if (query != null) query.close();

View File

@@ -18,14 +18,14 @@
package org.photonvision.common.dataflow.websocket;
import java.util.List;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings;
public class UIGeneralSettings {
public UIGeneralSettings(
String version,
String gpuAcceleration,
boolean mrCalWorking,
NeuralNetworkPropertyManager.ModelProperties[] availableModels,
NeuralNetworkModelsSettings.ModelProperties[] availableModels,
List<String> supportedBackends,
String hardwareModel,
String hardwarePlatform,
@@ -45,7 +45,7 @@ public class UIGeneralSettings {
public String version;
public String gpuAcceleration;
public boolean mrCalWorking;
public NeuralNetworkPropertyManager.ModelProperties[] availableModels;
public NeuralNetworkModelsSettings.ModelProperties[] availableModels;
public List<String> supportedBackends;
public String hardwareModel;
public String hardwarePlatform;

View File

@@ -43,6 +43,7 @@ public class VisionLED implements AutoCloseable {
private VisionLEDMode currentLedMode = VisionLEDMode.kDefault;
private BooleanSupplier pipelineModeSupplier;
private boolean currentOutputState = false;
private float mappedBrightness = 0.0f;
@@ -85,10 +86,15 @@ public class VisionLED implements AutoCloseable {
public void setBrightness(int percentage) {
mappedBrightness =
(float) (MathUtils.map(percentage, 0.0, 100.0, brightnessMin, brightnessMax) / 100.0);
setInternal(currentLedMode, false);
if (currentOutputState) {
for (PwmLed led : dimmableVisionLEDs) {
led.setValue(mappedBrightness);
}
}
}
public void blink(int pulseLengthMillis, int blinkCount) {
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
blinkImpl(pulseLengthMillis, blinkCount);
int blinkDuration = pulseLengthMillis * blinkCount * 2;
TimedTaskManager.getInstance()
@@ -96,19 +102,13 @@ public class VisionLED implements AutoCloseable {
}
private void blinkImpl(int pulseLengthMillis, int blinkCount) {
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
AtomicInteger blinks = new AtomicInteger();
TimedTaskManager.getInstance()
.addTask(
blinkTaskID,
() -> {
for (LED led : visionLEDs) {
led.toggle();
}
for (PwmLed led : dimmableVisionLEDs) {
led.setValue(mappedBrightness - led.getValue());
}
if (blinks.incrementAndGet() >= blinkCount * 2) {
setStateImpl(!currentOutputState);
if (blinkCount >= 0 && blinks.incrementAndGet() >= blinkCount * 2) {
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
}
},
@@ -116,12 +116,16 @@ public class VisionLED implements AutoCloseable {
}
private void setStateImpl(boolean state) {
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
currentOutputState = state;
for (LED led : visionLEDs) {
led.setOn(state);
}
for (PwmLed led : dimmableVisionLEDs) {
led.setValue(mappedBrightness);
if (state) {
led.setValue(mappedBrightness);
} else {
led.off();
}
}
}
@@ -162,6 +166,7 @@ public class VisionLED implements AutoCloseable {
var lastLedMode = currentLedMode;
if (fromNT || currentLedMode == VisionLEDMode.kDefault) {
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
switch (newLedMode) {
case kDefault -> setStateImpl(pipelineModeSupplier.getAsBoolean());
case kOff -> setStateImpl(false);

View File

@@ -228,7 +228,8 @@ public class TestUtils {
kRobots,
kTag1_640_480,
kTag1_16h5_1280,
kTag_corner_1280;
kTag_corner_1280,
k36h11_stress_test;
public final Path path;
@@ -237,6 +238,7 @@ public class TestUtils {
var filename = this.toString().substring(1).toLowerCase();
var extension = ".jpg";
if (filename.equals("tag1_16h5_1280")) extension = ".png";
if (filename.equals("36h11_stress_test")) extension = ".png";
return Path.of("apriltag", filename + extension);
}

View File

@@ -18,6 +18,7 @@
package org.photonvision.common.util.file;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.StreamReadFeature;
import com.fasterxml.jackson.core.json.JsonReadFeature;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.DeserializationFeature;
@@ -80,6 +81,7 @@ public class JacksonUtils {
pathModule.addKeyDeserializer(Path.class, new PathKeyDeserializer());
return JsonMapper.builder()
.enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION)
.configure(JsonReadFeature.ALLOW_JAVA_COMMENTS, true)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.activateDefaultTyping(ptv, ObjectMapper.DefaultTyping.JAVA_LANG_OBJECT)

View File

@@ -167,7 +167,27 @@ public class GenericUSBCameraSettables extends VisionSourceSettables {
}
}
@Override
public void setAutoExposure(boolean cameraAutoExposure) {
if (configuration.cameraQuirks.hasQuirk(CameraQuirk.ArduOV9281Controls)
&& !cameraAutoExposure) {
// OV9281 on Linux seems to sometimes ignore our exposure requests on first boot if we're in
// manual mode. Poking the camera into and out of auto exposure seems to fix it.
try {
setAutoExposureImpl(false);
Thread.sleep(2000);
setAutoExposureImpl(true);
Thread.sleep(2000);
setAutoExposureImpl(false);
} catch (InterruptedException e) {
logger.error("Thread interrupted while setting OV9281 exposure!", e);
}
} else {
setAutoExposureImpl(cameraAutoExposure);
}
}
public void setAutoExposureImpl(boolean cameraAutoExposure) {
logger.debug("Setting auto exposure to " + cameraAutoExposure);
if (!cameraAutoExposure) {

View File

@@ -18,7 +18,7 @@
package org.photonvision.vision.objects;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
public interface Model {
public ObjectDetector load();

View File

@@ -20,7 +20,7 @@ package org.photonvision.vision.objects;
import java.util.List;
import org.opencv.core.Mat;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
import org.photonvision.vision.pipe.impl.NeuralNetworkPipeResult;
/**

View File

@@ -21,7 +21,7 @@ import java.io.File;
import org.opencv.core.Size;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
public class RknnModel implements Model {
public final File modelFile;

View File

@@ -21,7 +21,7 @@ import java.io.File;
import org.opencv.core.Size;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
public class RubikModel implements Model {
public final File modelFile;

View File

@@ -23,8 +23,8 @@ import org.photonvision.vision.frame.FrameDivisor;
public class Draw2dAprilTagsPipe extends Draw2dTargetsPipe {
public static class Draw2dAprilTagsParams extends Draw2dTargetsPipe.Draw2dTargetsParams {
public Draw2dAprilTagsParams(
boolean shouldDraw, boolean showMultipleTargets, FrameDivisor divisor) {
super(shouldDraw, showMultipleTargets, divisor);
boolean shouldDraw, int outputMaximumTargets, FrameDivisor divisor) {
super(shouldDraw, outputMaximumTargets, divisor);
// We want to show the polygon, not the rotated box
this.showRotatedBox = false;
this.showMaximumBox = false;

View File

@@ -22,9 +22,8 @@ import org.photonvision.vision.frame.FrameDivisor;
public class Draw2dArucoPipe extends Draw2dTargetsPipe {
public static class Draw2dArucoParams extends Draw2dTargetsPipe.Draw2dTargetsParams {
public Draw2dArucoParams(
boolean shouldDraw, boolean showMultipleTargets, FrameDivisor divisor) {
super(shouldDraw, showMultipleTargets, divisor);
public Draw2dArucoParams(boolean shouldDraw, int outputMaximumTargets, FrameDivisor divisor) {
super(shouldDraw, outputMaximumTargets, divisor);
// We want to show the polygon, not the rotated box
this.showRotatedBox = false;
this.showMaximumBox = false;

View File

@@ -58,14 +58,10 @@ public class Draw2dTargetsPipe
var circleColor = ColorHelper.colorToScalar(params.circleColor);
var shapeColour = ColorHelper.colorToScalar(params.shapeOutlineColour);
for (int i = 0; i < (params.showMultipleTargets ? in.getSecond().size() : 1); i++) {
for (int i = 0; i < Math.min(params.outputMaximumTargets, in.getSecond().size()); i++) {
Point[] vertices = new Point[4];
MatOfPoint contour = new MatOfPoint();
if (i != 0 && !params.showMultipleTargets) {
break;
}
TrackedTarget target = in.getSecond().get(i);
RotatedRect r = target.getMinAreaRect();
@@ -233,8 +229,7 @@ public class Draw2dTargetsPipe
public Color shapeOutlineColour = Color.MAGENTA;
public Color textColor = Color.GREEN;
public Color circleColor = Color.RED;
public final boolean showMultipleTargets;
public int outputMaximumTargets;
public final boolean shouldDraw;
public final FrameDivisor divisor;
@@ -247,10 +242,9 @@ public class Draw2dTargetsPipe
return shape != null && shape.shape.equals(ContourShape.Circle);
}
public Draw2dTargetsParams(
boolean shouldDraw, boolean showMultipleTargets, FrameDivisor divisor) {
public Draw2dTargetsParams(boolean shouldDraw, int outputMaximumTargets, FrameDivisor divisor) {
this.shouldDraw = shouldDraw;
this.showMultipleTargets = showMultipleTargets;
this.outputMaximumTargets = outputMaximumTargets;
this.divisor = divisor;
}
}

View File

@@ -17,6 +17,7 @@
package org.photonvision.vision.pipeline;
import com.fasterxml.jackson.annotation.JsonAnySetter;
import java.util.Objects;
import org.opencv.core.Point;
import org.photonvision.common.util.numbers.DoubleCouple;
@@ -41,7 +42,7 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
public boolean hueInverted = false;
public boolean outputShouldDraw = true;
public boolean outputShowMultipleTargets = false;
public int outputMaximumTargets = 20;
public DoubleCouple contourArea = new DoubleCouple(0.0, 100.0);
public DoubleCouple contourRatio = new DoubleCouple(0.0, 20.0);
@@ -90,6 +91,22 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
public int cornerDetectionSideCount = 4;
public double cornerDetectionAccuracyPercentage = 10;
/**
* Handles backward compatibility for the deprecated outputShowMultipleTargets property. When
* outputShowMultipleTargets is encountered during deserialization, it sets outputMaximumTargets
* appropriately. If outputShowMultipleTargets is false, outputMaximumTargets is set to 1.
*/
@JsonAnySetter
public void handleUnknownProperty(String name, Object value) {
// Handle the old showMultipleTargets property for backward compatibility
if ("outputShowMultipleTargets".equals(name) && value instanceof Boolean showMultipleTargets) {
if (!showMultipleTargets) {
// If showMultipleTargets is false, limit to 1 target
outputMaximumTargets = 1;
}
}
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
@@ -97,7 +114,7 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
if (!super.equals(o)) return false;
AdvancedPipelineSettings that = (AdvancedPipelineSettings) o;
return outputShouldDraw == that.outputShouldDraw
&& outputShowMultipleTargets == that.outputShowMultipleTargets
&& outputMaximumTargets == that.outputMaximumTargets
&& contourSpecklePercentage == that.contourSpecklePercentage
&& Double.compare(that.offsetDualPointAArea, offsetDualPointAArea) == 0
&& Double.compare(that.offsetDualPointBArea, offsetDualPointBArea) == 0
@@ -136,7 +153,7 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
hsvValue,
hueInverted,
outputShouldDraw,
outputShowMultipleTargets,
outputMaximumTargets,
contourArea,
contourRatio,
contourFullness,

View File

@@ -30,6 +30,9 @@ import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import org.photonvision.common.configuration.ConfigManager;
import org.photonvision.common.dataflow.structures.Packet;
import org.photonvision.common.logging.LogGroup;
import org.photonvision.common.logging.Logger;
import org.photonvision.common.util.math.MathUtils;
import org.photonvision.estimation.TargetModel;
import org.photonvision.targeting.MultiTargetPNPResult;
@@ -49,6 +52,8 @@ import org.photonvision.vision.target.TrackedTarget;
import org.photonvision.vision.target.TrackedTarget.TargetCalculationParameters;
public class AprilTagPipeline extends CVPipeline<CVPipelineResult, AprilTagPipelineSettings> {
private static final Logger logger = new Logger(AprilTagPipeline.class, LogGroup.VisionModule);
private final AprilTagDetectionPipe aprilTagDetectionPipe = new AprilTagDetectionPipe();
private final AprilTagPoseEstimatorPipe singleTagPoseEstimatorPipe =
new AprilTagPoseEstimatorPipe();
@@ -232,6 +237,12 @@ public class AprilTagPipeline extends CVPipeline<CVPipelineResult, AprilTagPipel
}
}
if (targetList.size() > Packet.MAX_ARRAY_LEN) {
logger.error(
"We have " + targetList.size() + " targets! Arbitrarily dropping some on the floor");
targetList = targetList.subList(0, Packet.MAX_ARRAY_LEN);
}
var fpsResult = calculateFPSPipe.run(null);
var fps = fpsResult.output;

View File

@@ -40,7 +40,6 @@ public class AprilTagPipelineSettings extends AdvancedPipelineSettings {
public AprilTagPipelineSettings() {
super();
pipelineType = PipelineType.AprilTag;
outputShowMultipleTargets = true;
targetModel = TargetModel.kAprilTag6p5in_36h11;
cameraExposureRaw = 20;
cameraAutoExposure = false;

View File

@@ -15,23 +15,6 @@
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
/*
* Copyright (C) Photon Vision.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package org.photonvision.vision.pipeline;
import edu.wpi.first.apriltag.AprilTagPoseEstimate;
@@ -46,6 +29,9 @@ import org.opencv.core.Mat;
import org.opencv.imgproc.Imgproc;
import org.opencv.objdetect.Objdetect;
import org.photonvision.common.configuration.ConfigManager;
import org.photonvision.common.dataflow.structures.Packet;
import org.photonvision.common.logging.LogGroup;
import org.photonvision.common.logging.Logger;
import org.photonvision.common.util.math.MathUtils;
import org.photonvision.estimation.TargetModel;
import org.photonvision.targeting.MultiTargetPNPResult;
@@ -61,6 +47,8 @@ import org.photonvision.vision.target.TrackedTarget;
import org.photonvision.vision.target.TrackedTarget.TargetCalculationParameters;
public class ArucoPipeline extends CVPipeline<CVPipelineResult, ArucoPipelineSettings> {
private static final Logger logger = new Logger(ArucoPipeline.class, LogGroup.VisionModule);
private ArucoDetectionPipe arucoDetectionPipe = new ArucoDetectionPipe();
private ArucoPoseEstimatorPipe singleTagPoseEstimatorPipe = new ArucoPoseEstimatorPipe();
private final MultiTargetPNPPipe multiTagPNPPipe = new MultiTargetPNPPipe();
@@ -237,6 +225,12 @@ public class ArucoPipeline extends CVPipeline<CVPipelineResult, ArucoPipelineSet
}
}
if (targetList.size() > Packet.MAX_ARRAY_LEN) {
logger.error(
"We have " + targetList.size() + " targets! Arbitrarily dropping some on the floor");
targetList = targetList.subList(0, Packet.MAX_ARRAY_LEN);
}
var fpsResult = calculateFPSPipe.run(null);
var fps = fpsResult.output;

View File

@@ -45,7 +45,6 @@ public class ArucoPipelineSettings extends AdvancedPipelineSettings {
public ArucoPipelineSettings() {
super();
pipelineType = PipelineType.Aruco;
outputShowMultipleTargets = true;
targetModel = TargetModel.kAprilTag6p5in_36h11;
cameraExposureRaw = 20;
cameraAutoExposure = true;

View File

@@ -26,8 +26,6 @@ import org.photonvision.vision.pipeline.result.CVPipelineResult;
public abstract class CVPipeline<R extends CVPipelineResult, S extends CVPipelineSettings>
implements Releasable {
static final int MAX_MULTI_TARGET_RESULTS = 50;
protected S settings;
protected FrameStaticProperties frameStaticProperties;
protected QuirkyCamera cameraQuirks;

View File

@@ -101,11 +101,7 @@ public class ColoredShapePipeline
sortContoursPipe.setParams(
new SortContoursPipe.SortContoursParams(
settings.contourSortMode,
settings.outputShowMultipleTargets
? MAX_MULTI_TARGET_RESULTS // TODO don't hardcode?
: 1,
frameStaticProperties));
settings.contourSortMode, settings.outputMaximumTargets, frameStaticProperties));
collect2dTargetsPipe.setParams(
new Collect2dTargetsPipe.Collect2dTargetsParams(
@@ -131,7 +127,7 @@ public class ColoredShapePipeline
Draw2dTargetsPipe.Draw2dTargetsParams draw2DTargetsParams =
new Draw2dTargetsPipe.Draw2dTargetsParams(
settings.outputShouldDraw,
settings.outputShowMultipleTargets,
settings.outputMaximumTargets,
settings.streamingFrameDivisor);
draw2DTargetsParams.showShape = true;
draw2DTargetsParams.showMaximumBox = false;

View File

@@ -82,9 +82,7 @@ public class ObjectDetectionPipeline
sortContoursPipe.setParams(
new SortContoursPipe.SortContoursParams(
settings.contourSortMode,
settings.outputShowMultipleTargets ? MAX_MULTI_TARGET_RESULTS : 1,
frameStaticProperties));
settings.contourSortMode, settings.outputMaximumTargets, frameStaticProperties));
filterContoursPipe.setParams(
new FilterObjectDetectionsPipe.FilterContoursParams(

View File

@@ -18,18 +18,18 @@
package org.photonvision.vision.pipeline;
import org.photonvision.common.configuration.NeuralNetworkModelManager;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings;
import org.photonvision.vision.objects.Model;
public class ObjectDetectionPipelineSettings extends AdvancedPipelineSettings {
public double confidence;
public double nms; // non maximal suppression
public NeuralNetworkPropertyManager.ModelProperties model;
public NeuralNetworkModelsSettings.ModelProperties model;
public ObjectDetectionPipelineSettings() {
super();
this.pipelineType = PipelineType.ObjectDetection; // TODO: FIX this
this.outputShowMultipleTargets = true;
this.outputMaximumTargets = 20;
cameraExposureRaw = 20;
cameraAutoExposure = false;
ledMode = false;

View File

@@ -58,19 +58,19 @@ public class OutputStreamPipeline {
draw2dTargetsPipe.setParams(
new Draw2dTargetsPipe.Draw2dTargetsParams(
settings.outputShouldDraw,
settings.outputShowMultipleTargets,
settings.outputMaximumTargets,
settings.streamingFrameDivisor));
draw2dAprilTagsPipe.setParams(
new Draw2dAprilTagsPipe.Draw2dAprilTagsParams(
settings.outputShouldDraw,
settings.outputShowMultipleTargets,
settings.outputMaximumTargets,
settings.streamingFrameDivisor));
draw2dArucoPipe.setParams(
new Draw2dArucoPipe.Draw2dArucoParams(
settings.outputShouldDraw,
settings.outputShowMultipleTargets,
settings.outputMaximumTargets,
settings.streamingFrameDivisor));
draw2dCrosshairPipe.setParams(

View File

@@ -85,9 +85,7 @@ public class ReflectivePipeline extends CVPipeline<CVPipelineResult, ReflectiveP
sortContoursPipe.setParams(
new SortContoursPipe.SortContoursParams(
settings.contourSortMode,
settings.outputShowMultipleTargets ? MAX_MULTI_TARGET_RESULTS : 1,
frameStaticProperties));
settings.contourSortMode, settings.outputMaximumTargets, frameStaticProperties));
collect2dTargetsPipe.setParams(
new Collect2dTargetsPipe.Collect2dTargetsParams(

View File

@@ -465,13 +465,13 @@ public class VisionModule {
pipelineSettings.cameraExposureRaw = 10; // reasonable default
}
settables.setExposureRaw(pipelineSettings.cameraExposureRaw);
try {
settables.setAutoExposure(pipelineSettings.cameraAutoExposure);
} catch (VideoException e) {
logger.error("Unable to set camera auto exposure!");
logger.error(e.toString());
}
settables.setExposureRaw(pipelineSettings.cameraExposureRaw);
if (cameraQuirks.hasQuirk(CameraQuirk.Gain)) {
// If the gain is disabled for some reason, re-enable it
if (pipelineSettings.cameraGain == -1) pipelineSettings.cameraGain = 75;

View File

@@ -25,7 +25,7 @@ import java.util.List;
import java.util.Map;
import java.util.concurrent.locks.ReentrantLock;
import org.opencv.core.Point;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
import org.photonvision.common.dataflow.DataChangeSubscriber;
import org.photonvision.common.dataflow.events.DataChangeEvent;
import org.photonvision.common.dataflow.events.IncomingWebSocketEvent;

View File

@@ -51,7 +51,7 @@ public class BenchmarkTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
@@ -105,7 +105,7 @@ public class BenchmarkTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;

View File

@@ -66,7 +66,7 @@ public class ShapeBenchmarkTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
pipeline.getSettings().contourShape = ContourShape.Custom;
@@ -87,7 +87,7 @@ public class ShapeBenchmarkTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
pipeline.getSettings().contourShape = ContourShape.Custom;
@@ -109,7 +109,7 @@ public class ShapeBenchmarkTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
pipeline.getSettings().contourShape = ContourShape.Custom;
@@ -131,7 +131,7 @@ public class ShapeBenchmarkTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
pipeline.getSettings().contourShape = ContourShape.Custom;

View File

@@ -25,13 +25,13 @@ import java.util.LinkedList;
import org.junit.jupiter.api.Test;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
import org.photonvision.common.util.file.JacksonUtils;
public class NeuralNetworkPropertyManagerTest {
@Test
void testSerialization() {
var nnpm = new NeuralNetworkPropertyManager();
var nnpm = new NeuralNetworkModelsSettings();
// Path is always serialized as absolute; for the test to pass, this must also be made absolute
nnpm.addModelProperties(
new ModelProperties(
@@ -45,7 +45,7 @@ public class NeuralNetworkPropertyManagerTest {
String result = assertDoesNotThrow(() -> JacksonUtils.serializeToString(nnpm));
var deserializedNnpm =
assertDoesNotThrow(
() -> JacksonUtils.deserialize(result, NeuralNetworkPropertyManager.class));
() -> JacksonUtils.deserialize(result, NeuralNetworkModelsSettings.class));
assertEquals(nnpm.getModels()[0], deserializedNnpm.getModels()[0]);
}
}

View File

@@ -20,23 +20,32 @@ package org.photonvision.common.configuration;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import com.fasterxml.jackson.core.JsonProcessingException;
import edu.wpi.first.cscore.UsbCameraInfo;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Collection;
import java.util.List;
import org.apache.commons.io.FileUtils;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.photonvision.common.LoadJNI;
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
import org.photonvision.common.util.TestUtils;
import org.photonvision.vision.camera.CameraQuirk;
import org.photonvision.vision.camera.PVCameraInfo;
import org.photonvision.vision.pipeline.AdvancedPipelineSettings;
import org.photonvision.vision.pipeline.AprilTagPipelineSettings;
import org.photonvision.vision.pipeline.CVPipelineSettings;
import org.photonvision.vision.pipeline.ColoredShapePipelineSettings;
import org.photonvision.vision.pipeline.ObjectDetectionPipelineSettings;
import org.photonvision.vision.pipeline.PipelineType;
import org.photonvision.vision.pipeline.ReflectivePipelineSettings;
public class SQLConfigTest {
@TempDir private static Path tmpDir;
@TempDir private Path tmpDir;
@BeforeAll
public static void init() {
@@ -84,11 +93,15 @@ public class SQLConfigTest {
}
@Test
public void testLoad2024_3_1() {
var cfgLoader =
new SqlConfigProvider(
TestUtils.getConfigDirectoriesPath(false)
.resolve("photonvision_config_from_v2024.3.1"));
public void testLoad2024_3_1() throws IOException {
// Copy the 2024.3.1 config to a temp dir
FileUtils.copyDirectory(
TestUtils.getConfigDirectoriesPath(false)
.resolve("photonvision_config_from_v2024.3.1")
.toFile(),
tmpDir.resolve("photonvision_config_from_v2024.3.1").toFile());
var cfgLoader = new SqlConfigProvider(tmpDir.resolve("photonvision_config_from_v2024.3.1"));
assertDoesNotThrow(cfgLoader::load);
@@ -104,4 +117,97 @@ public class SQLConfigTest {
.hasQuirk(c));
}
}
void common2025p3p1Assertions(PhotonConfiguration config) {
// Make sure we got 8 cameras
assertEquals(8, config.getCameraConfigurations().size());
// Make sure exactly 2 have object detection pipelines
long count =
config.getCameraConfigurations().values().stream()
.filter(
c ->
c.pipelineSettings.stream()
.anyMatch(s -> s instanceof ObjectDetectionPipelineSettings))
.count();
assertEquals(2, count);
}
@Test
public void testLoadNewNNMM() throws JsonProcessingException, IOException {
var folder = tmpDir.resolve("2025.3.1-old-nnmm");
FileUtils.copyDirectory(
TestUtils.getConfigDirectoriesPath(false).resolve("2025.3.1-old-nnmm").toFile(),
folder.toFile());
var cfgManager = new ConfigManager(folder, new SqlConfigProvider(folder));
// Replace global configmanager
ConfigManager.INSTANCE = cfgManager;
assertDoesNotThrow(cfgManager::load);
System.out.println(cfgManager.getConfig());
common2025p3p1Assertions(cfgManager.getConfig());
// And we now see two models
NeuralNetworkModelManager.getInstance();
// force us to allow RKNN
NeuralNetworkModelManager.getInstance().supportedBackends.add(Family.RKNN);
NeuralNetworkModelManager.getInstance().discoverModels();
assertEquals(5, NeuralNetworkModelManager.getInstance().models.get(Family.RKNN).size());
ConfigManager.getInstance().saveToDisk();
// Now that we have the config saved, load it again
var reloadedProvider = new SqlConfigProvider(folder);
reloadedProvider.load();
common2025p3p1Assertions(reloadedProvider.getConfig());
// And make sure NNPM has all 5 models
assertEquals(5, reloadedProvider.getConfig().neuralNetworkPropertyManager().getModels().length);
ConfigManager.INSTANCE = null;
}
@Test
public void testMaxDetectionsMigration() throws IOException {
var folder = tmpDir.resolve("2025.3.1-old-nnmm");
FileUtils.copyDirectory(
TestUtils.getConfigDirectoriesPath(false).resolve("2025.3.1-old-nnmm").toFile(),
folder.toFile());
var cfgManager = new ConfigManager(folder, new SqlConfigProvider(folder));
// Replace global configmanager
ConfigManager.INSTANCE = cfgManager;
assertDoesNotThrow(cfgManager::load);
Collection<CameraConfiguration> cameraConfigs =
cfgManager.getConfig().getCameraConfigurations().values();
for (CameraConfiguration cc : cameraConfigs) {
for (CVPipelineSettings ps : cc.pipelineSettings) {
if (ps instanceof AdvancedPipelineSettings adps) {
AdvancedPipelineSettings finalPs = adps;
if (finalPs.pipelineType.equals(PipelineType.AprilTag)
|| finalPs.pipelineType.equals(PipelineType.Aruco)) {
// Tag pipelines don't have max detections, so skip
continue;
} else if (finalPs.pipelineNickname.equals("TEST MIGRATION")) {
// This is our colored shape pipeline that we set to 1 before saving
assertEquals(1, finalPs.outputMaximumTargets);
} else {
// All others should be at default 20
assertEquals(20, finalPs.outputMaximumTargets);
}
} else {
System.out.println("Skipping pipeline settings type: " + ps.getClass().getSimpleName());
}
}
}
ConfigManager.INSTANCE = null;
}
}

View File

@@ -137,4 +137,31 @@ public class AprilTagTest {
assertEquals(2, pose.getTranslation().getY(), 0.2);
assertEquals(0.0, pose.getTranslation().getZ(), 0.2);
}
@Test
public void testManyDetections() {
// Given a 36h11 pipeline
var pipeline = new AprilTagPipeline();
pipeline.getSettings().inputShouldShow = true;
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().solvePNPEnabled = true;
pipeline.getSettings().cornerDetectionAccuracyPercentage = 4;
pipeline.getSettings().cornerDetectionUseConvexHulls = true;
pipeline.getSettings().tagFamily = AprilTagFamily.kTag36h11;
pipeline.getSettings().outputMaximumTargets = 3; // bogus
// when we have a picture with 280 targets
var frameProvider =
new FileFrameProvider(
TestUtils.getApriltagImagePath(TestUtils.ApriltagTestImages.k36h11_stress_test, false),
TestUtils.WPI2020Image.FOV,
TestUtils.getCoeffs(TestUtils.LIMELIGHT_480P_CAL_FILE, false));
frameProvider.requestFrameThresholdType(pipeline.getThresholdType());
CVPipelineResult pipelineResult;
pipelineResult = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
// the pipeline will only give us Byte.MAX_VALUE many
assertEquals(Byte.MAX_VALUE, pipelineResult.targets.size());
}
}

View File

@@ -94,7 +94,7 @@ public class CirclePNPTest {
pipeline.getSettings().cameraCalibration = getCoeffs(LIFECAM_480P_CAL_FILE);
pipeline.getSettings().targetModel = TargetModel.kCircularPowerCell7in;
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = false;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
pipeline.getSettings().contourShape = ContourShape.Circle;
@@ -144,7 +144,7 @@ public class CirclePNPTest {
settings.hsvSaturation.set(100, 255);
settings.hsvValue.set(190, 255);
settings.outputShouldDraw = true;
settings.outputShowMultipleTargets = true;
settings.outputMaximumTargets = 20;
settings.contourGroupingMode = ContourGroupingMode.Dual;
settings.contourIntersection = ContourIntersectionDirection.Up;

View File

@@ -102,7 +102,7 @@ public class ColoredShapePipelineTest {
settings.hsvSaturation.set(100, 255);
settings.hsvValue.set(100, 255);
settings.outputShouldDraw = true;
settings.outputShowMultipleTargets = true;
settings.outputMaximumTargets = 20;
settings.contourGroupingMode = ContourGroupingMode.Single;
settings.contourIntersection = ContourIntersectionDirection.Up;
settings.contourShape = ContourShape.Triangle;

View File

@@ -0,0 +1,78 @@
/*
* Copyright (C) Photon Vision.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package org.photonvision.vision.pipeline;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.photonvision.common.LoadJNI;
import org.photonvision.common.configuration.ConfigManager;
import org.photonvision.common.util.TestUtils;
import org.photonvision.vision.camera.QuirkyCamera;
import org.photonvision.vision.frame.provider.FileFrameProvider;
import org.photonvision.vision.opencv.ContourShape;
import org.photonvision.vision.pipe.impl.HSVPipe;
import org.photonvision.vision.pipeline.result.CVPipelineResult;
public class MaxDetectionsTest {
@Test
public void testMaxDetections() {
LoadJNI.loadLibraries();
ConfigManager.getInstance().load();
ColoredShapePipeline pipeline = new ColoredShapePipeline();
pipeline.settings.contourShape = ContourShape.Circle;
pipeline.settings.hsvHue.set(140, 160);
pipeline.settings.hsvSaturation.set(226, 246);
pipeline.settings.hsvValue.set(188, 208);
pipeline.settings.maxCannyThresh = 90;
pipeline.settings.circleAccuracy = 20;
pipeline.settings.circleDetectThreshold = 5;
Path path =
TestUtils.getResourcesFolderPath(false).resolve("testimages/polygons/ColoredShapeTest.png");
var frameProvider = new FileFrameProvider(path, TestUtils.WPI2019Image.FOV);
// VisionRunner normally does this
var hsvParams =
new HSVPipe.HSVParams(
pipeline.getSettings().hsvHue,
pipeline.getSettings().hsvSaturation,
pipeline.getSettings().hsvValue,
pipeline.getSettings().hueInverted);
frameProvider.requestHsvSettings(hsvParams);
frameProvider.requestFrameThresholdType(pipeline.getThresholdType());
CVPipelineResult result = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
TestUtils.showImage(result.inputAndOutputFrame.processedImage.getMat(), "Max Detections Test");
assertEquals(20, result.targets.size());
pipeline.settings.outputMaximumTargets = 5;
result = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
assertEquals(5, result.targets.size());
pipeline.settings.outputMaximumTargets = 50;
result = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
// 24 circles, but we only detect 22
assertEquals(22, result.targets.size());
}
}

View File

@@ -41,7 +41,7 @@ public class ReflectivePipelineTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().outputMaximumTargets = 20;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
@@ -120,7 +120,7 @@ public class ReflectivePipelineTest {
settings.hsvSaturation.set(100, 255);
settings.hsvValue.set(190, 255);
settings.outputShouldDraw = true;
settings.outputShowMultipleTargets = true;
settings.outputMaximumTargets = 20;
settings.contourGroupingMode = ContourGroupingMode.Dual;
settings.contourIntersection = ContourIntersectionDirection.Up;

View File

@@ -89,7 +89,6 @@ public class SolvePNPTest {
pipeline.getSettings().hsvSaturation.set(100, 255);
pipeline.getSettings().hsvValue.set(190, 255);
pipeline.getSettings().outputShouldDraw = true;
pipeline.getSettings().outputShowMultipleTargets = true;
pipeline.getSettings().solvePNPEnabled = true;
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
@@ -225,7 +224,6 @@ public class SolvePNPTest {
settings.hsvSaturation.set(100, 255);
settings.hsvValue.set(190, 255);
settings.outputShouldDraw = true;
settings.outputShowMultipleTargets = true;
settings.contourGroupingMode = ContourGroupingMode.Dual;
settings.contourIntersection = ContourIntersectionDirection.Up;

View File

@@ -142,8 +142,8 @@ PhotonCamera::PhotonCamera(nt::NetworkTableInstance instance,
rootTable->GetIntegerTopic("pipelineIndexRequest").Publish()),
pipelineIndexSub(
rootTable->GetIntegerTopic("pipelineIndexState").Subscribe(0)),
ledModePub(mainTable->GetIntegerTopic("ledMode").Publish()),
ledModeSub(mainTable->GetIntegerTopic("ledMode").Subscribe(0)),
ledModePub(mainTable->GetIntegerTopic("ledModeRequest").Publish()),
ledModeSub(mainTable->GetIntegerTopic("ledModeState").Subscribe(0)),
versionEntry(mainTable->GetStringTopic("version").Subscribe("")),
cameraIntrinsicsSubscriber(
rootTable->GetDoubleArrayTopic("cameraIntrinsics").Subscribe({})),

View File

@@ -39,7 +39,7 @@ import org.opencv.imgcodecs.Imgcodecs;
import org.photonvision.common.configuration.ConfigManager;
import org.photonvision.common.configuration.NetworkConfig;
import org.photonvision.common.configuration.NeuralNetworkModelManager;
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
import org.photonvision.common.dataflow.DataChangeDestination;
import org.photonvision.common.dataflow.DataChangeService;
import org.photonvision.common.dataflow.events.IncomingWebSocketEvent;

View File

@@ -31,6 +31,8 @@ public class Packet {
// Read and write positions.
int readPos, writePos;
public static final int MAX_ARRAY_LEN = Byte.MAX_VALUE;
/**
* Constructs an empty packet. This buffer will dynamically expand if we need more data space.
*

View File

@@ -1,84 +1,99 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "1tMAqVl4p58r"
},
"source": [
"## YOLO to Rubik TFlite Conversion"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nAbygyUYp58s"
},
"source": [
"#### Requirements\n",
"\n",
"This notebook can be run on Colab. However, Colab has some incompatibility issues that result in needing to restart the notebook in the middle of the run. This is normal, and after restarting you should rerun the below cell.\n",
"\n",
"If you aren't using Google Colab, we recommend creating a [Python venv](https://docs.python.org/3/library/venv.html) so that the packages installed for conversion do not conflict with your existing setup.\n",
"\n",
"Prior to running the notebook, it is necessary to make an account on [Qualcomm's AI Hub](https://app.aihub.qualcomm.com/account/), and obtain your API token. Then, replace <YOUR_API_TOKEN> with your API token in the cell below.\n",
"\n",
"Documentation for the Qualcomm AI Hub can be found [here](https://app.aihub.qualcomm.com/docs/index.html).\n",
"\n",
"You should also have a PyTorch model (ending in `.pt`) that's been uploaded to the runtime that you intend to convert. After uploading, copy it's absolute path by right-clicking on the file, and replace /PATH/TO/WEIGHTS.\n",
"\n",
"**NOTE: your API key will be listed in the output, and should therefore be redacted if the output is shared.**\n",
"\n",
"Once the run has finished, open the AI Hub link, and download the tflite model for the job you just ran.\n",
"\n",
"If you want to use this notebook to convert a yolo11 model, you'll need to replace all instances of `yolov8` in the cell below with `yolov11`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "1tMAqVl4p58r"
},
"source": [
"## YOLO to Rubik TFlite Conversion"
]
},
"id": "aX3JcSFKp58s",
"outputId": "f2cdadd2-c448-4d8c-c681-c19decef7f3e"
},
"outputs": [],
"source": [
"# This installs Python package\n",
"!pip install qai-hub-models[yolov8_det]\n",
"# sets up AI Hub enviroment\n",
"!qai-hub configure --api_token <YOUR_API_TOKEN>\n",
"# Converts the model to be ran on RB3Gen2\n",
"!yes | python -m qai_hub_models.models.yolov8_det.export --quantize w8a8 --device=\"RB3 Gen 2 (Proxy)\" --ckpt-name /PATH/TO/WEIGHTS --device-os linux --target-runtime tflite --output-dir .\n"
]
{
"cell_type": "markdown",
"metadata": {
"id": "nAbygyUYp58s"
},
"source": [
"#### Requirements\n",
"\n",
"This notebook can be run on Colab. However, Colab has some incompatibility issues that result in needing to restart the notebook in the middle of the run. This is normal, and after restarting you should rerun the below cell.\n",
"\n",
"If you aren't using Google Colab, we recommend creating a [Python venv](https://docs.python.org/3/library/venv.html) so that the packages installed for conversion do not conflict with your existing setup.\n",
"\n",
"Prior to running the notebook, it is necessary to make an account on [Qualcomm's AI Hub](https://app.aihub.qualcomm.com/account/), and obtain your API token. Then, replace <YOUR_API_TOKEN> with your API token in the cell below.\n",
"\n",
"Documentation for the Qualcomm AI Hub can be found [here](https://app.aihub.qualcomm.com/docs/index.html).\n",
"\n",
"You should also have a PyTorch model (ending in `.pt`) that's been uploaded to the runtime that you intend to convert. After uploading, copy it's absolute path by right-clicking on the file, and replace /PATH/TO/WEIGHTS.\n",
"\n",
"**NOTE: your API key will be listed in the output, and should therefore be redacted if the output is shared.**\n",
"\n",
"Once the run has finished, open the AI Hub link, and download the tflite model for the job you just ran.\n",
"\n",
"If you want to use this notebook to convert a yolo11 model, you'll need to replace all instances of `yolov8` in the cell below with `yolov11`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "aX3JcSFKp58s",
"outputId": "7fffa581-fb85-4808-84d6-b737142011b2"
},
"outputs": [],
"source": [
"# This installs Python package\n",
"!pip install qai-hub-models[yolov8_det]\n",
"# sets up AI Hub enviroment\n",
"!qai-hub configure --api_token [YOUR API KEY]\n",
"# Converts the model to be ran on RB3Gen2\n",
"!yes | python -m qai_hub_models.models.yolov8_det.export --quantize w8a8 --device=\"Dragonwing RB3 Gen 2 Vision Kit\" --ckpt-name /content/PyTorch_model.pt --device-os 1.6 --target-runtime tflite --output-dir .\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "jl9M3nPDqlue",
"outputId": "2df2c1af-c4ee-4cbf-dad6-7ec3afd28337"
},
"outputs": [],
"source": [
"# check valid devices / os types\n",
"!qai-hub list-devices"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0I2cXQO4p58s"
},
"source": [
"Modified from https://github.com/ramalamadingdong/yolo-rb3gen2-trainer/blob/main/AI_Hub_Quanitization_RB3Gen2.ipynb"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.11.7"
}
},
{
"cell_type": "markdown",
"metadata": {
"id": "0I2cXQO4p58s"
},
"source": [
"Modified from https://github.com/ramalamadingdong/yolo-rb3gen2-trainer/blob/main/AI_Hub_Quanitization_RB3Gen2.ipynb"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.11.7"
}
},
"nbformat": 4,
"nbformat_minor": 0
"nbformat": 4,
"nbformat_minor": 0
}

View File

@@ -44,7 +44,6 @@ dependencies {
implementation "commons-io:commons-io:2.11.0"
implementation "commons-cli:commons-cli:1.5.0"
implementation "org.apache.commons:commons-exec:1.3"
testImplementation(platform('org.junit:junit-bom:5.11.4'))
testImplementation 'org.junit.jupiter:junit-jupiter-api'

Binary file not shown.

After

Width:  |  Height:  |  Size: 178 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB