mirror of
https://github.com/PhotonVision/photonvision
synced 2026-06-19 00:41:41 +00:00
Compare commits
16 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5a87e4c738 | ||
|
|
284e818e74 | ||
|
|
f4b30da6b3 | ||
|
|
798b01c3a6 | ||
|
|
23392f8d46 | ||
|
|
09e6d45e77 | ||
|
|
77457219c7 | ||
|
|
da88867c60 | ||
|
|
a39844328d | ||
|
|
1b5f4fa802 | ||
|
|
7cc22e52ea | ||
|
|
49629afe9b | ||
|
|
ae74b171aa | ||
|
|
cfd5773e7c | ||
|
|
c348f0e3ba | ||
|
|
4139566514 |
10
.github/pull_request_template.md
vendored
10
.github/pull_request_template.md
vendored
@@ -1,18 +1,18 @@
|
||||
## Description
|
||||
|
||||
<!-- What changed? Why? (the code + comments should speak for itself on the "how") -->
|
||||
What changed? Why? (the code + comments should speak for itself on the "how")
|
||||
|
||||
<!-- Fun screenshots or a cool video or something are super helpful as well. If this touches platform-specific behavior, this is where test evidence should be collected. -->
|
||||
Include fun testing screenshots or a cool video, to collect test evidence in a place where we can later reference it. Including proof this change was tested makes reviewing easier, helps us make sure we tested all our edge cases, and helps provide context for the future.
|
||||
|
||||
<!-- Any issues this pull request closes or pull requests this supersedes should be linked with `Closes #issuenumber`. -->
|
||||
Any issues this pull request closes or pull requests this supersedes should be linked with `Closes #issuenumber`.
|
||||
|
||||
## Meta
|
||||
|
||||
Merge checklist:
|
||||
- [ ] Pull Request title is [short, imperative summary](https://cbea.ms/git-commit/) of proposed changes
|
||||
- [ ] The description documents the _what_ and _why_
|
||||
- [ ] The description documents the _what_ and _why_, including events that led to this PR
|
||||
- [ ] If this PR changes behavior or adds a feature, user documentation is updated
|
||||
- [ ] If this PR touches photon-serde, all messages have been regenerated and hashes have not changed unexpectedly
|
||||
- [ ] If this PR touches configuration, this is backwards compatible with settings back to v2025.3.2
|
||||
- [ ] If this PR touches configuration, this is backwards compatible with all settings going back to the previous seasons's last release (seasons end after champs ends)
|
||||
- [ ] If this PR touches pipeline settings or anything related to data exchange, the frontend typing is updated
|
||||
- [ ] If this PR addresses a bug, a regression test for it is added
|
||||
|
||||
2
.github/workflows/build.yml
vendored
2
.github/workflows/build.yml
vendored
@@ -10,7 +10,7 @@ concurrency:
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
IMAGE_VERSION: v2026.1.1
|
||||
IMAGE_VERSION: v2026.1.2
|
||||
|
||||
jobs:
|
||||
|
||||
|
||||
38
README.md
38
README.md
@@ -8,18 +8,18 @@ The latest release of platform-specific jars and images is found [here](https://
|
||||
|
||||
If you are interested in contributing code or documentation to the project, please [read our getting started page for contributors](https://docs.photonvision.org/en/latest/docs/contributing/index.html) and **[join the Discord](https://discord.gg/wYxTwym) to introduce yourself!** We hope to provide a welcoming community to anyone who is interested in helping.
|
||||
|
||||
## Authors
|
||||
|
||||
<a href="https://github.com/PhotonVision/photonvision/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=PhotonVision/photonvision" />
|
||||
</a>
|
||||
|
||||
## Documentation
|
||||
|
||||
- Our main documentation page: [docs.photonvision.org](https://docs.photonvision.org)
|
||||
- Photon UI demo: [demo.photonvision.org](https://demo.photonvision.org)
|
||||
- Javadocs: [javadocs.photonvision.org](https://javadocs.photonvision.org)
|
||||
- C++ Doxygen [cppdocs.photonvision.org](https://cppdocs.photonvision.org)
|
||||
- C++ Doxygen: [cppdocs.photonvision.org](https://cppdocs.photonvision.org)
|
||||
|
||||
## Authors
|
||||
|
||||
<a href="https://github.com/PhotonVision/photonvision/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=PhotonVision/photonvision" />
|
||||
</a>
|
||||
|
||||
## Building
|
||||
|
||||
@@ -32,7 +32,6 @@ You can run one of the many built in examples straight from the command line, to
|
||||
Note that these are case sensitive!
|
||||
|
||||
* `-PArchOverride=foobar`: builds for a target system other than your current architecture. [Valid overrides](https://github.com/wpilibsuite/wpilib-tool-plugin/blob/main/src/main/java/edu/wpi/first/tools/NativePlatforms.java) are:
|
||||
* winx32
|
||||
* winx64
|
||||
* winarm64
|
||||
* macx64
|
||||
@@ -46,33 +45,34 @@ Note that these are case sensitive!
|
||||
- `-Pprofile`: enables JVM profiling
|
||||
- `-PwithSanitizers`: On Linux, enables `-fsanitize=address,undefined,leak`
|
||||
|
||||
If you're cross-compiling, you'll need the wpilib toolchain installed. This can be done via Gradle: for example `./gradlew installArm64Toolchain` or `./gradlew installRoboRioToolchain`
|
||||
If you're cross-compiling, you'll need the WPILib toolchain installed. This must be done via Gradle: for example `./gradlew installArm64Toolchain` or `./gradlew installRoboRioToolchain`
|
||||
|
||||
## Out-of-Source Dependencies
|
||||
|
||||
PhotonVision uses the following additional out-of-source repositories for building code.
|
||||
|
||||
- Base system images for Raspberry Pi & Orange Pi: https://github.com/PhotonVision/photon-image-modifier
|
||||
- Base system images for supported coprocessors: https://github.com/PhotonVision/photon-image-modifier
|
||||
- C++ driver for Raspberry Pi CSI cameras: https://github.com/PhotonVision/photon-libcamera-gl-driver
|
||||
- JNI code for [mrcal](https://mrcal.secretsauce.net/): https://github.com/PhotonVision/mrcal-java
|
||||
- Custom build of OpenCV with GStreamer/Protobuf/other custom flags: https://github.com/PhotonVision/thirdparty-opencv
|
||||
- JNI code for aruco-nano: https://github.com/PhotonVision/aruconano-jni
|
||||
- JNI code for RKNN: https://github.com/PhotonVision/rknn_jni
|
||||
- JNI code for Rubik Pi NPU: https://github.com/PhotonVision/rubik_jni
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
PhotonVision was forked from [Chameleon Vision](https://github.com/Chameleon-Vision/chameleon-vision/). Thank you to everyone who worked on the original project.
|
||||
|
||||
* [WPILib](https://github.com/wpilibsuite) - Specifically [cscore](https://github.com/wpilibsuite/allwpilib/tree/main/cscore), [CameraServer](https://github.com/wpilibsuite/allwpilib/tree/main/cameraserver), [NTCore](https://github.com/wpilibsuite/allwpilib/tree/main/ntcore), and [OpenCV](https://github.com/wpilibsuite/thirdparty-opencv).
|
||||
|
||||
* [Apache Commons](https://commons.apache.org/) - Specifically [Commons Math](https://commons.apache.org/proper/commons-math/), and [Commons Lang](https://commons.apache.org/proper/commons-lang/)
|
||||
|
||||
* [WPILib](https://github.com/wpilibsuite) - Specifically [allwpilib](https://github.com/wpilibsuite/allwpilib) and [their build of OpenCV](https://github.com/wpilibsuite/thirdparty-opencv).
|
||||
* [Apache Commons](https://commons.apache.org/) - Specifically [Commons IO](https://commons.apache.org/proper/commons-io/), and [Commons CLI](https://commons.apache.org/proper/commons-cli/)
|
||||
* [diozero](https://www.diozero.com/)
|
||||
* [EJML](https://github.com/lessthanoptimal/ejml)
|
||||
* [Javalin](https://javalin.io/)
|
||||
|
||||
* [JSON](https://json.org)
|
||||
|
||||
* [FasterXML](https://github.com/FasterXML) - Specifically [jackson](https://github.com/FasterXML/jackson)
|
||||
|
||||
* [MessagePack for Java](https://github.com/msgpack/msgpack-java)
|
||||
* [OSHI](https://github.com/oshi/oshi)
|
||||
* [QuickBuffers](https://github.com/HebiRobotics/QuickBuffers)
|
||||
* [SQLite JDBC](https://github.com/xerial/sqlite-jdbc)
|
||||
* [ZT ZIP](https://github.com/zeroturnaround/zt-zip)
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -3,6 +3,6 @@
|
||||
"supportURL" : "https://limelightvision.io",
|
||||
"ledPins" : [ 13, 18 ],
|
||||
"ledsCanDim" : true,
|
||||
"ledPWMFrequency" : 30000,
|
||||
"ledPWMFrequency" : 1000,
|
||||
"vendorFOV" : 75.76079874010732
|
||||
}
|
||||
|
||||
@@ -18,7 +18,7 @@ The {code}`Drivetrain` class includes functionality to fuse multiple sensor read
|
||||
|
||||
Please reference the [WPILib documentation](https://docs.wpilib.org/en/stable/docs/software/advanced-controls/state-space/state-space-pose_state-estimators.html) on using the {code}`SwerveDrivePoseEstimator` class.
|
||||
|
||||
We use the 2024 game's AprilTag Locations:
|
||||
We use the current game's AprilTag Locations:
|
||||
|
||||
```{eval-rst}
|
||||
.. tab-set-code::
|
||||
|
||||
@@ -73,27 +73,21 @@ If you were using custom LED commands from 2025 or earlier and still need custom
|
||||
|
||||
## Hardware Interaction Commands
|
||||
|
||||
For Non-Raspberry-Pi hardware, users must provide valid hardware-specific commands for some parts of the UI interaction (including performance metrics, and executing system restarts).
|
||||
For non-Linux hardware, users must provide the hardware-specific command for executing system restarts.
|
||||
|
||||
Leaving a command blank will disable the associated functionality.
|
||||
Leaving this command blank will disable the restart functionality.
|
||||
|
||||
```{eval-rst}
|
||||
.. tab-set-code::
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"cpuTempCommand" : "",
|
||||
"cpuMemoryCommand" : "",
|
||||
"cpuUtilCommand" : "",
|
||||
"gpuMemoryCommand" : "",
|
||||
"gpuTempCommand" : "",
|
||||
"ramUtilCommand" : "",
|
||||
"restartHardwareCommand" : "",
|
||||
}
|
||||
```
|
||||
|
||||
:::{note}
|
||||
These settings have no effect if PhotonVision detects it is running on a Raspberry Pi. See [the MetricsBase class](https://github.com/PhotonVision/photonvision/blob/dbd631da61b7c86b70fa6574c2565ad57d80a91a/photon-core/src/main/java/org/photonvision/common/hardware/metrics/MetricsBase.java) for the commands utilized.
|
||||
This setting has no effect if PhotonVision detects it is running on Linux. On Linux, the restart is accomplished by executing `reboot now` in a shell.
|
||||
:::
|
||||
|
||||
## Known Camera FOV
|
||||
@@ -150,13 +144,7 @@ Here is a complete example `hardwareConfig.json`:
|
||||
"setGPIOCommand" : "setGPIO {p} {s}",
|
||||
"setPWMCommand" : "setPWM {p} {v}",
|
||||
"setPWMFrequencyCommand" : "setPWMFrequency {p} {f}",
|
||||
"releaseGPIOCommand" : "releseGPIO {p}",
|
||||
"cpuTempCommand" : "",
|
||||
"cpuMemoryCommand" : "",
|
||||
"cpuUtilCommand" : "",
|
||||
"gpuMemoryCommand" : "",
|
||||
"gpuTempCommand" : "",
|
||||
"ramUtilCommand" : "",
|
||||
"releaseGPIOCommand" : "releaseGPIO {p}",
|
||||
"restartHardwareCommand" : "",
|
||||
"vendorFOV" : 72.5
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
PhotonVision supports object detection using neural network accelerator hardware, commonly known as an NPU. The two coprocessors currently supported are the {ref}`Orange Pi 5 <docs/objectDetection/opi:Orange Pi 5 (and variants) Object Detection>` and the {ref}`Rubik Pi 3 <docs/objectDetection/rubik:Rubik Pi 3 Object Detection>`.
|
||||
|
||||
PhotonVision currently ships with a model trained on the [COCO dataset](https://cocodataset.org/) by [Ultralytics](https://github.com/ultralytics/ultralytics) (this model is licensed under [AGPLv3](https://www.gnu.org/licenses/agpl-3.0.en.html)). This model is meant to be used for testing and other miscellaneous purposes. It is not meant to be used in competition. For the 2025 post-season, PhotonVision also ships with a pretrained ALGAE model. A model to detect coral is available in the PhotonVision discord, but will not be distributed with PhotonVision.
|
||||
PhotonVision currently ships with a model trained on the [COCO dataset](https://cocodataset.org/) by [Ultralytics](https://github.com/ultralytics/ultralytics) (this model is licensed under [AGPLv3](https://www.gnu.org/licenses/agpl-3.0.en.html)). This model is meant to be used for testing and other miscellaneous purposes. It is not meant to be used in competition. For the 2026 season, PhotonVision ships with a model to detect FUEL, this is also licensed under AGPL.
|
||||
|
||||
## Tracking Objects
|
||||
|
||||
|
||||
@@ -28,6 +28,10 @@ Unless otherwise noted in release notes or if updating from the prior years vers
|
||||
|
||||
Use the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) to flash the image onto the coprocessors microSD card. Select the downloaded `.img.xz` file, select your microSD card, and flash.
|
||||
|
||||
:::{warning}
|
||||
Avoid using Raspberry Pi Imager version 2.0.2 or later. Those versions fail to write the image to an SD card. Versions 2.0.0 and earlier write images successfully. [GitHub issue 1489](https://github.com/raspberrypi/rpi-imager/issues/1489) was created for this problem.
|
||||
:::
|
||||
|
||||
:::{warning}
|
||||
Balena Etcher has been recommended in the past, but should no longer be used due to instability and lack of ongoing support from developers.
|
||||
:::
|
||||
|
||||
@@ -44,7 +44,7 @@ let renderer: WebGLRenderer | undefined;
|
||||
let controls: TrackballControls | undefined;
|
||||
|
||||
let previousTargets: Object3D[] = [];
|
||||
const drawTargets = (targets: PhotonTarget[]) => {
|
||||
const drawTargets = async (targets: PhotonTarget[]) => {
|
||||
// Check here, since if we check in watchEffect this never gets called
|
||||
if (!scene || !camera || !renderer || !controls) {
|
||||
return;
|
||||
@@ -89,7 +89,11 @@ const drawTargets = (targets: PhotonTarget[]) => {
|
||||
|
||||
if (calibrationCoeffs) {
|
||||
// And show camera frustum
|
||||
const calibCamera = createPerspectiveCamera(calibrationCoeffs.resolution, calibrationCoeffs.cameraIntrinsics, 10);
|
||||
const calibCamera = await createPerspectiveCamera(
|
||||
calibrationCoeffs.resolution,
|
||||
calibrationCoeffs.cameraIntrinsics,
|
||||
10
|
||||
);
|
||||
const helper = new CameraHelper(calibCamera);
|
||||
const helperGroup = new Group();
|
||||
helperGroup.add(helper);
|
||||
|
||||
@@ -65,7 +65,7 @@ const createChessboard = (obs: BoardObservation, cal: CameraCalibrationResult):
|
||||
|
||||
let previousTargets: Object3D[] = [];
|
||||
let baseAspect: number | undefined;
|
||||
const drawCalibration = (cal: CameraCalibrationResult | null) => {
|
||||
const drawCalibration = async (cal: CameraCalibrationResult | null) => {
|
||||
// Check here, since if we check in watchEffect this never gets called
|
||||
if (!cal || !scene || !camera || !renderer || !controls) {
|
||||
return;
|
||||
@@ -95,7 +95,7 @@ const drawCalibration = (cal: CameraCalibrationResult | null) => {
|
||||
});
|
||||
|
||||
// And show camera frustum
|
||||
const calibCamera = createPerspectiveCamera(props.resolution, cal.cameraIntrinsics);
|
||||
const calibCamera = await createPerspectiveCamera(props.resolution, cal.cameraIntrinsics);
|
||||
const helper = new CameraHelper(calibCamera);
|
||||
|
||||
// Flip to +Z forward
|
||||
|
||||
@@ -14,6 +14,7 @@ import { getResolutionString, resolutionsAreEqual } from "@/lib/PhotonUtils";
|
||||
import CameraCalibrationInfoCard from "@/components/cameras/CameraCalibrationInfoCard.vue";
|
||||
import { useSettingsStore } from "@/stores/settings/GeneralSettingsStore";
|
||||
import { useTheme } from "vuetify";
|
||||
import TooltippedLabel from "@/components/common/pv-tooltipped-label.vue";
|
||||
|
||||
const PromptRegular = import("@/assets/fonts/PromptRegular");
|
||||
const jspdf = import("jspdf");
|
||||
@@ -243,7 +244,14 @@ const setSelectedVideoFormat = (format: VideoFormat) => {
|
||||
<v-card class="mb-3 rounded-12" color="surface" dark>
|
||||
<v-card-title>Camera Calibration</v-card-title>
|
||||
<v-card-text v-if="!isCalibrating" class="pb-0">
|
||||
<v-card-subtitle class="pa-0 pb-3 text-white">Current Calibrations</v-card-subtitle>
|
||||
<div class="pb-3">
|
||||
<tooltipped-label
|
||||
label="Curent Calibrations"
|
||||
icon="mdi-information"
|
||||
location="top"
|
||||
tooltip="Click on a resolution to view detailed calibration information and import/export a calibration."
|
||||
/>
|
||||
</div>
|
||||
<v-table fixed-header height="100%" density="compact">
|
||||
<thead>
|
||||
<tr>
|
||||
@@ -282,22 +290,10 @@ const setSelectedVideoFormat = (format: VideoFormat) => {
|
||||
</v-card-text>
|
||||
<v-card-text class="pt-0">
|
||||
<div v-if="useCameraSettingsStore().isConnected" class="d-flex flex-column">
|
||||
<v-card-subtitle v-if="!isCalibrating" class="pl-0 pb-3 pt-3 text-white"
|
||||
<v-card-subtitle v-if="!isCalibrating" class="pl-0 pb-3 pt-4 opacity-100"
|
||||
>Configure New Calibration</v-card-subtitle
|
||||
>
|
||||
<v-form ref="form" v-model="settingsValid">
|
||||
<v-alert
|
||||
closable
|
||||
density="compact"
|
||||
:variant="theme.global.name.value === 'LightTheme' ? 'elevated' : 'tonal'"
|
||||
:color="useSettingsStore().general.mrCalWorking ? 'buttonPassive' : 'error'"
|
||||
:icon="useSettingsStore().general.mrCalWorking ? 'mdi-check' : 'mdi-close'"
|
||||
:text="
|
||||
useSettingsStore().general.mrCalWorking
|
||||
? 'Mrcal was successfully loaded and will be used!'
|
||||
: 'MrCal failed to load, check journalctl logs for details.'
|
||||
"
|
||||
/>
|
||||
<pv-select
|
||||
v-model="uniqueVideoResolutionString"
|
||||
label="Resolution"
|
||||
@@ -470,7 +466,20 @@ const setSelectedVideoFormat = (format: VideoFormat) => {
|
||||
"
|
||||
/>
|
||||
</div>
|
||||
<div v-if="isCalibrating" class="d-flex justify-center align-center pt-10px pb-5">
|
||||
<v-alert
|
||||
closable
|
||||
density="compact"
|
||||
class="mb-5"
|
||||
:variant="theme.global.name.value === 'LightTheme' ? 'elevated' : 'tonal'"
|
||||
:color="useSettingsStore().general.mrCalWorking ? 'buttonPassive' : 'error'"
|
||||
:icon="useSettingsStore().general.mrCalWorking ? 'mdi-check' : 'mdi-close'"
|
||||
:text="
|
||||
useSettingsStore().general.mrCalWorking
|
||||
? 'Mrcal was successfully loaded and will be used!'
|
||||
: 'MrCal failed to load, check journalctl logs for details.'
|
||||
"
|
||||
/>
|
||||
<div v-if="isCalibrating" class="d-flex justify-center align-center pb-5">
|
||||
<v-chip
|
||||
:variant="theme.global.name.value === 'LightTheme' ? 'elevated' : 'tonal'"
|
||||
label
|
||||
|
||||
@@ -3,6 +3,7 @@ import PvSelect from "@/components/common/pv-select.vue";
|
||||
import { useCameraSettingsStore } from "@/stores/settings/CameraSettingsStore";
|
||||
import { type ActivePipelineSettings, PipelineType, RobotOffsetPointMode } from "@/types/PipelineTypes";
|
||||
import PvSwitch from "@/components/common/pv-switch.vue";
|
||||
import PvSlider from "@/components/common/pv-slider.vue";
|
||||
import { computed } from "vue";
|
||||
import { RobotOffsetType } from "@/types/SettingTypes";
|
||||
import { useStateStore } from "@/stores/StateStore";
|
||||
@@ -58,14 +59,17 @@ const interactiveCols = computed(() =>
|
||||
|
||||
<template>
|
||||
<div>
|
||||
<pv-switch
|
||||
v-model="useCameraSettingsStore().currentPipelineSettings.outputShowMultipleTargets"
|
||||
label="Show Multiple Targets"
|
||||
tooltip="If enabled, up to five targets will be displayed and sent via PhotonLib, instead of just one"
|
||||
:disabled="isTagPipeline"
|
||||
<pv-slider
|
||||
v-model="useCameraSettingsStore().currentPipelineSettings.outputMaximumTargets"
|
||||
label="Maximum Targets"
|
||||
tooltip="The maximum number of targets to display and send."
|
||||
:hidden="isTagPipeline"
|
||||
:min="1"
|
||||
:max="127"
|
||||
:step="1"
|
||||
:switch-cols="interactiveCols"
|
||||
@update:modelValue="
|
||||
(value) => useCameraSettingsStore().changeCurrentPipelineSetting({ outputShowMultipleTargets: value }, false)
|
||||
(value) => useCameraSettingsStore().changeCurrentPipelineSetting({ outputMaximumTargets: value }, false)
|
||||
"
|
||||
/>
|
||||
<pv-switch
|
||||
|
||||
@@ -13,13 +13,15 @@ import { metricsHistorySnapshot } from "@/stores/settings/GeneralSettingsStore";
|
||||
|
||||
const theme = useTheme();
|
||||
|
||||
const restartProgram = () => {
|
||||
axiosPost("/utils/restartProgram", "restart PhotonVision");
|
||||
forceReloadPage();
|
||||
const restartProgram = async () => {
|
||||
if (await axiosPost("/utils/restartProgram", "restart PhotonVision")) {
|
||||
forceReloadPage();
|
||||
}
|
||||
};
|
||||
const restartDevice = () => {
|
||||
axiosPost("/utils/restartDevice", "restart the device");
|
||||
forceReloadPage();
|
||||
const restartDevice = async () => {
|
||||
if (await axiosPost("/utils/restartDevice", "restart the device")) {
|
||||
forceReloadPage();
|
||||
}
|
||||
};
|
||||
|
||||
const address = inject<string>("backendHost");
|
||||
@@ -38,28 +40,30 @@ const handleOfflineUpdate = async () => {
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
await axiosPost("/utils/offlineUpdate", "upload new software", formData, {
|
||||
headers: { "Content-Type": "multipart/form-data" },
|
||||
onUploadProgress: ({ progress }) => {
|
||||
const uploadPercentage = (progress || 0) * 100.0;
|
||||
if (uploadPercentage < 99.5) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "New Software Upload in Progress",
|
||||
color: "secondary",
|
||||
timeout: -1,
|
||||
progressBar: uploadPercentage,
|
||||
progressBarColor: "primary"
|
||||
});
|
||||
} else {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Installing uploaded software...",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
if (
|
||||
await axiosPost("/utils/offlineUpdate", "upload new software", formData, {
|
||||
headers: { "Content-Type": "multipart/form-data" },
|
||||
onUploadProgress: ({ progress }) => {
|
||||
const uploadPercentage = (progress || 0) * 100.0;
|
||||
if (uploadPercentage < 99.5) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "New Software Upload in Progress",
|
||||
color: "secondary",
|
||||
timeout: -1,
|
||||
progressBar: uploadPercentage,
|
||||
progressBarColor: "primary"
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
forceReloadPage();
|
||||
})
|
||||
) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Installing uploaded software...",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
forceReloadPage();
|
||||
}
|
||||
};
|
||||
|
||||
const exportLogFile = ref();
|
||||
@@ -116,9 +120,10 @@ const handleSettingsImport = () => {
|
||||
};
|
||||
|
||||
const showFactoryReset = ref(false);
|
||||
const nukePhotonConfigDirectory = () => {
|
||||
axiosPost("/utils/nukeConfigDirectory", "delete the config directory");
|
||||
forceReloadPage();
|
||||
const nukePhotonConfigDirectory = async () => {
|
||||
if (await axiosPost("/utils/nukeConfigDirectory", "delete the config directory")) {
|
||||
forceReloadPage();
|
||||
}
|
||||
};
|
||||
|
||||
interface MetricItem {
|
||||
@@ -371,21 +376,33 @@ watch(metricsHistorySnapshot, () => {
|
||||
<span>CPU Usage</span>
|
||||
<span>{{ Math.round(cpuUsageData.at(-1)?.value ?? 0) }}%</span>
|
||||
</div>
|
||||
<MetricsChart id="chart" :data="cpuUsageData" type="percentage" :min="0" :max="100" color="blue" />
|
||||
<Suspense>
|
||||
<!-- Allows us to import echarts when it's actually needed -->
|
||||
<MetricsChart id="chart" :data="cpuUsageData" type="percentage" :min="0" :max="100" color="blue" />
|
||||
<template #fallback> Loading... </template>
|
||||
</Suspense>
|
||||
</v-card-text>
|
||||
<v-card-text class="pt-0 flex-0-0 pb-2">
|
||||
<div class="d-flex justify-space-between pb-3 pt-3">
|
||||
<span>CPU Memory Usage</span>
|
||||
<span>{{ Math.round(cpuMemoryUsageData.at(-1)?.value ?? 0) }}%</span>
|
||||
</div>
|
||||
<MetricsChart id="chart" :data="cpuMemoryUsageData" type="percentage" :min="0" :max="100" color="purple" />
|
||||
<Suspense>
|
||||
<!-- Allows us to import echarts when it's actually needed -->
|
||||
<MetricsChart id="chart" :data="cpuMemoryUsageData" type="percentage" :min="0" :max="100" color="purple" />
|
||||
<template #fallback> Loading... </template>
|
||||
</Suspense>
|
||||
</v-card-text>
|
||||
<v-card-text class="pt-0 flex-0-0 pb-2">
|
||||
<div class="d-flex justify-space-between pb-3 pt-3">
|
||||
<span>CPU Temperature</span>
|
||||
<span>{{ cpuTempData.at(-1)?.value == -1 ? "--- " : Math.round(cpuTempData.at(-1)?.value ?? 0) }}°C</span>
|
||||
</div>
|
||||
<MetricsChart id="chart" :data="cpuTempData" type="temperature" color="red" />
|
||||
<Suspense>
|
||||
<!-- Allows us to import echarts when it's actually needed -->
|
||||
<MetricsChart id="chart" :data="cpuTempData" type="temperature" color="red" />
|
||||
<template #fallback> Loading... </template>
|
||||
</Suspense>
|
||||
</v-card-text>
|
||||
<v-card-text class="pt-0 flex-0-0">
|
||||
<div class="d-flex justify-space-between pb-3 pt-3">
|
||||
@@ -399,7 +416,11 @@ watch(metricsHistorySnapshot, () => {
|
||||
>{{ networkUsageData.at(-1)?.value == -1 ? "---" : networkUsageData.at(-1)?.value.toFixed(3) }} Mb/s</span
|
||||
>
|
||||
</div>
|
||||
<MetricsChart id="chart" :data="networkUsageData" type="mb" :min="0" :max="10" color="green" />
|
||||
<Suspense>
|
||||
<!-- Allows us to import echarts when it's actually needed -->
|
||||
<MetricsChart id="chart" :data="networkUsageData" type="mb" :min="0" :max="10" color="green" />
|
||||
<template #fallback> Loading... </template>
|
||||
</Suspense>
|
||||
</v-card-text>
|
||||
</v-card>
|
||||
</v-col>
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
<script setup lang="ts">
|
||||
import * as echarts from "echarts";
|
||||
import { onMounted, ref, onBeforeUnmount, watch } from "vue";
|
||||
import { useTheme } from "vuetify";
|
||||
|
||||
@@ -198,7 +197,8 @@ const props = defineProps<{
|
||||
color?: string;
|
||||
}>();
|
||||
|
||||
onMounted(() => {
|
||||
onMounted(async () => {
|
||||
const echarts = await import("echarts");
|
||||
chart = echarts.init(chartRef.value);
|
||||
chart.setOption(getOptions(props.data));
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ const importWidth = ref<number | null>(null);
|
||||
const importVersion = ref<string | null>(null);
|
||||
|
||||
// TODO gray out the button when model is uploading
|
||||
const handleImport = () => {
|
||||
const handleImport = async () => {
|
||||
if (importModelFile.value === null) return;
|
||||
|
||||
const formData = new FormData();
|
||||
@@ -43,25 +43,27 @@ const handleImport = () => {
|
||||
timeout: -1
|
||||
});
|
||||
|
||||
axiosPost("/objectdetection/import", "import an object detection model", formData, {
|
||||
headers: { "Content-Type": "multipart/form-data" },
|
||||
onUploadProgress: ({ progress }) => {
|
||||
const uploadPercentage = (progress || 0) * 100.0;
|
||||
if (uploadPercentage < 99.5) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Object Detection Model Upload in Process, " + uploadPercentage.toFixed(2) + "% complete",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
} else {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Processing uploaded Object Detection Model...",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
if (
|
||||
await axiosPost("/objectdetection/import", "import an object detection model", formData, {
|
||||
headers: { "Content-Type": "multipart/form-data" },
|
||||
onUploadProgress: ({ progress }) => {
|
||||
const uploadPercentage = (progress || 0) * 100.0;
|
||||
if (uploadPercentage < 99.5) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Object Detection Model Upload in Process, " + uploadPercentage.toFixed(2) + "% complete",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
})
|
||||
) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Processing uploaded Object Detection Model...",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
}
|
||||
|
||||
showImportDialog.value = false;
|
||||
|
||||
@@ -121,33 +123,35 @@ const nukeModels = () => {
|
||||
|
||||
const showBulkImportDialog = ref(false);
|
||||
const importFile = ref<File | null>(null);
|
||||
const handleBulkImport = () => {
|
||||
const handleBulkImport = async () => {
|
||||
if (importFile.value === null) return;
|
||||
|
||||
const formData = new FormData();
|
||||
formData.append("data", importFile.value);
|
||||
|
||||
axiosPost("/objectdetection/bulkimport", "import object detection models", formData, {
|
||||
headers: { "Content-Type": "multipart/form-data" },
|
||||
onUploadProgress: ({ progress }) => {
|
||||
const uploadPercentage = (progress || 0) * 100.0;
|
||||
if (uploadPercentage < 99.5) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Object Detection Models Upload in Progress",
|
||||
color: "secondary",
|
||||
timeout: -1,
|
||||
progressBar: uploadPercentage,
|
||||
progressBarColor: "primary"
|
||||
});
|
||||
} else {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Importing New Object Detection Models...",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
if (
|
||||
await axiosPost("/objectdetection/bulkimport", "import object detection models", formData, {
|
||||
headers: { "Content-Type": "multipart/form-data" },
|
||||
onUploadProgress: ({ progress }) => {
|
||||
const uploadPercentage = (progress || 0) * 100.0;
|
||||
if (uploadPercentage < 99.5) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Object Detection Models Upload in Progress",
|
||||
color: "secondary",
|
||||
timeout: -1,
|
||||
progressBar: uploadPercentage,
|
||||
progressBarColor: "primary"
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
})
|
||||
) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Importing New Object Detection Models...",
|
||||
color: "secondary",
|
||||
timeout: -1
|
||||
});
|
||||
}
|
||||
showImportDialog.value = false;
|
||||
importFile.value = null;
|
||||
};
|
||||
|
||||
@@ -71,33 +71,33 @@ export const parseJsonFile = async <T extends Record<string, any>>(file: File):
|
||||
* @param description A brief description of the request for users, e.g., "import object detection models".
|
||||
* @param data Payload to be sent in the POST request
|
||||
* @param config Optional axios request configuration
|
||||
* @returns A promise that resolves when the POST request is complete
|
||||
* @returns A promise that resolves to true if the POST request is successful, or false if an error occurs.
|
||||
*/
|
||||
export const axiosPost = (url: string, description: string, data?: any, config?: any): Promise<void> => {
|
||||
return axios
|
||||
.post(url, data, config)
|
||||
.then(() => {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Successfully dispatched the request to " + description + ". Waiting for backend to respond",
|
||||
color: "success"
|
||||
});
|
||||
})
|
||||
.catch((error) => {
|
||||
if (error.response) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "The backend is unable to fulfill the request to " + description + ".",
|
||||
color: "error"
|
||||
});
|
||||
} else if (error.request) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Error while trying to process the request to " + description + "! The backend didn't respond.",
|
||||
color: "error"
|
||||
});
|
||||
} else {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "An error occurred while trying to process the request to " + description + ".",
|
||||
color: "error"
|
||||
});
|
||||
}
|
||||
export const axiosPost = async (url: string, description: string, data?: any, config?: any): Promise<boolean> => {
|
||||
try {
|
||||
await axios.post(url, data, config);
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Successfully dispatched the request to " + description + ". Waiting for backend to respond",
|
||||
color: "success"
|
||||
});
|
||||
return true;
|
||||
} catch (error: any) {
|
||||
if (error.response) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "The backend is unable to fulfill the request to " + description + ".",
|
||||
color: "error"
|
||||
});
|
||||
} else if (error.request) {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "Error while trying to process the request to " + description + "! The backend didn't respond.",
|
||||
color: "error"
|
||||
});
|
||||
} else {
|
||||
useStateStore().showSnackbarMessage({
|
||||
message: "An error occurred while trying to process the request to " + description + ".",
|
||||
color: "error"
|
||||
});
|
||||
}
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { JsonMatOfDouble, Resolution } from "@/types/SettingTypes";
|
||||
const { PerspectiveCamera } = await import("three");
|
||||
const three = import("three");
|
||||
|
||||
/**
|
||||
* Convert a camera intrinsics matrix and image resolution to a Three.js PerspectiveCamera. This assumes no skew and square pixels (same focal length in x and y), which is a sane assumption for most FRC cameras
|
||||
@@ -8,11 +8,12 @@ const { PerspectiveCamera } = await import("three");
|
||||
* @param intrinsicsCore camera intrinsics from the backend, row-major
|
||||
* @returns a Three.js PerspectiveCamera matching the provided intrinsics
|
||||
*/
|
||||
export const createPerspectiveCamera = (
|
||||
export const createPerspectiveCamera = async (
|
||||
resolution: Resolution,
|
||||
intrinsicsCore: JsonMatOfDouble,
|
||||
frustumMax: number = 1
|
||||
) => {
|
||||
const { PerspectiveCamera } = await three;
|
||||
const imageWidth = resolution.width;
|
||||
const imageHeight = resolution.height;
|
||||
const focalLengthY = intrinsicsCore.data[4];
|
||||
|
||||
@@ -66,7 +66,7 @@ export interface PipelineSettings {
|
||||
hsvHue: WebsocketNumberPair | [number, number];
|
||||
ledMode: boolean;
|
||||
hueInverted: boolean;
|
||||
outputShowMultipleTargets: boolean;
|
||||
outputMaximumTargets: number;
|
||||
contourSortMode: number;
|
||||
cameraExposureRaw: number;
|
||||
cameraMinExposureRaw: number;
|
||||
@@ -108,7 +108,7 @@ export type ConfigurablePipelineSettings = Partial<
|
||||
// Omitted settings are changed for all pipeline types
|
||||
export const DefaultPipelineSettings: Omit<
|
||||
PipelineSettings,
|
||||
"cameraGain" | "targetModel" | "ledMode" | "outputShowMultipleTargets" | "cameraExposureRaw" | "pipelineType"
|
||||
"cameraGain" | "targetModel" | "ledMode" | "cameraExposureRaw" | "pipelineType"
|
||||
> = {
|
||||
offsetRobotOffsetMode: RobotOffsetPointMode.None,
|
||||
streamingFrameDivisor: 0,
|
||||
@@ -137,6 +137,7 @@ export const DefaultPipelineSettings: Omit<
|
||||
offsetDualPointB: { x: 0, y: 0 },
|
||||
hsvHue: { first: 50, second: 180 },
|
||||
hueInverted: false,
|
||||
outputMaximumTargets: 20,
|
||||
contourSortMode: 0,
|
||||
offsetSinglePoint: { x: 0, y: 0 },
|
||||
cameraBrightness: 50,
|
||||
@@ -166,7 +167,7 @@ export const DefaultReflectivePipelineSettings: ReflectivePipelineSettings = {
|
||||
cameraGain: 20,
|
||||
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
|
||||
ledMode: true,
|
||||
outputShowMultipleTargets: false,
|
||||
outputMaximumTargets: 20,
|
||||
cameraExposureRaw: 6,
|
||||
pipelineType: PipelineType.Reflective,
|
||||
|
||||
@@ -197,7 +198,7 @@ export const DefaultColoredShapePipelineSettings: ColoredShapePipelineSettings =
|
||||
cameraGain: 75,
|
||||
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
|
||||
ledMode: true,
|
||||
outputShowMultipleTargets: false,
|
||||
outputMaximumTargets: 20,
|
||||
cameraExposureRaw: 20,
|
||||
pipelineType: PipelineType.ColoredShape,
|
||||
|
||||
@@ -237,10 +238,9 @@ export const DefaultAprilTagPipelineSettings: AprilTagPipelineSettings = {
|
||||
cameraGain: 75,
|
||||
targetModel: TargetModel.AprilTag6p5in_36h11,
|
||||
ledMode: false,
|
||||
outputShowMultipleTargets: true,
|
||||
outputMaximumTargets: 127,
|
||||
cameraExposureRaw: 20,
|
||||
pipelineType: PipelineType.AprilTag,
|
||||
|
||||
hammingDist: 0,
|
||||
numIterations: 40,
|
||||
decimate: 1,
|
||||
@@ -278,13 +278,12 @@ export type ConfigurableArucoPipelineSettings = Partial<Omit<ArucoPipelineSettin
|
||||
export const DefaultArucoPipelineSettings: ArucoPipelineSettings = {
|
||||
...DefaultPipelineSettings,
|
||||
cameraGain: 75,
|
||||
outputShowMultipleTargets: true,
|
||||
outputMaximumTargets: 127,
|
||||
targetModel: TargetModel.AprilTag6p5in_36h11,
|
||||
cameraExposureRaw: -1,
|
||||
cameraAutoExposure: true,
|
||||
ledMode: false,
|
||||
pipelineType: PipelineType.Aruco,
|
||||
|
||||
tagFamily: AprilTagFamily.Family36h11,
|
||||
threshWinSizes: { first: 11, second: 91 },
|
||||
threshStepSize: 40,
|
||||
@@ -316,7 +315,7 @@ export const DefaultObjectDetectionPipelineSettings: ObjectDetectionPipelineSett
|
||||
cameraGain: 20,
|
||||
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
|
||||
ledMode: true,
|
||||
outputShowMultipleTargets: false,
|
||||
outputMaximumTargets: 20,
|
||||
cameraExposureRaw: 6,
|
||||
confidence: 0.9,
|
||||
nms: 0.45,
|
||||
@@ -335,7 +334,7 @@ export const DefaultCalibration3dPipelineSettings: Calibration3dPipelineSettings
|
||||
cameraGain: 20,
|
||||
targetModel: TargetModel.InfiniteRechargeHighGoalOuter,
|
||||
ledMode: true,
|
||||
outputShowMultipleTargets: false,
|
||||
outputMaximumTargets: 1,
|
||||
cameraExposureRaw: 6,
|
||||
drawAllSnapshots: false
|
||||
};
|
||||
|
||||
@@ -39,7 +39,7 @@ import org.photonvision.vision.processes.VisionSource;
|
||||
import org.zeroturnaround.zip.ZipUtil;
|
||||
|
||||
public class ConfigManager {
|
||||
private static ConfigManager INSTANCE;
|
||||
static ConfigManager INSTANCE;
|
||||
|
||||
public static final String HW_CFG_FNAME = "hardwareConfig.json";
|
||||
public static final String HW_SET_FNAME = "hardwareSettings.json";
|
||||
|
||||
@@ -218,7 +218,7 @@ class LegacyConfigProvider extends ConfigProvider {
|
||||
hardwareSettings,
|
||||
networkConfig,
|
||||
atfl,
|
||||
new NeuralNetworkPropertyManager(),
|
||||
new NeuralNetworkModelsSettings(),
|
||||
cameraConfigurations);
|
||||
}
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ import java.util.Optional;
|
||||
import java.util.jar.JarEntry;
|
||||
import java.util.jar.JarFile;
|
||||
import java.util.stream.Stream;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
import org.photonvision.common.hardware.Platform;
|
||||
import org.photonvision.common.logging.LogGroup;
|
||||
import org.photonvision.common.logging.Logger;
|
||||
@@ -50,20 +50,20 @@ import org.photonvision.vision.objects.RubikModel;
|
||||
* extracted to the filesystem, it will not be extracted again.
|
||||
*
|
||||
* <p>Each model must have a corresponding {@link ModelProperties} entry in {@link
|
||||
* NeuralNetworkPropertyManager}.
|
||||
* NeuralNetworkModelsSettings}.
|
||||
*/
|
||||
public class NeuralNetworkModelManager {
|
||||
/** Singleton instance of the NeuralNetworkModelManager */
|
||||
private static NeuralNetworkModelManager INSTANCE;
|
||||
|
||||
private final List<Family> supportedBackends = new ArrayList<>();
|
||||
final List<Family> supportedBackends = new ArrayList<>();
|
||||
|
||||
/**
|
||||
* This function stores the properties of the shipped object detection models. It is stored as a
|
||||
* function so that it can be dynamic, to adjust for the models directory.
|
||||
*/
|
||||
private NeuralNetworkPropertyManager getShippedProperties(File modelsDirectory) {
|
||||
NeuralNetworkPropertyManager nnProps = new NeuralNetworkPropertyManager();
|
||||
private NeuralNetworkModelsSettings getShippedProperties(File modelsDirectory) {
|
||||
NeuralNetworkModelsSettings nnProps = new NeuralNetworkModelsSettings();
|
||||
|
||||
LinkedList<String> cocoLabels =
|
||||
new LinkedList<String>(
|
||||
@@ -149,16 +149,6 @@ public class NeuralNetworkModelManager {
|
||||
"hair drier", // Typo in official COCO documentation
|
||||
"toothbrush"));
|
||||
|
||||
nnProps.addModelProperties(
|
||||
new ModelProperties(
|
||||
Path.of(modelsDirectory.getAbsolutePath(), "algaeV1-640-640-yolov8n.rknn"),
|
||||
"Algae v8n",
|
||||
new LinkedList<String>(List.of("Algae")),
|
||||
640,
|
||||
480,
|
||||
Family.RKNN,
|
||||
Version.YOLOV8));
|
||||
|
||||
nnProps.addModelProperties(
|
||||
new ModelProperties(
|
||||
Path.of(modelsDirectory.getAbsolutePath(), "yolov8nCOCO.rknn"),
|
||||
@@ -171,13 +161,13 @@ public class NeuralNetworkModelManager {
|
||||
|
||||
nnProps.addModelProperties(
|
||||
new ModelProperties(
|
||||
Path.of(modelsDirectory.getAbsolutePath(), "algae-coral-yolov8s.tflite"),
|
||||
"Algae Coral v8s",
|
||||
new LinkedList<String>(List.of("Algae", "Coral")),
|
||||
Path.of(modelsDirectory.getAbsolutePath(), "fuelV1-yolo11n.rknn"),
|
||||
"Fuel v11n",
|
||||
new LinkedList<String>(List.of("Fuel")),
|
||||
640,
|
||||
640,
|
||||
Family.RUBIK,
|
||||
Version.YOLOV8));
|
||||
Family.RKNN,
|
||||
Version.YOLOV11));
|
||||
|
||||
nnProps.addModelProperties(
|
||||
new ModelProperties(
|
||||
@@ -189,6 +179,16 @@ public class NeuralNetworkModelManager {
|
||||
Family.RUBIK,
|
||||
Version.YOLOV8));
|
||||
|
||||
nnProps.addModelProperties(
|
||||
new ModelProperties(
|
||||
Path.of(modelsDirectory.getAbsolutePath(), "fuelV1-yolo11n.tflite"),
|
||||
"Fuel v11n",
|
||||
new LinkedList<String>(List.of("Fuel")),
|
||||
640,
|
||||
640,
|
||||
Family.RUBIK,
|
||||
Version.YOLOV11));
|
||||
|
||||
return nnProps;
|
||||
}
|
||||
|
||||
@@ -272,7 +272,7 @@ public class NeuralNetworkModelManager {
|
||||
*
|
||||
* <p>The first model in the list is the default model.
|
||||
*/
|
||||
private Map<Family, ArrayList<Model>> models;
|
||||
Map<Family, ArrayList<Model>> models;
|
||||
|
||||
/**
|
||||
* Retrieves the model with the specified name, assuming it is available under a supported
|
||||
@@ -321,13 +321,27 @@ public class NeuralNetworkModelManager {
|
||||
ConfigManager.getInstance().getConfig().neuralNetworkPropertyManager().getModel(path);
|
||||
|
||||
if (properties == null) {
|
||||
logger.error(
|
||||
logger.warn(
|
||||
"Model properties are null. This could mean the config for model "
|
||||
+ path
|
||||
+ " was unable to be found in the database.");
|
||||
return;
|
||||
+ " was unable to be found in the database. Trying legacy...");
|
||||
try {
|
||||
properties = ModelProperties.createFromFilename(path.getFileName().toString());
|
||||
|
||||
// At this point this property is not serialized or known to our configuration. add to
|
||||
// NeuralNetworkModelsSettings
|
||||
ConfigManager.getInstance()
|
||||
.getConfig()
|
||||
.neuralNetworkPropertyManager()
|
||||
.addModelProperties(properties);
|
||||
} catch (IllegalArgumentException | IOException e) {
|
||||
logger.error("Failed to translate legacy model filename to properties: " + path, e);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
logger.debug(properties.toString());
|
||||
|
||||
if (!supportedBackends.contains(properties.family())) {
|
||||
logger.warn(
|
||||
"Model "
|
||||
@@ -412,7 +426,7 @@ public class NeuralNetworkModelManager {
|
||||
File modelsDirectory = ConfigManager.getInstance().getModelsDirectory();
|
||||
|
||||
// Filter shippedProprties by supportedBackends
|
||||
NeuralNetworkPropertyManager supportedProperties = new NeuralNetworkPropertyManager();
|
||||
NeuralNetworkModelsSettings supportedProperties = new NeuralNetworkModelsSettings();
|
||||
for (ModelProperties model : getShippedProperties(modelsDirectory).getModels()) {
|
||||
if (supportedBackends.contains(model.family())) {
|
||||
supportedProperties.addModelProperties(model);
|
||||
|
||||
@@ -0,0 +1,277 @@
|
||||
/*
|
||||
* Copyright (C) Photon Vision.
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package org.photonvision.common.configuration;
|
||||
|
||||
import com.fasterxml.jackson.annotation.JsonCreator;
|
||||
import com.fasterxml.jackson.annotation.JsonIgnore;
|
||||
import com.fasterxml.jackson.annotation.JsonProperty;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.Arrays;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
|
||||
|
||||
public class NeuralNetworkModelsSettings {
|
||||
/*
|
||||
* The properties of the model. This is used to determine which model to load.
|
||||
* The only families currently supported are RKNN and Rubik (custom .tflite)
|
||||
*/
|
||||
public record ModelProperties(
|
||||
@JsonProperty("modelPath") Path modelPath,
|
||||
@JsonProperty("nickname") String nickname,
|
||||
@JsonProperty("labels") List<String> labels,
|
||||
@JsonProperty("resolutionWidth") int resolutionWidth,
|
||||
@JsonProperty("resolutionHeight") int resolutionHeight,
|
||||
@JsonProperty("family") Family family,
|
||||
@JsonProperty("version") Version version) {
|
||||
@JsonCreator
|
||||
public ModelProperties {}
|
||||
|
||||
ModelProperties(ModelProperties other) {
|
||||
this(
|
||||
other.modelPath,
|
||||
other.nickname,
|
||||
other.labels, // note this does not clone the underlying list
|
||||
other.resolutionWidth,
|
||||
other.resolutionHeight,
|
||||
other.family,
|
||||
other.version);
|
||||
}
|
||||
|
||||
// In v2025.3.1, this was single string for the model path. but the first argument
|
||||
// is now nickname
|
||||
public ModelProperties(@JsonProperty("nickname") String filename)
|
||||
throws IllegalArgumentException, IOException {
|
||||
this(createFromFilename(filename));
|
||||
}
|
||||
|
||||
// ============= Migration code from v2025.3.1 ===========
|
||||
|
||||
private static Pattern modelPattern =
|
||||
Pattern.compile("^([a-zA-Z0-9._]+)-(\\d+)-(\\d+)-(yolov(?:5|8|11)[nsmlx]*)\\.rknn$");
|
||||
|
||||
static ModelProperties createFromFilename(String modelFileName)
|
||||
throws IllegalArgumentException, IOException {
|
||||
// Used to point to default models directory
|
||||
var model =
|
||||
ConfigManager.getInstance().getModelsDirectory().toPath().resolve(modelFileName).toFile();
|
||||
|
||||
// Get the model extension and check if it is supported
|
||||
String modelExtension = model.getName().substring(model.getName().lastIndexOf('.'));
|
||||
if (!modelExtension.equals(".rknn")) {
|
||||
throw new IllegalArgumentException("Model " + modelFileName + " is not a supported format");
|
||||
}
|
||||
|
||||
var backend =
|
||||
Arrays.stream(NeuralNetworkModelManager.Family.values())
|
||||
.filter(b -> b.extension().equals(modelExtension))
|
||||
.findFirst();
|
||||
|
||||
if (!backend.isPresent()) {
|
||||
throw new IllegalArgumentException("Model " + modelFileName + " cannot find backend");
|
||||
}
|
||||
|
||||
String labelFile = model.getAbsolutePath().replace(backend.get().extension(), "-labels.txt");
|
||||
List<String> labels = Files.readAllLines(Paths.get(labelFile));
|
||||
|
||||
String[] parts = parseRKNNName(modelFileName);
|
||||
var version = getModelVersion(parts[3]);
|
||||
int width = Integer.parseInt(parts[1]);
|
||||
int height = Integer.parseInt(parts[2]);
|
||||
|
||||
return new ModelProperties(
|
||||
model.toPath(),
|
||||
model.getName(),
|
||||
labels,
|
||||
// all files used to be 640x640
|
||||
width,
|
||||
height,
|
||||
Family.RKNN,
|
||||
version);
|
||||
}
|
||||
|
||||
/**
|
||||
* Determines the model version based on the model's filename.
|
||||
*
|
||||
* <p>"yolov5" -> "YOLO_V5"
|
||||
*
|
||||
* <p>"yolov8" -> "YOLO_V8"
|
||||
*
|
||||
* <p>"yolov11" -> "YOLO_V11"
|
||||
*
|
||||
* @param modelName The model's filename
|
||||
* @return The model version
|
||||
*/
|
||||
private static Version getModelVersion(String modelName) throws IllegalArgumentException {
|
||||
if (modelName.contains("yolov5")) {
|
||||
return Version.YOLOV5;
|
||||
} else if (modelName.contains("yolov8")) {
|
||||
return Version.YOLOV8;
|
||||
} else if (modelName.contains("yolov11")) {
|
||||
return Version.YOLOV11;
|
||||
} else {
|
||||
throw new IllegalArgumentException("Unknown model version for model " + modelName);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse RKNN name and return the name, width, height, and model type.
|
||||
*
|
||||
* <p>This is static as it is not dependent on the state of the class.
|
||||
*
|
||||
* @param modelName the name of the model
|
||||
* @throws IllegalArgumentException if the model name does not follow the naming convention
|
||||
* @return an array containing the name, width, height, and model type
|
||||
*/
|
||||
public static String[] parseRKNNName(String modelName) {
|
||||
Matcher modelMatcher = modelPattern.matcher(modelName);
|
||||
|
||||
if (!modelMatcher.matches()) {
|
||||
throw new IllegalArgumentException(
|
||||
"Model name must follow the naming convention of name-widthResolution-heightResolution-modelType.rknn");
|
||||
}
|
||||
|
||||
return new String[] {
|
||||
modelMatcher.group(1), modelMatcher.group(2), modelMatcher.group(3), modelMatcher.group(4)
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// The path to the model is used as the key in the map because it is unique to
|
||||
// the model, and should not change
|
||||
@JsonProperty("modelPathToProperties")
|
||||
private HashMap<Path, ModelProperties> modelPathToProperties =
|
||||
new HashMap<Path, ModelProperties>();
|
||||
|
||||
/**
|
||||
* Constructor for the NeuralNetworkProperties class.
|
||||
*
|
||||
* <p>This object holds a LinkedList of {@link ModelProperties} objects
|
||||
*/
|
||||
public NeuralNetworkModelsSettings() {}
|
||||
|
||||
/**
|
||||
* Constructor for the NeuralNetworkProperties class.
|
||||
*
|
||||
* <p>This object holds a LinkedList of {@link ModelProperties} objects.
|
||||
*
|
||||
* @param modelPropertiesList When the class is constructed, it will hold the provided list
|
||||
*/
|
||||
public NeuralNetworkModelsSettings(HashMap<Path, ModelProperties> modelPropertiesList) {}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
String toReturn = "";
|
||||
|
||||
toReturn += "NeuralNetworkProperties [";
|
||||
|
||||
toReturn += modelPathToProperties.toString() + "]";
|
||||
|
||||
return toReturn;
|
||||
}
|
||||
|
||||
/**
|
||||
* Add a model to the list of models.
|
||||
*
|
||||
* @param modelProperties
|
||||
*/
|
||||
public void addModelProperties(ModelProperties modelProperties) {
|
||||
modelPathToProperties.put(modelProperties.modelPath, modelProperties);
|
||||
}
|
||||
|
||||
/**
|
||||
* Add two Neural Network Properties together.
|
||||
*
|
||||
* <p>Any properties that are the same will be overwritten by the second
|
||||
*
|
||||
* @param nnProps
|
||||
* @return itself, so it can be chained and used fluently
|
||||
*/
|
||||
public NeuralNetworkModelsSettings sum(NeuralNetworkModelsSettings nnProps) {
|
||||
modelPathToProperties.putAll(nnProps.modelPathToProperties);
|
||||
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a model from the list of models.
|
||||
*
|
||||
* @param modelPath
|
||||
* @return True if the model was removed, false if it was not found
|
||||
*/
|
||||
public boolean removeModel(Path modelPath) {
|
||||
return modelPathToProperties.remove(modelPath) != null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the model properties for a given model path.
|
||||
*
|
||||
* @param modelPath
|
||||
* @return {@link ModelProperties} object
|
||||
*/
|
||||
public ModelProperties getModel(Path modelPath) {
|
||||
return modelPathToProperties.get(modelPath);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all models
|
||||
*
|
||||
* @return A list of all models
|
||||
*/
|
||||
@JsonIgnore
|
||||
public ModelProperties[] getModels() {
|
||||
return modelPathToProperties.values().toArray(new ModelProperties[0]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Change the nickname of a {@link ModelProperties} object.
|
||||
*
|
||||
* @param modelPath
|
||||
* @param newName
|
||||
* @return True if the model was found and renamed, false if it was not found
|
||||
*/
|
||||
public boolean renameModel(Path modelPath, String newName) {
|
||||
ModelProperties temp = modelPathToProperties.get(modelPath);
|
||||
if (temp != null) {
|
||||
modelPathToProperties.remove(modelPath);
|
||||
modelPathToProperties.put(
|
||||
modelPath,
|
||||
new ModelProperties(
|
||||
temp.modelPath,
|
||||
newName,
|
||||
temp.labels,
|
||||
temp.resolutionWidth,
|
||||
temp.resolutionHeight,
|
||||
temp.family,
|
||||
temp.version));
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
public boolean clear() {
|
||||
modelPathToProperties.clear();
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -1,164 +0,0 @@
|
||||
/*
|
||||
* Copyright (C) Photon Vision.
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package org.photonvision.common.configuration;
|
||||
|
||||
import com.fasterxml.jackson.annotation.JsonCreator;
|
||||
import com.fasterxml.jackson.annotation.JsonIgnore;
|
||||
import com.fasterxml.jackson.annotation.JsonProperty;
|
||||
import java.nio.file.Path;
|
||||
import java.util.HashMap;
|
||||
import java.util.LinkedList;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
|
||||
|
||||
public class NeuralNetworkPropertyManager {
|
||||
/*
|
||||
* The properties of the model. This is used to determine which model to load.
|
||||
* The only families currently supported are RKNN and Rubik (custom .tflite)
|
||||
*/
|
||||
public record ModelProperties(
|
||||
@JsonProperty("modelPath") Path modelPath,
|
||||
@JsonProperty("nickname") String nickname,
|
||||
@JsonProperty("labels") LinkedList<String> labels,
|
||||
@JsonProperty("resolutionWidth") int resolutionWidth,
|
||||
@JsonProperty("resolutionHeight") int resolutionHeight,
|
||||
@JsonProperty("family") Family family,
|
||||
@JsonProperty("version") Version version) {
|
||||
@JsonCreator
|
||||
public ModelProperties {
|
||||
// Record constructor is automatically annotated with @JsonCreator
|
||||
}
|
||||
}
|
||||
|
||||
// The path to the model is used as the key in the map because it is unique to
|
||||
// the model, and should not change
|
||||
@JsonProperty("modelPathToProperties")
|
||||
private HashMap<Path, ModelProperties> modelPathToProperties =
|
||||
new HashMap<Path, ModelProperties>();
|
||||
|
||||
/**
|
||||
* Constructor for the NeuralNetworkProperties class.
|
||||
*
|
||||
* <p>This object holds a LinkedList of {@link ModelProperties} objects
|
||||
*/
|
||||
public NeuralNetworkPropertyManager() {}
|
||||
|
||||
/**
|
||||
* Constructor for the NeuralNetworkProperties class.
|
||||
*
|
||||
* <p>This object holds a LinkedList of {@link ModelProperties} objects.
|
||||
*
|
||||
* @param modelPropertiesList When the class is constructed, it will hold the provided list
|
||||
*/
|
||||
public NeuralNetworkPropertyManager(HashMap<Path, ModelProperties> modelPropertiesList) {}
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
String toReturn = "";
|
||||
|
||||
toReturn += "NeuralNetworkProperties [";
|
||||
|
||||
toReturn += modelPathToProperties.toString() + "]";
|
||||
|
||||
return toReturn;
|
||||
}
|
||||
|
||||
/**
|
||||
* Add a model to the list of models.
|
||||
*
|
||||
* @param modelProperties
|
||||
*/
|
||||
public void addModelProperties(ModelProperties modelProperties) {
|
||||
modelPathToProperties.put(modelProperties.modelPath, modelProperties);
|
||||
}
|
||||
|
||||
/**
|
||||
* Add two Neural Network Properties together.
|
||||
*
|
||||
* <p>Any properties that are the same will be overwritten by the second
|
||||
*
|
||||
* @param nnProps
|
||||
* @return itself, so it can be chained and used fluently
|
||||
*/
|
||||
public NeuralNetworkPropertyManager sum(NeuralNetworkPropertyManager nnProps) {
|
||||
modelPathToProperties.putAll(nnProps.modelPathToProperties);
|
||||
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a model from the list of models.
|
||||
*
|
||||
* @param modelPath
|
||||
* @return True if the model was removed, false if it was not found
|
||||
*/
|
||||
public boolean removeModel(Path modelPath) {
|
||||
return modelPathToProperties.remove(modelPath) != null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the model properties for a given model path.
|
||||
*
|
||||
* @param modelPath
|
||||
* @return {@link ModelProperties} object
|
||||
*/
|
||||
public ModelProperties getModel(Path modelPath) {
|
||||
return modelPathToProperties.get(modelPath);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all models
|
||||
*
|
||||
* @return A list of all models
|
||||
*/
|
||||
@JsonIgnore
|
||||
public ModelProperties[] getModels() {
|
||||
return modelPathToProperties.values().toArray(new ModelProperties[0]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Change the nickname of a {@link ModelProperties} object.
|
||||
*
|
||||
* @param modelPath
|
||||
* @param newName
|
||||
* @return True if the model was found and renamed, false if it was not found
|
||||
*/
|
||||
public boolean renameModel(Path modelPath, String newName) {
|
||||
ModelProperties temp = modelPathToProperties.get(modelPath);
|
||||
if (temp != null) {
|
||||
modelPathToProperties.remove(modelPath);
|
||||
modelPathToProperties.put(
|
||||
modelPath,
|
||||
new ModelProperties(
|
||||
temp.modelPath,
|
||||
newName,
|
||||
temp.labels,
|
||||
temp.resolutionWidth,
|
||||
temp.resolutionHeight,
|
||||
temp.family,
|
||||
temp.version));
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
public boolean clear() {
|
||||
modelPathToProperties.clear();
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -28,7 +28,7 @@ public class PhotonConfiguration {
|
||||
private final HardwareSettings hardwareSettings;
|
||||
private NetworkConfig networkConfig;
|
||||
private AprilTagFieldLayout atfl;
|
||||
private NeuralNetworkPropertyManager neuralNetworkProperties;
|
||||
private NeuralNetworkModelsSettings neuralNetworkProperties;
|
||||
private HashMap<String, CameraConfiguration> cameraConfigurations;
|
||||
|
||||
public PhotonConfiguration(
|
||||
@@ -36,7 +36,7 @@ public class PhotonConfiguration {
|
||||
HardwareSettings hardwareSettings,
|
||||
NetworkConfig networkConfig,
|
||||
AprilTagFieldLayout atfl,
|
||||
NeuralNetworkPropertyManager neuralNetworkProperties) {
|
||||
NeuralNetworkModelsSettings neuralNetworkProperties) {
|
||||
this(
|
||||
hardwareConfig,
|
||||
hardwareSettings,
|
||||
@@ -51,7 +51,7 @@ public class PhotonConfiguration {
|
||||
HardwareSettings hardwareSettings,
|
||||
NetworkConfig networkConfig,
|
||||
AprilTagFieldLayout atfl,
|
||||
NeuralNetworkPropertyManager neuralNetworkProperties,
|
||||
NeuralNetworkModelsSettings neuralNetworkProperties,
|
||||
HashMap<String, CameraConfiguration> cameraConfigurations) {
|
||||
this.hardwareConfig = hardwareConfig;
|
||||
this.hardwareSettings = hardwareSettings;
|
||||
@@ -67,7 +67,7 @@ public class PhotonConfiguration {
|
||||
new HardwareSettings(),
|
||||
new NetworkConfig(),
|
||||
new AprilTagFieldLayout(List.of(), 0, 0),
|
||||
new NeuralNetworkPropertyManager());
|
||||
new NeuralNetworkModelsSettings());
|
||||
}
|
||||
|
||||
public HardwareConfig getHardwareConfig() {
|
||||
@@ -86,7 +86,7 @@ public class PhotonConfiguration {
|
||||
return atfl;
|
||||
}
|
||||
|
||||
public NeuralNetworkPropertyManager neuralNetworkPropertyManager() {
|
||||
public NeuralNetworkModelsSettings neuralNetworkPropertyManager() {
|
||||
return neuralNetworkProperties;
|
||||
}
|
||||
|
||||
@@ -98,7 +98,7 @@ public class PhotonConfiguration {
|
||||
this.networkConfig = networkConfig;
|
||||
}
|
||||
|
||||
public void setNeuralNetworkProperties(NeuralNetworkPropertyManager neuralNetworkProperties) {
|
||||
public void setNeuralNetworkProperties(NeuralNetworkModelsSettings neuralNetworkProperties) {
|
||||
this.neuralNetworkProperties = neuralNetworkProperties;
|
||||
}
|
||||
|
||||
@@ -132,6 +132,16 @@ public class PhotonConfiguration {
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
StringBuilder cameraConfigurationsString = new StringBuilder();
|
||||
cameraConfigurations.forEach(
|
||||
(key, value) -> {
|
||||
cameraConfigurationsString
|
||||
.append("\n ")
|
||||
.append(key)
|
||||
.append(" -> ")
|
||||
.append(value.toString());
|
||||
});
|
||||
|
||||
return "PhotonConfiguration [\n hardwareConfig="
|
||||
+ hardwareConfig
|
||||
+ "\n hardwareSettings="
|
||||
@@ -142,8 +152,8 @@ public class PhotonConfiguration {
|
||||
+ atfl
|
||||
+ "\n neuralNetworkProperties="
|
||||
+ neuralNetworkProperties
|
||||
+ "\n cameraConfigurations="
|
||||
+ cameraConfigurations
|
||||
+ "\n]";
|
||||
+ "\n cameraConfigurations={"
|
||||
+ cameraConfigurationsString
|
||||
+ "}\n]";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -269,7 +269,7 @@ public class SqlConfigProvider extends ConfigProvider {
|
||||
} else {
|
||||
logger.debug("No " + ref.getSimpleName() + " in database");
|
||||
}
|
||||
// either the config entry is empty or Jackson threw and exception
|
||||
// either the config entry is empty or Jackson threw an exception
|
||||
try {
|
||||
configObj = factory.get();
|
||||
logger.info("Loaded default " + ref.getSimpleName());
|
||||
@@ -313,8 +313,8 @@ public class SqlConfigProvider extends ConfigProvider {
|
||||
loadConfigOrDefault(
|
||||
conn,
|
||||
GlobalKeys.NEURAL_NETWORK_PROPERTIES,
|
||||
NeuralNetworkPropertyManager.class,
|
||||
NeuralNetworkPropertyManager::new);
|
||||
NeuralNetworkModelsSettings.class,
|
||||
NeuralNetworkModelsSettings::new);
|
||||
var atfl =
|
||||
loadConfigOrDefault(
|
||||
conn, GlobalKeys.ATFL_CONFIG_FILE, AprilTagFieldLayout.class, this::atflDefault);
|
||||
@@ -612,52 +612,65 @@ public class SqlConfigProvider extends ConfigProvider {
|
||||
|
||||
// Iterate over every row/"camera" in the table
|
||||
while (result.next()) {
|
||||
List<String> dummyList = new ArrayList<>();
|
||||
String uniqueName = "";
|
||||
try {
|
||||
List<String> dummyList = new ArrayList<>();
|
||||
|
||||
var uniqueName = result.getString(Columns.CAM_UNIQUE_NAME);
|
||||
uniqueName = result.getString(Columns.CAM_UNIQUE_NAME);
|
||||
|
||||
// A horrifying hack to keep backward compat with otherpaths
|
||||
// We -really- need to delete this -stupid- otherpaths column. I hate it.
|
||||
var configStr = result.getString(Columns.CAM_CONFIG_JSON);
|
||||
CameraConfiguration config = JacksonUtils.deserialize(configStr, CameraConfiguration.class);
|
||||
// A horrifying hack to keep backward compat with otherpaths
|
||||
// We -really- need to delete this -stupid- otherpaths column. I hate it.
|
||||
var configStr = result.getString(Columns.CAM_CONFIG_JSON);
|
||||
CameraConfiguration config =
|
||||
JacksonUtils.deserialize(configStr, CameraConfiguration.class);
|
||||
|
||||
if (config.matchedCameraInfo == null) {
|
||||
logger.info("Legacy CameraConfiguration detected - upgrading");
|
||||
if (config.matchedCameraInfo == null) {
|
||||
logger.info("Legacy CameraConfiguration detected - upgrading");
|
||||
|
||||
// manually create the matchedCameraInfo ourselves. Need to upgrade:
|
||||
// baseName, path, otherPaths, cameraType, usbvid/pid -> matchedCameraInfo
|
||||
config.matchedCameraInfo =
|
||||
JacksonUtils.deserialize(configStr, LegacyCameraConfigStruct.class).matchedCameraInfo;
|
||||
// manually create the matchedCameraInfo ourselves. Need to upgrade:
|
||||
// baseName, path, otherPaths, cameraType, usbvid/pid -> matchedCameraInfo
|
||||
config.matchedCameraInfo =
|
||||
JacksonUtils.deserialize(configStr, LegacyCameraConfigStruct.class)
|
||||
.matchedCameraInfo;
|
||||
|
||||
// Except that otherPaths used to be its own column. so hack that in here as well
|
||||
var otherPaths =
|
||||
// Except that otherPaths used to be its own column. so hack that in here as well
|
||||
var otherPaths =
|
||||
JacksonUtils.deserialize(
|
||||
result.getString(Columns.CAM_OTHERPATHS_JSON), String[].class);
|
||||
if (config.matchedCameraInfo instanceof UsbCameraInfo usbInfo) {
|
||||
usbInfo.otherPaths = otherPaths;
|
||||
}
|
||||
}
|
||||
|
||||
var driverMode =
|
||||
JacksonUtils.deserialize(
|
||||
result.getString(Columns.CAM_OTHERPATHS_JSON), String[].class);
|
||||
if (config.matchedCameraInfo instanceof UsbCameraInfo usbInfo) {
|
||||
usbInfo.otherPaths = otherPaths;
|
||||
result.getString(Columns.CAM_DRIVERMODE_JSON), DriverModePipelineSettings.class);
|
||||
List<?> pipelineSettings =
|
||||
JacksonUtils.deserialize(
|
||||
result.getString(Columns.CAM_PIPELINE_JSONS), dummyList.getClass());
|
||||
|
||||
List<CVPipelineSettings> loadedSettings = new ArrayList<>();
|
||||
for (var setting : pipelineSettings) {
|
||||
if (setting instanceof String str) {
|
||||
try {
|
||||
loadedSettings.add(JacksonUtils.deserialize(str, CVPipelineSettings.class));
|
||||
} catch (IOException e) {
|
||||
logger.error(
|
||||
"Could not deserialize pipeline setting for camera " + config.nickname, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
config.pipelineSettings = loadedSettings;
|
||||
config.driveModeSettings = driverMode;
|
||||
loadedConfigurations.put(uniqueName, config);
|
||||
} catch (IOException e) {
|
||||
logger.error(
|
||||
"Could not deserialize camera configuration " + uniqueName + " from database!", e);
|
||||
}
|
||||
|
||||
var driverMode =
|
||||
JacksonUtils.deserialize(
|
||||
result.getString(Columns.CAM_DRIVERMODE_JSON), DriverModePipelineSettings.class);
|
||||
List<?> pipelineSettings =
|
||||
JacksonUtils.deserialize(
|
||||
result.getString(Columns.CAM_PIPELINE_JSONS), dummyList.getClass());
|
||||
|
||||
List<CVPipelineSettings> loadedSettings = new ArrayList<>();
|
||||
for (var setting : pipelineSettings) {
|
||||
if (setting instanceof String str) {
|
||||
loadedSettings.add(JacksonUtils.deserialize(str, CVPipelineSettings.class));
|
||||
}
|
||||
}
|
||||
|
||||
config.pipelineSettings = loadedSettings;
|
||||
config.driveModeSettings = driverMode;
|
||||
loadedConfigurations.put(uniqueName, config);
|
||||
}
|
||||
} catch (SQLException | IOException e) {
|
||||
logger.error("Err loading cameras: ", e);
|
||||
} catch (SQLException e) {
|
||||
logger.error("Err querying database to load cameras: ", e);
|
||||
} finally {
|
||||
try {
|
||||
if (query != null) query.close();
|
||||
|
||||
@@ -18,14 +18,14 @@
|
||||
package org.photonvision.common.dataflow.websocket;
|
||||
|
||||
import java.util.List;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings;
|
||||
|
||||
public class UIGeneralSettings {
|
||||
public UIGeneralSettings(
|
||||
String version,
|
||||
String gpuAcceleration,
|
||||
boolean mrCalWorking,
|
||||
NeuralNetworkPropertyManager.ModelProperties[] availableModels,
|
||||
NeuralNetworkModelsSettings.ModelProperties[] availableModels,
|
||||
List<String> supportedBackends,
|
||||
String hardwareModel,
|
||||
String hardwarePlatform,
|
||||
@@ -45,7 +45,7 @@ public class UIGeneralSettings {
|
||||
public String version;
|
||||
public String gpuAcceleration;
|
||||
public boolean mrCalWorking;
|
||||
public NeuralNetworkPropertyManager.ModelProperties[] availableModels;
|
||||
public NeuralNetworkModelsSettings.ModelProperties[] availableModels;
|
||||
public List<String> supportedBackends;
|
||||
public String hardwareModel;
|
||||
public String hardwarePlatform;
|
||||
|
||||
@@ -43,6 +43,7 @@ public class VisionLED implements AutoCloseable {
|
||||
|
||||
private VisionLEDMode currentLedMode = VisionLEDMode.kDefault;
|
||||
private BooleanSupplier pipelineModeSupplier;
|
||||
private boolean currentOutputState = false;
|
||||
|
||||
private float mappedBrightness = 0.0f;
|
||||
|
||||
@@ -85,10 +86,15 @@ public class VisionLED implements AutoCloseable {
|
||||
public void setBrightness(int percentage) {
|
||||
mappedBrightness =
|
||||
(float) (MathUtils.map(percentage, 0.0, 100.0, brightnessMin, brightnessMax) / 100.0);
|
||||
setInternal(currentLedMode, false);
|
||||
if (currentOutputState) {
|
||||
for (PwmLed led : dimmableVisionLEDs) {
|
||||
led.setValue(mappedBrightness);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public void blink(int pulseLengthMillis, int blinkCount) {
|
||||
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
|
||||
blinkImpl(pulseLengthMillis, blinkCount);
|
||||
int blinkDuration = pulseLengthMillis * blinkCount * 2;
|
||||
TimedTaskManager.getInstance()
|
||||
@@ -96,19 +102,13 @@ public class VisionLED implements AutoCloseable {
|
||||
}
|
||||
|
||||
private void blinkImpl(int pulseLengthMillis, int blinkCount) {
|
||||
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
|
||||
AtomicInteger blinks = new AtomicInteger();
|
||||
TimedTaskManager.getInstance()
|
||||
.addTask(
|
||||
blinkTaskID,
|
||||
() -> {
|
||||
for (LED led : visionLEDs) {
|
||||
led.toggle();
|
||||
}
|
||||
for (PwmLed led : dimmableVisionLEDs) {
|
||||
led.setValue(mappedBrightness - led.getValue());
|
||||
}
|
||||
if (blinks.incrementAndGet() >= blinkCount * 2) {
|
||||
setStateImpl(!currentOutputState);
|
||||
if (blinkCount >= 0 && blinks.incrementAndGet() >= blinkCount * 2) {
|
||||
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
|
||||
}
|
||||
},
|
||||
@@ -116,12 +116,16 @@ public class VisionLED implements AutoCloseable {
|
||||
}
|
||||
|
||||
private void setStateImpl(boolean state) {
|
||||
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
|
||||
currentOutputState = state;
|
||||
for (LED led : visionLEDs) {
|
||||
led.setOn(state);
|
||||
}
|
||||
for (PwmLed led : dimmableVisionLEDs) {
|
||||
led.setValue(mappedBrightness);
|
||||
if (state) {
|
||||
led.setValue(mappedBrightness);
|
||||
} else {
|
||||
led.off();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -162,6 +166,7 @@ public class VisionLED implements AutoCloseable {
|
||||
var lastLedMode = currentLedMode;
|
||||
|
||||
if (fromNT || currentLedMode == VisionLEDMode.kDefault) {
|
||||
TimedTaskManager.getInstance().cancelTask(blinkTaskID);
|
||||
switch (newLedMode) {
|
||||
case kDefault -> setStateImpl(pipelineModeSupplier.getAsBoolean());
|
||||
case kOff -> setStateImpl(false);
|
||||
|
||||
@@ -228,7 +228,8 @@ public class TestUtils {
|
||||
kRobots,
|
||||
kTag1_640_480,
|
||||
kTag1_16h5_1280,
|
||||
kTag_corner_1280;
|
||||
kTag_corner_1280,
|
||||
k36h11_stress_test;
|
||||
|
||||
public final Path path;
|
||||
|
||||
@@ -237,6 +238,7 @@ public class TestUtils {
|
||||
var filename = this.toString().substring(1).toLowerCase();
|
||||
var extension = ".jpg";
|
||||
if (filename.equals("tag1_16h5_1280")) extension = ".png";
|
||||
if (filename.equals("36h11_stress_test")) extension = ".png";
|
||||
return Path.of("apriltag", filename + extension);
|
||||
}
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
package org.photonvision.common.util.file;
|
||||
|
||||
import com.fasterxml.jackson.core.JsonGenerator;
|
||||
import com.fasterxml.jackson.core.StreamReadFeature;
|
||||
import com.fasterxml.jackson.core.json.JsonReadFeature;
|
||||
import com.fasterxml.jackson.databind.DeserializationContext;
|
||||
import com.fasterxml.jackson.databind.DeserializationFeature;
|
||||
@@ -80,6 +81,7 @@ public class JacksonUtils {
|
||||
pathModule.addKeyDeserializer(Path.class, new PathKeyDeserializer());
|
||||
|
||||
return JsonMapper.builder()
|
||||
.enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION)
|
||||
.configure(JsonReadFeature.ALLOW_JAVA_COMMENTS, true)
|
||||
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
|
||||
.activateDefaultTyping(ptv, ObjectMapper.DefaultTyping.JAVA_LANG_OBJECT)
|
||||
|
||||
@@ -167,7 +167,27 @@ public class GenericUSBCameraSettables extends VisionSourceSettables {
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void setAutoExposure(boolean cameraAutoExposure) {
|
||||
if (configuration.cameraQuirks.hasQuirk(CameraQuirk.ArduOV9281Controls)
|
||||
&& !cameraAutoExposure) {
|
||||
// OV9281 on Linux seems to sometimes ignore our exposure requests on first boot if we're in
|
||||
// manual mode. Poking the camera into and out of auto exposure seems to fix it.
|
||||
try {
|
||||
setAutoExposureImpl(false);
|
||||
Thread.sleep(2000);
|
||||
setAutoExposureImpl(true);
|
||||
Thread.sleep(2000);
|
||||
setAutoExposureImpl(false);
|
||||
} catch (InterruptedException e) {
|
||||
logger.error("Thread interrupted while setting OV9281 exposure!", e);
|
||||
}
|
||||
} else {
|
||||
setAutoExposureImpl(cameraAutoExposure);
|
||||
}
|
||||
}
|
||||
|
||||
public void setAutoExposureImpl(boolean cameraAutoExposure) {
|
||||
logger.debug("Setting auto exposure to " + cameraAutoExposure);
|
||||
|
||||
if (!cameraAutoExposure) {
|
||||
|
||||
@@ -18,7 +18,7 @@
|
||||
package org.photonvision.vision.objects;
|
||||
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
|
||||
public interface Model {
|
||||
public ObjectDetector load();
|
||||
|
||||
@@ -20,7 +20,7 @@ package org.photonvision.vision.objects;
|
||||
import java.util.List;
|
||||
import org.opencv.core.Mat;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
import org.photonvision.vision.pipe.impl.NeuralNetworkPipeResult;
|
||||
|
||||
/**
|
||||
|
||||
@@ -21,7 +21,7 @@ import java.io.File;
|
||||
import org.opencv.core.Size;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
|
||||
public class RknnModel implements Model {
|
||||
public final File modelFile;
|
||||
|
||||
@@ -21,7 +21,7 @@ import java.io.File;
|
||||
import org.opencv.core.Size;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
|
||||
public class RubikModel implements Model {
|
||||
public final File modelFile;
|
||||
|
||||
@@ -23,8 +23,8 @@ import org.photonvision.vision.frame.FrameDivisor;
|
||||
public class Draw2dAprilTagsPipe extends Draw2dTargetsPipe {
|
||||
public static class Draw2dAprilTagsParams extends Draw2dTargetsPipe.Draw2dTargetsParams {
|
||||
public Draw2dAprilTagsParams(
|
||||
boolean shouldDraw, boolean showMultipleTargets, FrameDivisor divisor) {
|
||||
super(shouldDraw, showMultipleTargets, divisor);
|
||||
boolean shouldDraw, int outputMaximumTargets, FrameDivisor divisor) {
|
||||
super(shouldDraw, outputMaximumTargets, divisor);
|
||||
// We want to show the polygon, not the rotated box
|
||||
this.showRotatedBox = false;
|
||||
this.showMaximumBox = false;
|
||||
|
||||
@@ -22,9 +22,8 @@ import org.photonvision.vision.frame.FrameDivisor;
|
||||
|
||||
public class Draw2dArucoPipe extends Draw2dTargetsPipe {
|
||||
public static class Draw2dArucoParams extends Draw2dTargetsPipe.Draw2dTargetsParams {
|
||||
public Draw2dArucoParams(
|
||||
boolean shouldDraw, boolean showMultipleTargets, FrameDivisor divisor) {
|
||||
super(shouldDraw, showMultipleTargets, divisor);
|
||||
public Draw2dArucoParams(boolean shouldDraw, int outputMaximumTargets, FrameDivisor divisor) {
|
||||
super(shouldDraw, outputMaximumTargets, divisor);
|
||||
// We want to show the polygon, not the rotated box
|
||||
this.showRotatedBox = false;
|
||||
this.showMaximumBox = false;
|
||||
|
||||
@@ -58,14 +58,10 @@ public class Draw2dTargetsPipe
|
||||
var circleColor = ColorHelper.colorToScalar(params.circleColor);
|
||||
var shapeColour = ColorHelper.colorToScalar(params.shapeOutlineColour);
|
||||
|
||||
for (int i = 0; i < (params.showMultipleTargets ? in.getSecond().size() : 1); i++) {
|
||||
for (int i = 0; i < Math.min(params.outputMaximumTargets, in.getSecond().size()); i++) {
|
||||
Point[] vertices = new Point[4];
|
||||
MatOfPoint contour = new MatOfPoint();
|
||||
|
||||
if (i != 0 && !params.showMultipleTargets) {
|
||||
break;
|
||||
}
|
||||
|
||||
TrackedTarget target = in.getSecond().get(i);
|
||||
RotatedRect r = target.getMinAreaRect();
|
||||
|
||||
@@ -233,8 +229,7 @@ public class Draw2dTargetsPipe
|
||||
public Color shapeOutlineColour = Color.MAGENTA;
|
||||
public Color textColor = Color.GREEN;
|
||||
public Color circleColor = Color.RED;
|
||||
|
||||
public final boolean showMultipleTargets;
|
||||
public int outputMaximumTargets;
|
||||
public final boolean shouldDraw;
|
||||
|
||||
public final FrameDivisor divisor;
|
||||
@@ -247,10 +242,9 @@ public class Draw2dTargetsPipe
|
||||
return shape != null && shape.shape.equals(ContourShape.Circle);
|
||||
}
|
||||
|
||||
public Draw2dTargetsParams(
|
||||
boolean shouldDraw, boolean showMultipleTargets, FrameDivisor divisor) {
|
||||
public Draw2dTargetsParams(boolean shouldDraw, int outputMaximumTargets, FrameDivisor divisor) {
|
||||
this.shouldDraw = shouldDraw;
|
||||
this.showMultipleTargets = showMultipleTargets;
|
||||
this.outputMaximumTargets = outputMaximumTargets;
|
||||
this.divisor = divisor;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,6 +17,7 @@
|
||||
|
||||
package org.photonvision.vision.pipeline;
|
||||
|
||||
import com.fasterxml.jackson.annotation.JsonAnySetter;
|
||||
import java.util.Objects;
|
||||
import org.opencv.core.Point;
|
||||
import org.photonvision.common.util.numbers.DoubleCouple;
|
||||
@@ -41,7 +42,7 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
|
||||
public boolean hueInverted = false;
|
||||
|
||||
public boolean outputShouldDraw = true;
|
||||
public boolean outputShowMultipleTargets = false;
|
||||
public int outputMaximumTargets = 20;
|
||||
|
||||
public DoubleCouple contourArea = new DoubleCouple(0.0, 100.0);
|
||||
public DoubleCouple contourRatio = new DoubleCouple(0.0, 20.0);
|
||||
@@ -90,6 +91,22 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
|
||||
public int cornerDetectionSideCount = 4;
|
||||
public double cornerDetectionAccuracyPercentage = 10;
|
||||
|
||||
/**
|
||||
* Handles backward compatibility for the deprecated outputShowMultipleTargets property. When
|
||||
* outputShowMultipleTargets is encountered during deserialization, it sets outputMaximumTargets
|
||||
* appropriately. If outputShowMultipleTargets is false, outputMaximumTargets is set to 1.
|
||||
*/
|
||||
@JsonAnySetter
|
||||
public void handleUnknownProperty(String name, Object value) {
|
||||
// Handle the old showMultipleTargets property for backward compatibility
|
||||
if ("outputShowMultipleTargets".equals(name) && value instanceof Boolean showMultipleTargets) {
|
||||
if (!showMultipleTargets) {
|
||||
// If showMultipleTargets is false, limit to 1 target
|
||||
outputMaximumTargets = 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean equals(Object o) {
|
||||
if (this == o) return true;
|
||||
@@ -97,7 +114,7 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
|
||||
if (!super.equals(o)) return false;
|
||||
AdvancedPipelineSettings that = (AdvancedPipelineSettings) o;
|
||||
return outputShouldDraw == that.outputShouldDraw
|
||||
&& outputShowMultipleTargets == that.outputShowMultipleTargets
|
||||
&& outputMaximumTargets == that.outputMaximumTargets
|
||||
&& contourSpecklePercentage == that.contourSpecklePercentage
|
||||
&& Double.compare(that.offsetDualPointAArea, offsetDualPointAArea) == 0
|
||||
&& Double.compare(that.offsetDualPointBArea, offsetDualPointBArea) == 0
|
||||
@@ -136,7 +153,7 @@ public class AdvancedPipelineSettings extends CVPipelineSettings {
|
||||
hsvValue,
|
||||
hueInverted,
|
||||
outputShouldDraw,
|
||||
outputShowMultipleTargets,
|
||||
outputMaximumTargets,
|
||||
contourArea,
|
||||
contourRatio,
|
||||
contourFullness,
|
||||
|
||||
@@ -30,6 +30,9 @@ import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
import org.photonvision.common.configuration.ConfigManager;
|
||||
import org.photonvision.common.dataflow.structures.Packet;
|
||||
import org.photonvision.common.logging.LogGroup;
|
||||
import org.photonvision.common.logging.Logger;
|
||||
import org.photonvision.common.util.math.MathUtils;
|
||||
import org.photonvision.estimation.TargetModel;
|
||||
import org.photonvision.targeting.MultiTargetPNPResult;
|
||||
@@ -49,6 +52,8 @@ import org.photonvision.vision.target.TrackedTarget;
|
||||
import org.photonvision.vision.target.TrackedTarget.TargetCalculationParameters;
|
||||
|
||||
public class AprilTagPipeline extends CVPipeline<CVPipelineResult, AprilTagPipelineSettings> {
|
||||
private static final Logger logger = new Logger(AprilTagPipeline.class, LogGroup.VisionModule);
|
||||
|
||||
private final AprilTagDetectionPipe aprilTagDetectionPipe = new AprilTagDetectionPipe();
|
||||
private final AprilTagPoseEstimatorPipe singleTagPoseEstimatorPipe =
|
||||
new AprilTagPoseEstimatorPipe();
|
||||
@@ -232,6 +237,12 @@ public class AprilTagPipeline extends CVPipeline<CVPipelineResult, AprilTagPipel
|
||||
}
|
||||
}
|
||||
|
||||
if (targetList.size() > Packet.MAX_ARRAY_LEN) {
|
||||
logger.error(
|
||||
"We have " + targetList.size() + " targets! Arbitrarily dropping some on the floor");
|
||||
targetList = targetList.subList(0, Packet.MAX_ARRAY_LEN);
|
||||
}
|
||||
|
||||
var fpsResult = calculateFPSPipe.run(null);
|
||||
var fps = fpsResult.output;
|
||||
|
||||
|
||||
@@ -40,7 +40,6 @@ public class AprilTagPipelineSettings extends AdvancedPipelineSettings {
|
||||
public AprilTagPipelineSettings() {
|
||||
super();
|
||||
pipelineType = PipelineType.AprilTag;
|
||||
outputShowMultipleTargets = true;
|
||||
targetModel = TargetModel.kAprilTag6p5in_36h11;
|
||||
cameraExposureRaw = 20;
|
||||
cameraAutoExposure = false;
|
||||
|
||||
@@ -15,23 +15,6 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
/*
|
||||
* Copyright (C) Photon Vision.
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package org.photonvision.vision.pipeline;
|
||||
|
||||
import edu.wpi.first.apriltag.AprilTagPoseEstimate;
|
||||
@@ -46,6 +29,9 @@ import org.opencv.core.Mat;
|
||||
import org.opencv.imgproc.Imgproc;
|
||||
import org.opencv.objdetect.Objdetect;
|
||||
import org.photonvision.common.configuration.ConfigManager;
|
||||
import org.photonvision.common.dataflow.structures.Packet;
|
||||
import org.photonvision.common.logging.LogGroup;
|
||||
import org.photonvision.common.logging.Logger;
|
||||
import org.photonvision.common.util.math.MathUtils;
|
||||
import org.photonvision.estimation.TargetModel;
|
||||
import org.photonvision.targeting.MultiTargetPNPResult;
|
||||
@@ -61,6 +47,8 @@ import org.photonvision.vision.target.TrackedTarget;
|
||||
import org.photonvision.vision.target.TrackedTarget.TargetCalculationParameters;
|
||||
|
||||
public class ArucoPipeline extends CVPipeline<CVPipelineResult, ArucoPipelineSettings> {
|
||||
private static final Logger logger = new Logger(ArucoPipeline.class, LogGroup.VisionModule);
|
||||
|
||||
private ArucoDetectionPipe arucoDetectionPipe = new ArucoDetectionPipe();
|
||||
private ArucoPoseEstimatorPipe singleTagPoseEstimatorPipe = new ArucoPoseEstimatorPipe();
|
||||
private final MultiTargetPNPPipe multiTagPNPPipe = new MultiTargetPNPPipe();
|
||||
@@ -237,6 +225,12 @@ public class ArucoPipeline extends CVPipeline<CVPipelineResult, ArucoPipelineSet
|
||||
}
|
||||
}
|
||||
|
||||
if (targetList.size() > Packet.MAX_ARRAY_LEN) {
|
||||
logger.error(
|
||||
"We have " + targetList.size() + " targets! Arbitrarily dropping some on the floor");
|
||||
targetList = targetList.subList(0, Packet.MAX_ARRAY_LEN);
|
||||
}
|
||||
|
||||
var fpsResult = calculateFPSPipe.run(null);
|
||||
var fps = fpsResult.output;
|
||||
|
||||
|
||||
@@ -45,7 +45,6 @@ public class ArucoPipelineSettings extends AdvancedPipelineSettings {
|
||||
public ArucoPipelineSettings() {
|
||||
super();
|
||||
pipelineType = PipelineType.Aruco;
|
||||
outputShowMultipleTargets = true;
|
||||
targetModel = TargetModel.kAprilTag6p5in_36h11;
|
||||
cameraExposureRaw = 20;
|
||||
cameraAutoExposure = true;
|
||||
|
||||
@@ -26,8 +26,6 @@ import org.photonvision.vision.pipeline.result.CVPipelineResult;
|
||||
|
||||
public abstract class CVPipeline<R extends CVPipelineResult, S extends CVPipelineSettings>
|
||||
implements Releasable {
|
||||
static final int MAX_MULTI_TARGET_RESULTS = 50;
|
||||
|
||||
protected S settings;
|
||||
protected FrameStaticProperties frameStaticProperties;
|
||||
protected QuirkyCamera cameraQuirks;
|
||||
|
||||
@@ -101,11 +101,7 @@ public class ColoredShapePipeline
|
||||
|
||||
sortContoursPipe.setParams(
|
||||
new SortContoursPipe.SortContoursParams(
|
||||
settings.contourSortMode,
|
||||
settings.outputShowMultipleTargets
|
||||
? MAX_MULTI_TARGET_RESULTS // TODO don't hardcode?
|
||||
: 1,
|
||||
frameStaticProperties));
|
||||
settings.contourSortMode, settings.outputMaximumTargets, frameStaticProperties));
|
||||
|
||||
collect2dTargetsPipe.setParams(
|
||||
new Collect2dTargetsPipe.Collect2dTargetsParams(
|
||||
@@ -131,7 +127,7 @@ public class ColoredShapePipeline
|
||||
Draw2dTargetsPipe.Draw2dTargetsParams draw2DTargetsParams =
|
||||
new Draw2dTargetsPipe.Draw2dTargetsParams(
|
||||
settings.outputShouldDraw,
|
||||
settings.outputShowMultipleTargets,
|
||||
settings.outputMaximumTargets,
|
||||
settings.streamingFrameDivisor);
|
||||
draw2DTargetsParams.showShape = true;
|
||||
draw2DTargetsParams.showMaximumBox = false;
|
||||
|
||||
@@ -82,9 +82,7 @@ public class ObjectDetectionPipeline
|
||||
|
||||
sortContoursPipe.setParams(
|
||||
new SortContoursPipe.SortContoursParams(
|
||||
settings.contourSortMode,
|
||||
settings.outputShowMultipleTargets ? MAX_MULTI_TARGET_RESULTS : 1,
|
||||
frameStaticProperties));
|
||||
settings.contourSortMode, settings.outputMaximumTargets, frameStaticProperties));
|
||||
|
||||
filterContoursPipe.setParams(
|
||||
new FilterObjectDetectionsPipe.FilterContoursParams(
|
||||
|
||||
@@ -18,18 +18,18 @@
|
||||
package org.photonvision.vision.pipeline;
|
||||
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings;
|
||||
import org.photonvision.vision.objects.Model;
|
||||
|
||||
public class ObjectDetectionPipelineSettings extends AdvancedPipelineSettings {
|
||||
public double confidence;
|
||||
public double nms; // non maximal suppression
|
||||
public NeuralNetworkPropertyManager.ModelProperties model;
|
||||
public NeuralNetworkModelsSettings.ModelProperties model;
|
||||
|
||||
public ObjectDetectionPipelineSettings() {
|
||||
super();
|
||||
this.pipelineType = PipelineType.ObjectDetection; // TODO: FIX this
|
||||
this.outputShowMultipleTargets = true;
|
||||
this.outputMaximumTargets = 20;
|
||||
cameraExposureRaw = 20;
|
||||
cameraAutoExposure = false;
|
||||
ledMode = false;
|
||||
|
||||
@@ -58,19 +58,19 @@ public class OutputStreamPipeline {
|
||||
draw2dTargetsPipe.setParams(
|
||||
new Draw2dTargetsPipe.Draw2dTargetsParams(
|
||||
settings.outputShouldDraw,
|
||||
settings.outputShowMultipleTargets,
|
||||
settings.outputMaximumTargets,
|
||||
settings.streamingFrameDivisor));
|
||||
|
||||
draw2dAprilTagsPipe.setParams(
|
||||
new Draw2dAprilTagsPipe.Draw2dAprilTagsParams(
|
||||
settings.outputShouldDraw,
|
||||
settings.outputShowMultipleTargets,
|
||||
settings.outputMaximumTargets,
|
||||
settings.streamingFrameDivisor));
|
||||
|
||||
draw2dArucoPipe.setParams(
|
||||
new Draw2dArucoPipe.Draw2dArucoParams(
|
||||
settings.outputShouldDraw,
|
||||
settings.outputShowMultipleTargets,
|
||||
settings.outputMaximumTargets,
|
||||
settings.streamingFrameDivisor));
|
||||
|
||||
draw2dCrosshairPipe.setParams(
|
||||
|
||||
@@ -85,9 +85,7 @@ public class ReflectivePipeline extends CVPipeline<CVPipelineResult, ReflectiveP
|
||||
|
||||
sortContoursPipe.setParams(
|
||||
new SortContoursPipe.SortContoursParams(
|
||||
settings.contourSortMode,
|
||||
settings.outputShowMultipleTargets ? MAX_MULTI_TARGET_RESULTS : 1,
|
||||
frameStaticProperties));
|
||||
settings.contourSortMode, settings.outputMaximumTargets, frameStaticProperties));
|
||||
|
||||
collect2dTargetsPipe.setParams(
|
||||
new Collect2dTargetsPipe.Collect2dTargetsParams(
|
||||
|
||||
@@ -465,13 +465,13 @@ public class VisionModule {
|
||||
pipelineSettings.cameraExposureRaw = 10; // reasonable default
|
||||
}
|
||||
|
||||
settables.setExposureRaw(pipelineSettings.cameraExposureRaw);
|
||||
try {
|
||||
settables.setAutoExposure(pipelineSettings.cameraAutoExposure);
|
||||
} catch (VideoException e) {
|
||||
logger.error("Unable to set camera auto exposure!");
|
||||
logger.error(e.toString());
|
||||
}
|
||||
settables.setExposureRaw(pipelineSettings.cameraExposureRaw);
|
||||
if (cameraQuirks.hasQuirk(CameraQuirk.Gain)) {
|
||||
// If the gain is disabled for some reason, re-enable it
|
||||
if (pipelineSettings.cameraGain == -1) pipelineSettings.cameraGain = 75;
|
||||
|
||||
@@ -25,7 +25,7 @@ import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.locks.ReentrantLock;
|
||||
import org.opencv.core.Point;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
import org.photonvision.common.dataflow.DataChangeSubscriber;
|
||||
import org.photonvision.common.dataflow.events.DataChangeEvent;
|
||||
import org.photonvision.common.dataflow.events.IncomingWebSocketEvent;
|
||||
|
||||
@@ -51,7 +51,7 @@ public class BenchmarkTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
|
||||
@@ -105,7 +105,7 @@ public class BenchmarkTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
|
||||
|
||||
@@ -66,7 +66,7 @@ public class ShapeBenchmarkTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
pipeline.getSettings().contourShape = ContourShape.Custom;
|
||||
@@ -87,7 +87,7 @@ public class ShapeBenchmarkTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
pipeline.getSettings().contourShape = ContourShape.Custom;
|
||||
@@ -109,7 +109,7 @@ public class ShapeBenchmarkTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
pipeline.getSettings().contourShape = ContourShape.Custom;
|
||||
@@ -131,7 +131,7 @@ public class ShapeBenchmarkTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
pipeline.getSettings().contourShape = ContourShape.Custom;
|
||||
|
||||
@@ -25,13 +25,13 @@ import java.util.LinkedList;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Version;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
import org.photonvision.common.util.file.JacksonUtils;
|
||||
|
||||
public class NeuralNetworkPropertyManagerTest {
|
||||
@Test
|
||||
void testSerialization() {
|
||||
var nnpm = new NeuralNetworkPropertyManager();
|
||||
var nnpm = new NeuralNetworkModelsSettings();
|
||||
// Path is always serialized as absolute; for the test to pass, this must also be made absolute
|
||||
nnpm.addModelProperties(
|
||||
new ModelProperties(
|
||||
@@ -45,7 +45,7 @@ public class NeuralNetworkPropertyManagerTest {
|
||||
String result = assertDoesNotThrow(() -> JacksonUtils.serializeToString(nnpm));
|
||||
var deserializedNnpm =
|
||||
assertDoesNotThrow(
|
||||
() -> JacksonUtils.deserialize(result, NeuralNetworkPropertyManager.class));
|
||||
() -> JacksonUtils.deserialize(result, NeuralNetworkModelsSettings.class));
|
||||
assertEquals(nnpm.getModels()[0], deserializedNnpm.getModels()[0]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -20,23 +20,32 @@ package org.photonvision.common.configuration;
|
||||
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import edu.wpi.first.cscore.UsbCameraInfo;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Path;
|
||||
import java.util.Collection;
|
||||
import java.util.List;
|
||||
import org.apache.commons.io.FileUtils;
|
||||
import org.junit.jupiter.api.BeforeAll;
|
||||
import org.junit.jupiter.api.Order;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.photonvision.common.LoadJNI;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager.Family;
|
||||
import org.photonvision.common.util.TestUtils;
|
||||
import org.photonvision.vision.camera.CameraQuirk;
|
||||
import org.photonvision.vision.camera.PVCameraInfo;
|
||||
import org.photonvision.vision.pipeline.AdvancedPipelineSettings;
|
||||
import org.photonvision.vision.pipeline.AprilTagPipelineSettings;
|
||||
import org.photonvision.vision.pipeline.CVPipelineSettings;
|
||||
import org.photonvision.vision.pipeline.ColoredShapePipelineSettings;
|
||||
import org.photonvision.vision.pipeline.ObjectDetectionPipelineSettings;
|
||||
import org.photonvision.vision.pipeline.PipelineType;
|
||||
import org.photonvision.vision.pipeline.ReflectivePipelineSettings;
|
||||
|
||||
public class SQLConfigTest {
|
||||
@TempDir private static Path tmpDir;
|
||||
@TempDir private Path tmpDir;
|
||||
|
||||
@BeforeAll
|
||||
public static void init() {
|
||||
@@ -84,11 +93,15 @@ public class SQLConfigTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testLoad2024_3_1() {
|
||||
var cfgLoader =
|
||||
new SqlConfigProvider(
|
||||
TestUtils.getConfigDirectoriesPath(false)
|
||||
.resolve("photonvision_config_from_v2024.3.1"));
|
||||
public void testLoad2024_3_1() throws IOException {
|
||||
// Copy the 2024.3.1 config to a temp dir
|
||||
FileUtils.copyDirectory(
|
||||
TestUtils.getConfigDirectoriesPath(false)
|
||||
.resolve("photonvision_config_from_v2024.3.1")
|
||||
.toFile(),
|
||||
tmpDir.resolve("photonvision_config_from_v2024.3.1").toFile());
|
||||
|
||||
var cfgLoader = new SqlConfigProvider(tmpDir.resolve("photonvision_config_from_v2024.3.1"));
|
||||
|
||||
assertDoesNotThrow(cfgLoader::load);
|
||||
|
||||
@@ -104,4 +117,97 @@ public class SQLConfigTest {
|
||||
.hasQuirk(c));
|
||||
}
|
||||
}
|
||||
|
||||
void common2025p3p1Assertions(PhotonConfiguration config) {
|
||||
// Make sure we got 8 cameras
|
||||
assertEquals(8, config.getCameraConfigurations().size());
|
||||
|
||||
// Make sure exactly 2 have object detection pipelines
|
||||
long count =
|
||||
config.getCameraConfigurations().values().stream()
|
||||
.filter(
|
||||
c ->
|
||||
c.pipelineSettings.stream()
|
||||
.anyMatch(s -> s instanceof ObjectDetectionPipelineSettings))
|
||||
.count();
|
||||
assertEquals(2, count);
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testLoadNewNNMM() throws JsonProcessingException, IOException {
|
||||
var folder = tmpDir.resolve("2025.3.1-old-nnmm");
|
||||
FileUtils.copyDirectory(
|
||||
TestUtils.getConfigDirectoriesPath(false).resolve("2025.3.1-old-nnmm").toFile(),
|
||||
folder.toFile());
|
||||
|
||||
var cfgManager = new ConfigManager(folder, new SqlConfigProvider(folder));
|
||||
|
||||
// Replace global configmanager
|
||||
ConfigManager.INSTANCE = cfgManager;
|
||||
|
||||
assertDoesNotThrow(cfgManager::load);
|
||||
|
||||
System.out.println(cfgManager.getConfig());
|
||||
common2025p3p1Assertions(cfgManager.getConfig());
|
||||
|
||||
// And we now see two models
|
||||
NeuralNetworkModelManager.getInstance();
|
||||
// force us to allow RKNN
|
||||
NeuralNetworkModelManager.getInstance().supportedBackends.add(Family.RKNN);
|
||||
NeuralNetworkModelManager.getInstance().discoverModels();
|
||||
assertEquals(5, NeuralNetworkModelManager.getInstance().models.get(Family.RKNN).size());
|
||||
|
||||
ConfigManager.getInstance().saveToDisk();
|
||||
|
||||
// Now that we have the config saved, load it again
|
||||
var reloadedProvider = new SqlConfigProvider(folder);
|
||||
reloadedProvider.load();
|
||||
common2025p3p1Assertions(reloadedProvider.getConfig());
|
||||
|
||||
// And make sure NNPM has all 5 models
|
||||
assertEquals(5, reloadedProvider.getConfig().neuralNetworkPropertyManager().getModels().length);
|
||||
|
||||
ConfigManager.INSTANCE = null;
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testMaxDetectionsMigration() throws IOException {
|
||||
var folder = tmpDir.resolve("2025.3.1-old-nnmm");
|
||||
FileUtils.copyDirectory(
|
||||
TestUtils.getConfigDirectoriesPath(false).resolve("2025.3.1-old-nnmm").toFile(),
|
||||
folder.toFile());
|
||||
|
||||
var cfgManager = new ConfigManager(folder, new SqlConfigProvider(folder));
|
||||
|
||||
// Replace global configmanager
|
||||
ConfigManager.INSTANCE = cfgManager;
|
||||
|
||||
assertDoesNotThrow(cfgManager::load);
|
||||
|
||||
Collection<CameraConfiguration> cameraConfigs =
|
||||
cfgManager.getConfig().getCameraConfigurations().values();
|
||||
|
||||
for (CameraConfiguration cc : cameraConfigs) {
|
||||
for (CVPipelineSettings ps : cc.pipelineSettings) {
|
||||
if (ps instanceof AdvancedPipelineSettings adps) {
|
||||
AdvancedPipelineSettings finalPs = adps;
|
||||
if (finalPs.pipelineType.equals(PipelineType.AprilTag)
|
||||
|| finalPs.pipelineType.equals(PipelineType.Aruco)) {
|
||||
// Tag pipelines don't have max detections, so skip
|
||||
continue;
|
||||
} else if (finalPs.pipelineNickname.equals("TEST MIGRATION")) {
|
||||
// This is our colored shape pipeline that we set to 1 before saving
|
||||
assertEquals(1, finalPs.outputMaximumTargets);
|
||||
} else {
|
||||
// All others should be at default 20
|
||||
assertEquals(20, finalPs.outputMaximumTargets);
|
||||
}
|
||||
} else {
|
||||
System.out.println("Skipping pipeline settings type: " + ps.getClass().getSimpleName());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ConfigManager.INSTANCE = null;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -137,4 +137,31 @@ public class AprilTagTest {
|
||||
assertEquals(2, pose.getTranslation().getY(), 0.2);
|
||||
assertEquals(0.0, pose.getTranslation().getZ(), 0.2);
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testManyDetections() {
|
||||
// Given a 36h11 pipeline
|
||||
var pipeline = new AprilTagPipeline();
|
||||
pipeline.getSettings().inputShouldShow = true;
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().solvePNPEnabled = true;
|
||||
pipeline.getSettings().cornerDetectionAccuracyPercentage = 4;
|
||||
pipeline.getSettings().cornerDetectionUseConvexHulls = true;
|
||||
pipeline.getSettings().tagFamily = AprilTagFamily.kTag36h11;
|
||||
pipeline.getSettings().outputMaximumTargets = 3; // bogus
|
||||
|
||||
// when we have a picture with 280 targets
|
||||
var frameProvider =
|
||||
new FileFrameProvider(
|
||||
TestUtils.getApriltagImagePath(TestUtils.ApriltagTestImages.k36h11_stress_test, false),
|
||||
TestUtils.WPI2020Image.FOV,
|
||||
TestUtils.getCoeffs(TestUtils.LIMELIGHT_480P_CAL_FILE, false));
|
||||
frameProvider.requestFrameThresholdType(pipeline.getThresholdType());
|
||||
|
||||
CVPipelineResult pipelineResult;
|
||||
pipelineResult = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
|
||||
|
||||
// the pipeline will only give us Byte.MAX_VALUE many
|
||||
assertEquals(Byte.MAX_VALUE, pipelineResult.targets.size());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -94,7 +94,7 @@ public class CirclePNPTest {
|
||||
pipeline.getSettings().cameraCalibration = getCoeffs(LIFECAM_480P_CAL_FILE);
|
||||
pipeline.getSettings().targetModel = TargetModel.kCircularPowerCell7in;
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = false;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Single;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
pipeline.getSettings().contourShape = ContourShape.Circle;
|
||||
@@ -144,7 +144,7 @@ public class CirclePNPTest {
|
||||
settings.hsvSaturation.set(100, 255);
|
||||
settings.hsvValue.set(190, 255);
|
||||
settings.outputShouldDraw = true;
|
||||
settings.outputShowMultipleTargets = true;
|
||||
settings.outputMaximumTargets = 20;
|
||||
settings.contourGroupingMode = ContourGroupingMode.Dual;
|
||||
settings.contourIntersection = ContourIntersectionDirection.Up;
|
||||
|
||||
|
||||
@@ -102,7 +102,7 @@ public class ColoredShapePipelineTest {
|
||||
settings.hsvSaturation.set(100, 255);
|
||||
settings.hsvValue.set(100, 255);
|
||||
settings.outputShouldDraw = true;
|
||||
settings.outputShowMultipleTargets = true;
|
||||
settings.outputMaximumTargets = 20;
|
||||
settings.contourGroupingMode = ContourGroupingMode.Single;
|
||||
settings.contourIntersection = ContourIntersectionDirection.Up;
|
||||
settings.contourShape = ContourShape.Triangle;
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
/*
|
||||
* Copyright (C) Photon Vision.
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package org.photonvision.vision.pipeline;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
|
||||
import java.nio.file.Path;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.photonvision.common.LoadJNI;
|
||||
import org.photonvision.common.configuration.ConfigManager;
|
||||
import org.photonvision.common.util.TestUtils;
|
||||
import org.photonvision.vision.camera.QuirkyCamera;
|
||||
import org.photonvision.vision.frame.provider.FileFrameProvider;
|
||||
import org.photonvision.vision.opencv.ContourShape;
|
||||
import org.photonvision.vision.pipe.impl.HSVPipe;
|
||||
import org.photonvision.vision.pipeline.result.CVPipelineResult;
|
||||
|
||||
public class MaxDetectionsTest {
|
||||
@Test
|
||||
public void testMaxDetections() {
|
||||
LoadJNI.loadLibraries();
|
||||
ConfigManager.getInstance().load();
|
||||
|
||||
ColoredShapePipeline pipeline = new ColoredShapePipeline();
|
||||
|
||||
pipeline.settings.contourShape = ContourShape.Circle;
|
||||
pipeline.settings.hsvHue.set(140, 160);
|
||||
pipeline.settings.hsvSaturation.set(226, 246);
|
||||
pipeline.settings.hsvValue.set(188, 208);
|
||||
pipeline.settings.maxCannyThresh = 90;
|
||||
pipeline.settings.circleAccuracy = 20;
|
||||
pipeline.settings.circleDetectThreshold = 5;
|
||||
|
||||
Path path =
|
||||
TestUtils.getResourcesFolderPath(false).resolve("testimages/polygons/ColoredShapeTest.png");
|
||||
|
||||
var frameProvider = new FileFrameProvider(path, TestUtils.WPI2019Image.FOV);
|
||||
|
||||
// VisionRunner normally does this
|
||||
var hsvParams =
|
||||
new HSVPipe.HSVParams(
|
||||
pipeline.getSettings().hsvHue,
|
||||
pipeline.getSettings().hsvSaturation,
|
||||
pipeline.getSettings().hsvValue,
|
||||
pipeline.getSettings().hueInverted);
|
||||
frameProvider.requestHsvSettings(hsvParams);
|
||||
frameProvider.requestFrameThresholdType(pipeline.getThresholdType());
|
||||
|
||||
CVPipelineResult result = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
|
||||
TestUtils.showImage(result.inputAndOutputFrame.processedImage.getMat(), "Max Detections Test");
|
||||
|
||||
assertEquals(20, result.targets.size());
|
||||
|
||||
pipeline.settings.outputMaximumTargets = 5;
|
||||
result = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
|
||||
assertEquals(5, result.targets.size());
|
||||
|
||||
pipeline.settings.outputMaximumTargets = 50;
|
||||
result = pipeline.run(frameProvider.get(), QuirkyCamera.DefaultCamera);
|
||||
// 24 circles, but we only detect 22
|
||||
assertEquals(22, result.targets.size());
|
||||
}
|
||||
}
|
||||
@@ -41,7 +41,7 @@ public class ReflectivePipelineTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().outputMaximumTargets = 20;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
|
||||
@@ -120,7 +120,7 @@ public class ReflectivePipelineTest {
|
||||
settings.hsvSaturation.set(100, 255);
|
||||
settings.hsvValue.set(190, 255);
|
||||
settings.outputShouldDraw = true;
|
||||
settings.outputShowMultipleTargets = true;
|
||||
settings.outputMaximumTargets = 20;
|
||||
settings.contourGroupingMode = ContourGroupingMode.Dual;
|
||||
settings.contourIntersection = ContourIntersectionDirection.Up;
|
||||
|
||||
|
||||
@@ -89,7 +89,6 @@ public class SolvePNPTest {
|
||||
pipeline.getSettings().hsvSaturation.set(100, 255);
|
||||
pipeline.getSettings().hsvValue.set(190, 255);
|
||||
pipeline.getSettings().outputShouldDraw = true;
|
||||
pipeline.getSettings().outputShowMultipleTargets = true;
|
||||
pipeline.getSettings().solvePNPEnabled = true;
|
||||
pipeline.getSettings().contourGroupingMode = ContourGroupingMode.Dual;
|
||||
pipeline.getSettings().contourIntersection = ContourIntersectionDirection.Up;
|
||||
@@ -225,7 +224,6 @@ public class SolvePNPTest {
|
||||
settings.hsvSaturation.set(100, 255);
|
||||
settings.hsvValue.set(190, 255);
|
||||
settings.outputShouldDraw = true;
|
||||
settings.outputShowMultipleTargets = true;
|
||||
settings.contourGroupingMode = ContourGroupingMode.Dual;
|
||||
settings.contourIntersection = ContourIntersectionDirection.Up;
|
||||
|
||||
|
||||
@@ -142,8 +142,8 @@ PhotonCamera::PhotonCamera(nt::NetworkTableInstance instance,
|
||||
rootTable->GetIntegerTopic("pipelineIndexRequest").Publish()),
|
||||
pipelineIndexSub(
|
||||
rootTable->GetIntegerTopic("pipelineIndexState").Subscribe(0)),
|
||||
ledModePub(mainTable->GetIntegerTopic("ledMode").Publish()),
|
||||
ledModeSub(mainTable->GetIntegerTopic("ledMode").Subscribe(0)),
|
||||
ledModePub(mainTable->GetIntegerTopic("ledModeRequest").Publish()),
|
||||
ledModeSub(mainTable->GetIntegerTopic("ledModeState").Subscribe(0)),
|
||||
versionEntry(mainTable->GetStringTopic("version").Subscribe("")),
|
||||
cameraIntrinsicsSubscriber(
|
||||
rootTable->GetDoubleArrayTopic("cameraIntrinsics").Subscribe({})),
|
||||
|
||||
@@ -39,7 +39,7 @@ import org.opencv.imgcodecs.Imgcodecs;
|
||||
import org.photonvision.common.configuration.ConfigManager;
|
||||
import org.photonvision.common.configuration.NetworkConfig;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelManager;
|
||||
import org.photonvision.common.configuration.NeuralNetworkPropertyManager.ModelProperties;
|
||||
import org.photonvision.common.configuration.NeuralNetworkModelsSettings.ModelProperties;
|
||||
import org.photonvision.common.dataflow.DataChangeDestination;
|
||||
import org.photonvision.common.dataflow.DataChangeService;
|
||||
import org.photonvision.common.dataflow.events.IncomingWebSocketEvent;
|
||||
|
||||
BIN
photon-server/src/main/resources/models/fuelV1-yolo11n.rknn
Normal file
BIN
photon-server/src/main/resources/models/fuelV1-yolo11n.rknn
Normal file
Binary file not shown.
BIN
photon-server/src/main/resources/models/fuelV1-yolo11n.tflite
Normal file
BIN
photon-server/src/main/resources/models/fuelV1-yolo11n.tflite
Normal file
Binary file not shown.
@@ -31,6 +31,8 @@ public class Packet {
|
||||
// Read and write positions.
|
||||
int readPos, writePos;
|
||||
|
||||
public static final int MAX_ARRAY_LEN = Byte.MAX_VALUE;
|
||||
|
||||
/**
|
||||
* Constructs an empty packet. This buffer will dynamically expand if we need more data space.
|
||||
*
|
||||
|
||||
@@ -1,84 +1,99 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1tMAqVl4p58r"
|
||||
},
|
||||
"source": [
|
||||
"## YOLO to Rubik TFlite Conversion"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "nAbygyUYp58s"
|
||||
},
|
||||
"source": [
|
||||
"#### Requirements\n",
|
||||
"\n",
|
||||
"This notebook can be run on Colab. However, Colab has some incompatibility issues that result in needing to restart the notebook in the middle of the run. This is normal, and after restarting you should rerun the below cell.\n",
|
||||
"\n",
|
||||
"If you aren't using Google Colab, we recommend creating a [Python venv](https://docs.python.org/3/library/venv.html) so that the packages installed for conversion do not conflict with your existing setup.\n",
|
||||
"\n",
|
||||
"Prior to running the notebook, it is necessary to make an account on [Qualcomm's AI Hub](https://app.aihub.qualcomm.com/account/), and obtain your API token. Then, replace <YOUR_API_TOKEN> with your API token in the cell below.\n",
|
||||
"\n",
|
||||
"Documentation for the Qualcomm AI Hub can be found [here](https://app.aihub.qualcomm.com/docs/index.html).\n",
|
||||
"\n",
|
||||
"You should also have a PyTorch model (ending in `.pt`) that's been uploaded to the runtime that you intend to convert. After uploading, copy it's absolute path by right-clicking on the file, and replace /PATH/TO/WEIGHTS.\n",
|
||||
"\n",
|
||||
"**NOTE: your API key will be listed in the output, and should therefore be redacted if the output is shared.**\n",
|
||||
"\n",
|
||||
"Once the run has finished, open the AI Hub link, and download the tflite model for the job you just ran.\n",
|
||||
"\n",
|
||||
"If you want to use this notebook to convert a yolo11 model, you'll need to replace all instances of `yolov8` in the cell below with `yolov11`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 1000
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "1tMAqVl4p58r"
|
||||
},
|
||||
"source": [
|
||||
"## YOLO to Rubik TFlite Conversion"
|
||||
]
|
||||
},
|
||||
"id": "aX3JcSFKp58s",
|
||||
"outputId": "f2cdadd2-c448-4d8c-c681-c19decef7f3e"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# This installs Python package\n",
|
||||
"!pip install qai-hub-models[yolov8_det]\n",
|
||||
"# sets up AI Hub enviroment\n",
|
||||
"!qai-hub configure --api_token <YOUR_API_TOKEN>\n",
|
||||
"# Converts the model to be ran on RB3Gen2\n",
|
||||
"!yes | python -m qai_hub_models.models.yolov8_det.export --quantize w8a8 --device=\"RB3 Gen 2 (Proxy)\" --ckpt-name /PATH/TO/WEIGHTS --device-os linux --target-runtime tflite --output-dir .\n"
|
||||
]
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "nAbygyUYp58s"
|
||||
},
|
||||
"source": [
|
||||
"#### Requirements\n",
|
||||
"\n",
|
||||
"This notebook can be run on Colab. However, Colab has some incompatibility issues that result in needing to restart the notebook in the middle of the run. This is normal, and after restarting you should rerun the below cell.\n",
|
||||
"\n",
|
||||
"If you aren't using Google Colab, we recommend creating a [Python venv](https://docs.python.org/3/library/venv.html) so that the packages installed for conversion do not conflict with your existing setup.\n",
|
||||
"\n",
|
||||
"Prior to running the notebook, it is necessary to make an account on [Qualcomm's AI Hub](https://app.aihub.qualcomm.com/account/), and obtain your API token. Then, replace <YOUR_API_TOKEN> with your API token in the cell below.\n",
|
||||
"\n",
|
||||
"Documentation for the Qualcomm AI Hub can be found [here](https://app.aihub.qualcomm.com/docs/index.html).\n",
|
||||
"\n",
|
||||
"You should also have a PyTorch model (ending in `.pt`) that's been uploaded to the runtime that you intend to convert. After uploading, copy it's absolute path by right-clicking on the file, and replace /PATH/TO/WEIGHTS.\n",
|
||||
"\n",
|
||||
"**NOTE: your API key will be listed in the output, and should therefore be redacted if the output is shared.**\n",
|
||||
"\n",
|
||||
"Once the run has finished, open the AI Hub link, and download the tflite model for the job you just ran.\n",
|
||||
"\n",
|
||||
"If you want to use this notebook to convert a yolo11 model, you'll need to replace all instances of `yolov8` in the cell below with `yolov11`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "aX3JcSFKp58s",
|
||||
"outputId": "7fffa581-fb85-4808-84d6-b737142011b2"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# This installs Python package\n",
|
||||
"!pip install qai-hub-models[yolov8_det]\n",
|
||||
"# sets up AI Hub enviroment\n",
|
||||
"!qai-hub configure --api_token [YOUR API KEY]\n",
|
||||
"# Converts the model to be ran on RB3Gen2\n",
|
||||
"!yes | python -m qai_hub_models.models.yolov8_det.export --quantize w8a8 --device=\"Dragonwing RB3 Gen 2 Vision Kit\" --ckpt-name /content/PyTorch_model.pt --device-os 1.6 --target-runtime tflite --output-dir .\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "jl9M3nPDqlue",
|
||||
"outputId": "2df2c1af-c4ee-4cbf-dad6-7ec3afd28337"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# check valid devices / os types\n",
|
||||
"!qai-hub list-devices"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0I2cXQO4p58s"
|
||||
},
|
||||
"source": [
|
||||
"Modified from https://github.com/ramalamadingdong/yolo-rb3gen2-trainer/blob/main/AI_Hub_Quanitization_RB3Gen2.ipynb"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.7"
|
||||
}
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "0I2cXQO4p58s"
|
||||
},
|
||||
"source": [
|
||||
"Modified from https://github.com/ramalamadingdong/yolo-rb3gen2-trainer/blob/main/AI_Hub_Quanitization_RB3Gen2.ipynb"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"name": "python",
|
||||
"version": "3.11.7"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
||||
@@ -44,7 +44,6 @@ dependencies {
|
||||
|
||||
implementation "commons-io:commons-io:2.11.0"
|
||||
implementation "commons-cli:commons-cli:1.5.0"
|
||||
implementation "org.apache.commons:commons-exec:1.3"
|
||||
|
||||
testImplementation(platform('org.junit:junit-bom:5.11.4'))
|
||||
testImplementation 'org.junit.jupiter:junit-jupiter-api'
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
algae
|
||||
@@ -0,0 +1,2 @@
|
||||
bumpers
|
||||
fuel
|
||||
@@ -0,0 +1,2 @@
|
||||
Coral
|
||||
Algae
|
||||
@@ -0,0 +1 @@
|
||||
fuel
|
||||
@@ -0,0 +1,2 @@
|
||||
Bumper
|
||||
Fuel
|
||||
BIN
test-resources/old_configs/2025.3.1-old-nnmm/photon.sqlite
Normal file
BIN
test-resources/old_configs/2025.3.1-old-nnmm/photon.sqlite
Normal file
Binary file not shown.
BIN
test-resources/testimages/apriltag/36h11_stress_test.png
Normal file
BIN
test-resources/testimages/apriltag/36h11_stress_test.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 178 KiB |
BIN
test-resources/testimages/polygons/ColoredShapeTest.png
Normal file
BIN
test-resources/testimages/polygons/ColoredShapeTest.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 126 KiB |
Reference in New Issue
Block a user