## Description
See #2026 for the previous iteration of this PR.
This adds the RKNN model trained on the COCO dataset as one of the
models shipped with PV. This model is fairly general, and has been
trained to identify a number of objects, including people, animals,
cars, and more. This model is meant for teams to test object detection,
particularly for teams who might not have access to the game elements
that our other models are trained on.
It additionally acknowledges Ultralytics for the model, and includes the
AGPL copyleft license.
## Meta
Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_
- [x] If this PR changes behavior or adds a feature, user documentation
is updated
- [x] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [x] If this PR touches configuration, this is backwards compatible
with settings back to v2024.3.1
- [x] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [x] If this PR addresses a bug, a regression test for it is added
This reverts commit 753123844b.
## Description
The COCO dataset contains images that use the NC and/or ND variants of
the CC license, and distributing a model based on that dataset is most
likely a violation of licenses. Additionally, the model is licensed under AGPL,
which might be a concern for PhotonVision, and at a minimum, there's no
license file bundled with the model right now.
## Meta
Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_
- [x] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with settings back to v2024.3.1
- [ ] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [ ] If this PR addresses a bug, a regression test for it is added
## Description
This adds the RKNN model trained on the COCO dataset as one of the
models shipped with PV. This model is fairly general, and has been
trained to identify a number of objects, including people, animals,
cars, and more. This model is meant for teams to test object detection,
particularly for teams who might not have access to the game elements
that our other models are trained on.
## Meta
Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_
- [x] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with settings back to v2024.3.1
- [ ] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [ ] If this PR addresses a bug, a regression test for it is added
This PR is for part 1 of #1354. It focuses on adding a model selection
interface for models that exist in `photonvision_config/models/`. Upon
completion we can ship more than 1 model and users could upload their
own through `ssh` without deleting the shipped model. This PR also adds
the abstractions need to support more DNN backends (say OpenCV, or RPI
AI Kit)
Up next is adding a CRUD interface for managing models through the UI.
Tested on Orange Pi 5 and Cool Pi 4B. Merge with parts of the OpenCV DNN PR.
Adds support for YOLOv5s models for Rockchip CPUs with a NPU. Right now hard coded to a note model from alex_idk. Very much still incubating and largely untested.
* Serialize all calibration data
* Run lint
* typing nit
* fix code
* move these tables around some
* Add cool formatting
* add request to get snapshots by resolution and camera
* re-enable all resolutions
* add wip so i can change computers (SQUASH ME AND KILL ME AHHHH)
* Get everything working but viewing snapshots
* Update RequestHandler.java
* Update CameraCalibrationInfoCard.vue
* Update CameraCalibrationInfoCard.vue
* add observation viewer
* round
* fix illiegal import
* Swap to PNG and serialize insolution
* move import/export buttons TO THE TOP
* Update WebsocketDataTypes.ts
* Add snapshotname to observation
* Refactor to serialize snapshot image itself
* Run lint
* Use new base64 image data in info card
* Update SettingTypes.ts
* Create calibration json -> mrcal converter script
* Update calibrationUtils.py
* Fix calibrate NPEs in teest
* Run lint
* Always run cornersubpix
* Update CameraCalibrationInfoCard.vue
Update CameraCalibrationInfoCard.vue
* Update OpenCVHelp.java
* Update OpenCVHelp.java
* Replace test mode camera JSONs
* Run wpiformat
* Revert intrinsics but keep other data
* Remove misc comments
* Rename JsonMat->JsonImageMat and add calobject_warp
* Update Server.java
* Rename cameraExtrinsics to distCoeffs
* fix typing issues
* use util methods
* Formatting fixes
* fix styling
* move to devTools
* remove unneeded or unused imports
* Remove fixed-right css
If its really that big of a deal, we can add it back later, kind of a drag to fix rn.
* Create util method
* Remove extra legacy calibration things
---------
Co-authored-by: Sriman Achanta <68172138+srimanachanta@users.noreply.github.com>
Deals with new otherpaths on pi 5 CSI cameras and bumps libcamera driver to latest from the pi 5 PR
---------
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
* Reordered ov video modes to be lowest-to-highest res
* Save off sensor model on init. Guard against low, crashy exposures.
* Pulled in matt's fixups from https://github.com/PhotonVision/photon-libcamera-gl-driver/suites/10144555465/artifacts/495489276
* Further autoexposure tweaks for picam v1
* Allow undercores in camera rename
* Additional guarding against output images being empty
* lock out auto-exposure on ov9281's
* Guarding stream pipelines against empty frames from cameras. Rearranged driver stream to resize first, then draw crosshairs (matchces with other pipelines now).
* NT Priority fixup - if client is sending commands on NT, its nt value should win over anything done from the UI
* Synchronous pipline adjustmet fix, method cleanup
* lint
* circle pipe and data publish bugfixes
* lint
* Pulled in matt's latest .so and re-enabled auto exposure on 9281's
* Added throttling reasons and cpu uptime
* spotless
* adding tooltips for the acronyms used
* Added icon for suggesting folks should attempt a hover-over for tooltip
* wip making the implementaiton more platform independent
* spotless
* wpiformat
* wpilibformat pt 2
* Add UI-side changes for invertable hue slider
* Add hue inverted range
* Add new slider backgrounds to threshold tab
* Run spotless
* Updated libpicam.so to artifact built from commit c458bab87740 in that repo on gerth2's pi.
* undo the java-side hack since isVCSMSupported is fixed
* Hook up hue inversion frontend to backend and UI tweaks
* Remove unused .flipped class
* Fix hueInverted name in Vue.js store
Co-authored-by: Declan Freeman-Gleason <declanfreemangleason@gmail.com>
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
Co-authored-by: Chris Gerth <chrisgerth010592@gmail.com>
* Fixup colored shape backend code
* More colored shape stuff
* Start adding shape change to drawing
* Mostly works!
* Add powercell image for shape test mode
* Make super-duper-sure to release stuff
Fixes colored shape leak
* Move approx poly dp into Contour
* Adjust epsilon threshold
* Add dialog to change pipeline type
* Run spotless
* Make yes red :>
* Move "no" button
* Fix duplication/deletion name logic and switching
* Fix compilation errors from rebase
* Update VisionSourceManager.java
* Update type dialog, remove duplicate popup
The dropdown still switches even if the user says "no" tho
Co-authored-by: Banks Troutman <btrout.dhrs@gmail.com>
* Add libpicam with gain slider bugfix
* Patches to get zero-copy working with Pi3.
-- Success/Failure mistmatch assumptions - lots of functions in the JNI return true on failure (not true on success)
-- isVSCMSupported() is currently implemented to be "isVSCMNotSupported()"
Likely, we'll want at least some .so changes
Co-authored-by: Chris Gerth <chrisgerth010592@gmail.com>
* Add native stuff
* use runtimeloader
* add more native methods
* more stuff
* Switch JNI methods to static
* Remove non-java classes from the picam jni
* Add gradle task for JNI generation
* Migrate my previous GPU accel work
* Initial work on defining JNI interface
* Change libpicam to a symlink for now
* Initial work on adding no-copy OMX GPU accel on the pi
* Make DIRECT_OMX GPU accel mode not crash
* Clean up OMX changes (still not getting valid data back)
* Re-add GPU unit test
* A couple debugging tweaks/notes
* Add temporary special cases to get RGB out of ProcessingMode.NONE
* Code clarity improvements; fix possible VBO bug
* Get DIRECT_OMX working
* Remove some debugging switches in GPUAccelerator
* Pipe in VCSM stuff to read out pixels FAST
* Apply Spotless
* Revert versioningHelper changes
* Add missing import
* Convert to MMAL and move everything to native
* Re-add shared object
* Rework to use MMAL and do everything natively
* Condense pipeline settings classes
* Add OutputStreamPipeline
* Apply spotless
* Fix duplicate variable inits and add more video modes
* Integrate color frames and latency measurements for GPU
* Fix camera detection on pi and other platforms
* Add proper color copy disabling and camera settings calls
* Fix things that were broken by rebase
* Fix spotless issues and remove uneeded prints
* Remove libpicam symlink
* Fix stream resolution limiting
* Remove testing code in GPUAcceleratedHSVPipe
* Make profiling options general to all computers
* Make PicamJNI load from resources
* run spotlessApply
* Address review comments
* Update Maven repo for JOGL
* Fix release race condition
* Only run GPU accel test on the pi
* Lint fix and merge conflict accident fixes
* Make Jackson ignore extra fields when unmarshalling HardwareConfig
* Fix Mat releasing data race
* Spotless apply
* Remove broken header generation task
* Fix shared library loading typo
* Add a ZeroCopyPicam quirk to allow setting gain with the MMAL backend
* Make sure that exposure/brightness/gain get set after res changes
* Make rawInputMat properly local
* Remove bogus set of shouldRun flag
* Clean up small GPUHSVPipe print
* Add in some things that missed the ZeroCopyPiCameraSource rename
* Fix incorrect scoping introduced in past rebase
* Don't filter out too-low resolutions
* Only show latency when GPU accel is enabled
* Don't free Mats in stream thread before we use them
* Fix use-after-free and latency caluclation bugs on USB camera source
* Update libpicam
* Remove unwanted print
* Add libpicam forceLoad in unit test
* Fix streaming during camera calibration
* Fix zerocopy Picam calculation
* Use logger trace method instead of raw prints
* Fix calibration and driver mode pipes with the Picam
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
Co-authored-by: Banks Troutman <btrout.dhrs@gmail.com>
* Add camera config for calibration to test mode
* Add 3D capabilities to test mode
* Add resources to main for jar testmode
* Refactor path utilities for test mode
* Apply spotless
* Rework settings page; touch up contour, output, and 3D tabs; font sizing
No stream placeholder; driver mode refined; cameras page
Make settings snackbar work
Lint fix
Fix settings page padding
Actually hide settings fields if unsupported
* Make toggle buttons less confusing; fix driver toggle; form validation
* Make eyedropper work and make input/select styling more consistent
* Fix color picker and tabbing bugs
* Set up camera and settings pages to talk to the backend
* Add auto reconnect
* Add lots of tooltips and improve related thematic consistency
* Only show output stream while color picking
* Unbreak robot offset
* Increase tooltip delay and refactor tooltip label into a component
* Remove toggle button switching behavior
* Fix PnP tab and add a flag to disable FOV configuration
* Move FPS indicator
* Make GPU acceleration status use one value in the store
* Only allow IPv4 static IPs and remove accidentally committed index
* Did some stuff
* Fix gradle, start implementing mjpeg frame consumer
* Did some stuff
* bade changes
* rename camera config to USBCameraConfiguration, add name
* unrename cameraconfiguration
* Add pub/sub framework
* Add setResolution to mjpeg frame consumer
* add NTDataConsumer
* Add some totally broken hsv hacks
* Start refactoring UI data
* Update index.js
* Commit and push, he says
* Fix up some errors
* Fix input tab
* Fix fps
* Update index.js
* Add pipeline field setting, update PipelineManager, fix nullpointers and USBCameraSettables
* Change v-model to point to data()
* update hsv to use mutations
* Work on saving, fix hsv
* Rename shouldErode/shouldDilate to erode and dilate
* Hook all the tabs up to the Store
* Change handleData to handlePipelineData
* camera quirk redo, add ICCSub to SocketHandler
* Fix some property names
* Fixed tons of naming in UI, fix backend for multi-val PSCs, fix PSC enums
* change pipeline type to an int in store
* Fix mutation naming
* Attempt threshold fix
* Update SocketHandler.java
* Add truthy data sending
* Start adding logging support
* [UI] Add delay to slider input boxes (#1)
* [UI] [Backend] potentially fix camera settings, various logging tweaks
* Don't release raw input mat
* add setVideoModeIndex to vision settables
* Implement pipeline index in socket handler, add framework for renaming/changing pipes
* (ish) get pipeline change working
* Create index.html
* Cleanups, fix pipeline index bug, fix stream res for MJPG, add dashboard stream (unused)
* Refactor UI to use mutatePipeline, send pipeline results
* Update NetworkConfig.java
* Change double to number
* Run spotless
* Fix reversal of large/small comparators
* Fix left/right
* Fix pitch/yaw calculation bug, fix area bug
* Use Vue.set instead of assignment
This fixes {{ }}
* Update App.vue
* run spotless
* Actually add pipelines and reassign indecies
* Delete old pipeline configs
Fixes duplication on renaming pipeline
* Start working on deleting pipes
* Fix camera nickname change
* run spotless
* Fix some test stuff
* Update VisionModuleManagerTest.java
* vision source manager test is still broken
* Fix VisionSourceManager test
* Apply spotless 2 electric boogaloo
Co-authored-by: Banks Troutman <btrout.dhrs@gmail.com>
Co-authored-by: Declan Freeman-Gleason <declanfreemangleason@gmail.com>
Co-authored-by: Aaryan Agrawal <54345060+13Ducks@users.noreply.github.com>
