This adds support for PNP_DISTANCE_TRIG_SOLVE in the the python
PhotonPoseEstimator, mirroring the implementation in the Java
PhotonPoseEstimator.
Changes:
- Add PoseStrategy.PNP_DISTANCE_TRIG_SOLVE
- Add addHeadingData() and resetHeadingData() to PhotonPoseEstimator
- Fix PhotonCameraSim.process() to set ntReceiveTimestampMicros in the
result
- Minor readability improvements to PhotonPipelineResult
- Minor test improvements to PhotonPoseEstimatorTest
- Add .vscode/settings.json (to make running python tests in VSCode
easier)
Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_
- [ ] If this PR changes behavior or adds a feature, user documentation
is updated
- [ ] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [ ] If this PR touches configuration, this is backwards compatible
with settings back to v2024.3.1
- [ ] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [ ] If this PR addresses a bug, a regression test for it is added
---------
Co-authored-by: Sam948-byte <samf.236@proton.me>
## Description
Updates the PhotonPoseEstimator programming documentation page to
reflect the new PhotonPoseEstimator. and add a more comprehensive
step-by-step tutorial using code from the PV Pose Estimation examples.
Updates various code examples to ensure they are matching the latest
documentation or example code on GitHub.
This PR is a successor of, and therefore closes#1765.
This PR is blocked by #1706, as the linked PR updates examples.
Closes#1757 and closes#1800 and closes#1632 and closes#1773 and
closes#1465.
## Meta
Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_
- [x] If this PR changes behavior or adds a feature, user documentation
is updated
- [x] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [x] If this PR touches configuration, this is backwards compatible
with settings back to v2024.3.1
- [x] If this PR touches pipeline settings or anything related to data
exchange, the frontend typing is updated
- [x] If this PR addresses a bug, a regression test for it is added
---------
Co-authored-by: Kevin Reas <76408202+PaarkG@users.noreply.github.com>
Co-authored-by: Matt Morley <matthew.morley.ca@gmail.com>
## Description
We forgot to go to the correct directory prior to running ``npm ci``, so
now I'm adding it.
PROOF: see the passing website test
## Meta
Merge checklist:
- [x] Pull Request title is [short, imperative
summary](https://cbea.ms/git-commit/) of proposed changes
- [x] The description documents the _what_ and _why_
- [x] If this PR changes behavior or adds a feature, user documentation
is updated
- [x] If this PR touches photon-serde, all messages have been
regenerated and hashes have not changed unexpectedly
- [x] If this PR touches configuration, this is backwards compatible
with settings back to v2024.3.1
- [x] If this PR addresses a bug, a regression test for it is added
---------
Co-authored-by: Gold856 <117957790+Gold856@users.noreply.github.com>
Fixes#1564. Also copies vendordep JSONs to the examples as advised by
Thad. Removes unused shared/javacpp/setupBuild.gradle. Also removes
unnecessary `chmod +x gradlew` from CI workflows.
* Serialize all calibration data
* Run lint
* typing nit
* fix code
* move these tables around some
* Add cool formatting
* add request to get snapshots by resolution and camera
* re-enable all resolutions
* add wip so i can change computers (SQUASH ME AND KILL ME AHHHH)
* Get everything working but viewing snapshots
* Update RequestHandler.java
* Update CameraCalibrationInfoCard.vue
* Update CameraCalibrationInfoCard.vue
* add observation viewer
* round
* fix illiegal import
* Swap to PNG and serialize insolution
* move import/export buttons TO THE TOP
* Update WebsocketDataTypes.ts
* Add snapshotname to observation
* Refactor to serialize snapshot image itself
* Run lint
* Use new base64 image data in info card
* Update SettingTypes.ts
* Create calibration json -> mrcal converter script
* Update calibrationUtils.py
* Fix calibrate NPEs in teest
* Run lint
* Always run cornersubpix
* Update CameraCalibrationInfoCard.vue
Update CameraCalibrationInfoCard.vue
* Update OpenCVHelp.java
* Update OpenCVHelp.java
* Replace test mode camera JSONs
* Run wpiformat
* Revert intrinsics but keep other data
* Remove misc comments
* Rename JsonMat->JsonImageMat and add calobject_warp
* Update Server.java
* Rename cameraExtrinsics to distCoeffs
* fix typing issues
* use util methods
* Formatting fixes
* fix styling
* move to devTools
* remove unneeded or unused imports
* Remove fixed-right css
If its really that big of a deal, we can add it back later, kind of a drag to fix rn.
* Create util method
* Remove extra legacy calibration things
---------
Co-authored-by: Sriman Achanta <68172138+srimanachanta@users.noreply.github.com>
### What does this do?
- Deprecates previous sim classes
- Has a `CameraProperties` class for describing a camera's basic/calibration info, and performance values for simulation. Calibration values can be loaded from the `config.json` in the settings exported by photonvision.
- `OpenCVHelp` provides convenience functions for using opencv methods with wpilib/photonvision classes, mainly to project 3d points to a camera's 2d image and perform solvePnP with the above camera calibration info.
- `TargetModel`s describe the 3d shape of a target, both for projecting into the camera's 2d image and use in solvePnP.
- `PhotonCameraSim` uses camera properties to simulate how 3d targets would appear in its view, and has simulated noise, latency, and FPS. For apriltags, the best/alternate camera-to-target transform is also estimated with solvePnP.
- `VideoSimUtil` has helper functions for drawing apriltags to a simulated raw and processed MJPEG stream for each camera using the projected tag corners.
- `VisionSystemSim` stores `VisionTargetSim`s and `PhotonCameraSim`s, and is periodically updated with the robot's simulated pose. When updating, camera sims are automatically processed and published with their visible targets from their respective poses with proper latency.
### What's still not working?
- Mac Arm builds are broken
- More examples
- Update website/docs
Serializes settings using a sqlite database instead of just putting them on the filesystem. Ideally since sqlite deals with filesystem robustness stuff this should work a lot better
Merging this now so we have lots of time to stabilize pre-beta
- Made alternate constructor for ```PhotonPoseEstimator``` that doesn't need ```PhotonCamera```
- Changed ```update``` function to take in missing cameraMatrixData and coeffsData that were previously received from PhotonCamera.
- Changed the internal ```update``` and ```multiTagPNP``` function to take in cameraMatrixData & coeffsData
- If not needed for the specified strategy then the parameters are simply not used. Also if PhotonCamera is used in constructor it instead backs up to that.
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
* Add native stuff
* use runtimeloader
* add more native methods
* more stuff
* Switch JNI methods to static
* Remove non-java classes from the picam jni
* Add gradle task for JNI generation
* Migrate my previous GPU accel work
* Initial work on defining JNI interface
* Change libpicam to a symlink for now
* Initial work on adding no-copy OMX GPU accel on the pi
* Make DIRECT_OMX GPU accel mode not crash
* Clean up OMX changes (still not getting valid data back)
* Re-add GPU unit test
* A couple debugging tweaks/notes
* Add temporary special cases to get RGB out of ProcessingMode.NONE
* Code clarity improvements; fix possible VBO bug
* Get DIRECT_OMX working
* Remove some debugging switches in GPUAccelerator
* Pipe in VCSM stuff to read out pixels FAST
* Apply Spotless
* Revert versioningHelper changes
* Add missing import
* Convert to MMAL and move everything to native
* Re-add shared object
* Rework to use MMAL and do everything natively
* Condense pipeline settings classes
* Add OutputStreamPipeline
* Apply spotless
* Fix duplicate variable inits and add more video modes
* Integrate color frames and latency measurements for GPU
* Fix camera detection on pi and other platforms
* Add proper color copy disabling and camera settings calls
* Fix things that were broken by rebase
* Fix spotless issues and remove uneeded prints
* Remove libpicam symlink
* Fix stream resolution limiting
* Remove testing code in GPUAcceleratedHSVPipe
* Make profiling options general to all computers
* Make PicamJNI load from resources
* run spotlessApply
* Address review comments
* Update Maven repo for JOGL
* Fix release race condition
* Only run GPU accel test on the pi
* Lint fix and merge conflict accident fixes
* Make Jackson ignore extra fields when unmarshalling HardwareConfig
* Fix Mat releasing data race
* Spotless apply
* Remove broken header generation task
* Fix shared library loading typo
* Add a ZeroCopyPicam quirk to allow setting gain with the MMAL backend
* Make sure that exposure/brightness/gain get set after res changes
* Make rawInputMat properly local
* Remove bogus set of shouldRun flag
* Clean up small GPUHSVPipe print
* Add in some things that missed the ZeroCopyPiCameraSource rename
* Fix incorrect scoping introduced in past rebase
* Don't filter out too-low resolutions
* Only show latency when GPU accel is enabled
* Don't free Mats in stream thread before we use them
* Fix use-after-free and latency caluclation bugs on USB camera source
* Update libpicam
* Remove unwanted print
* Add libpicam forceLoad in unit test
* Fix streaming during camera calibration
* Fix zerocopy Picam calculation
* Use logger trace method instead of raw prints
* Fix calibration and driver mode pipes with the Picam
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
Co-authored-by: Banks Troutman <btrout.dhrs@gmail.com>
* Did some stuff
* Fix gradle, start implementing mjpeg frame consumer
* Did some stuff
* bade changes
* rename camera config to USBCameraConfiguration, add name
* unrename cameraconfiguration
* Add pub/sub framework
* Add setResolution to mjpeg frame consumer
* add NTDataConsumer
* Add some totally broken hsv hacks
* Start refactoring UI data
* Update index.js
* Commit and push, he says
* Fix up some errors
* Fix input tab
* Fix fps
* Update index.js
* Add pipeline field setting, update PipelineManager, fix nullpointers and USBCameraSettables
* Change v-model to point to data()
* update hsv to use mutations
* Work on saving, fix hsv
* Rename shouldErode/shouldDilate to erode and dilate
* Hook all the tabs up to the Store
* Change handleData to handlePipelineData
* camera quirk redo, add ICCSub to SocketHandler
* Fix some property names
* Fixed tons of naming in UI, fix backend for multi-val PSCs, fix PSC enums
* change pipeline type to an int in store
* Fix mutation naming
* Attempt threshold fix
* Update SocketHandler.java
* Add truthy data sending
* Start adding logging support
* [UI] Add delay to slider input boxes (#1)
* [UI] [Backend] potentially fix camera settings, various logging tweaks
* Don't release raw input mat
* add setVideoModeIndex to vision settables
* Implement pipeline index in socket handler, add framework for renaming/changing pipes
* (ish) get pipeline change working
* Create index.html
* Cleanups, fix pipeline index bug, fix stream res for MJPG, add dashboard stream (unused)
* Refactor UI to use mutatePipeline, send pipeline results
* Update NetworkConfig.java
* Change double to number
* Run spotless
* Fix reversal of large/small comparators
* Fix left/right
* Fix pitch/yaw calculation bug, fix area bug
* Use Vue.set instead of assignment
This fixes {{ }}
* Update App.vue
* run spotless
* Actually add pipelines and reassign indecies
* Delete old pipeline configs
Fixes duplication on renaming pipeline
* Start working on deleting pipes
* Fix camera nickname change
* run spotless
* Fix some test stuff
* Update VisionModuleManagerTest.java
* vision source manager test is still broken
* Fix VisionSourceManager test
* Apply spotless 2 electric boogaloo
Co-authored-by: Banks Troutman <btrout.dhrs@gmail.com>
Co-authored-by: Declan Freeman-Gleason <declanfreemangleason@gmail.com>
Co-authored-by: Aaryan Agrawal <54345060+13Ducks@users.noreply.github.com>
* Add some config stuff, run format
* Create JacksonUtils.java
* Fix deserialization, run wpiformat
* Update ConfigTest.java
* Change config root folder, auto-sort pipelines when added to CameraConfiguration
* Split ConfigTest tests in to multiple tests
* Fix unsatisfied link error in config test
* Add equals and hashCode to all PipelineSettings classes
* Fix NPE in config test
* run spotless
* Address changes
Co-authored-by: Banks Troutman <btrout.dhrs@gmail.com>
Co-authored-by: ori agranat <oriagranat9@gmail.com>
* Refactor package structure, various cleanups
* Add pipeline classes, settings, separate enums
* updated Largest ContourSortMode and added centermost
* Add DriverPipeline classes, apply spotless
* Add crosshair to DriverMode, cleanups
* Add FrameStaticProperties as member in Frame
Add FrameStaticProperties as member in Frame
* Finish ReflectivePipeline, various tweaks
* Apply Spotless
* Move test images
* add Releasable interface, implement in classes
* add TestUtils class, move testimages
* Refactor CVPipeline, add ReflectivePipelineTest
* Fix ConcurrentModificationException bug in group contours pipe with potential targets
* Resolve memory leaks due to unnecessary instantiation of Points
* Apply spotless
* Add CVMat, ReflectionUtils to help track rogue Mats
* various cleanups, add DummyFrameConsumer
* Add logback
* Add slv4j logger to replace the current debugLogger
I'm waiting on stuff to be less skeletoned to add more
* Add perimeter, MatOfPoint2f getters to Contour
* Create CornerDetectionPipe based on old solvePNPPipe
* Add ContourShape class for approxPolyDp Start on ColoredShape tracking
* Add point detection, fix convex hull calculation in Contour
* Make Draw2dContours pipe respect showMultiple
* Update Contour.java
* Clean up draw 3d, fix convex hull bug in corner detection
* Update geometry classes
* Add lifecam calibration data
* Implement solvePNP, bounding box top and bottom
* Fix JSON mat bug and lifecam default calibration for tests, fix 3d drawing
* run spotless
* Refactor calibration into `common.calibration`
* Update .gitignore
* Add offset method to get2020Target
* Various cleanups, add PipelineType enum
* Apply spotless
Co-authored-by: ori agranat <oriagranat9@gmail.com>
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
* Toggle mode
* RangeSlider able to be disabled
* Added another window from the press of the button
* removed warning
* Major changes to colorpicker page
* Major changes to threshold tab
* fix for BGR not RGB
* Reverted changes of colorpicker.vue
* Updated Readme.md
Mainly Development Setup
* Update readme
small changes
* updated readme.md
Authors credit update
* updated readme.md
added usb camera to hardware
* Updated UI readme
added nodejs installation and explanation
* Unit test vision (and implement networktable sending)
* finished color picker in the ui
* bugfix for correct tab
* cleanup and commenting
* Reveted changes to range slider
* bug fix for color picker
* Pipeline mat copy fix
* Added crosshair pipe
* compilation fix
* removed unsued comment
* moved calibration stuff to static class under 2d pipeline settings
* Added debug print commented
* Reverted subclass "Calibration"
Fixed bug where single point calibration causes crash when no target is found
* disable dual point crosshair for now
* Proper no target found fix, and snackbar error now showing
* Removed java fix for crash bug
* fixed problem with clear points in single point calibration
* change compatator to sqared values to make caluclation faster (no need for sqare root function)
* fixed roated camera too heigh for the UI
* Revert "fixed roated camera too heigh for the UI"
This reverts commit 50c8ecd345.
* fixed rotated camera too heigh for UI and corrected scalling
* updated pom to fix the camera buffer issue
* added pom variables
* Created new CaptureStaticProperties when rotating cam
* File stucture change, moved Pipeline implementation and their settings to /impl folder, refracted imports
* ignore auto created iml file
* ignore auto created iml file
* Fix lack of access problems after moving pipleline impl out of the same package
* Removed commented lines
* Fixed bug when selecting the same 90 deg rotation twice causes crosshair to move
* cleaned up single point calibration and bug fixed isBinary in driverMode
* Enum fix for the UI
* typo
* fixed Stream divisor problems see issue 19 in github issues
* fixed compilation error for test class
* Fixed problem when rotating camera with stream divisor
* Spelling + button style change
* Add speed limit to UI Updates
* Added runtime arg to fix settings permissions, and on-first-init
* Added FileHelper class to manage setting permissions for all files.
* Removed unnecessary argument
* NT can set pipeline to out on bound indexes
* bugfix in platform check
* added pipeline popup
* Round steam divisor resolution
* added regex test for camera name and pipe name
* Fix for Renaming pipeline leaves old config file #14
* Version change
* adde ( ) - and . to regex
* Replace NT timestamp with latency
* added back NT flush
* Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35)
* Rebase solvePNP on master
* added 3D tab minimap and csv reader
* More solvePNP
* Create draw pipe for pnp data
* SolvePNP piping work
* Move sorting into solvepnppipe
* Create calibration pipeline
* Update CalibrateSolvePNPPipeline.java
* add camera tilt angle
* Add calibration slider and snapshot button to 3D view
* Mirror updates in the socket handler
* add 3d calibration mode to the pipeline manager
* created calibration functions in ui and backend
* Start plumbing calibration
* Add snapshot and other handling to the RequestHandler
* added select resolution before starting calibration
* Rename solvePNPPipe to bounding box solve pnp pipe
* Update BoundingBoxSolvePNPPipe.java
* Add Mat serializer and CameraCalibrationConfig
* Begun calibration saving, fixed UI/Backend snapshot count mismatch
* Add (unplumbed) option to set checkerboard size
This will allow users to change the units their calibration is in
* Create chessboard.png
* Fix calibration NPE
* changed string serialization to a json send
* bug fixed cancellation button
* Fix spelling of snapshot in 3d.vue
* Plumb resolution change
* Set resolution during config, start on config serialization
* Update .gitignore
* Config fixes
* Start transition away from cvpipeline3d
* fix NPE on uncalibrated cameras
* clear list on fail
* Fix video mode index error
* ignore getters in camera calibration config
* Create json constructor for jsonmat
* get solvePNP mostly returning sane values
* Fix solvePNP bug and add unit test
* FIx calibration mat truncation
* added capture amount model upload and minimap data
* Standardize on meters in calibration and bounding box
* fix json out of bounds and handle null calibration more gracefully
* don't put text on calibrate image, go back to inches
* convert distance to meters
this means calibration will need to be in inches
* Actually save raw contor
* Update GroupContoursPipe.java
* Add all calibration return to camera capture
* hard code 2019 target
* bugfixed draw2d added fail calib popup, merge end and cancel
added the res index to the calib start
* Clarify error message and draw more fancy rectangles
* Cleanup memory in solvepnp
* re did minimap component
* fix npe if left/right is null
* remove references to 2d
* try-catch running the current pipeline
* Add method to find corners using the harris corner detector
* Possibly fix left/right missmatch
* Fix 3D Tab error
* FIx file permissions, mat serializer adjustments
* fixed mini map for field coordinates
* mini map changes fov
* Update SolvePNPPipe.java
* get rid of target corners
* some memory leak fixes
* fixed mini map location
* added position under minimap
* changed player fov look
* put all targets in the web send
* re did target send to ui added target tables, bugfix calibration
* fixed y position
* Add tilt angle to capture properties
* maybe fix y axis in minimap
* Add square size to onCalibrationEnding
* Possibly add square size to UI
* fix NPE with pitch
* Fix bug with sending multiple targets
* Only instantiate 3d stuff if we are in 3d mode
* Fix array list exceptions
* Fix bug in sort contors
list was truncated too early
* added download chess, tilt setting and ordinal tilt,
* added square size connection
* removed unused code
* Update pom version to 2.1-RELEASE
* Send camera calibrations to UI
* Stream pose list to a LIst
* Only stream necessary parts of the aux list entry
* Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions
* added fps counter changed squaresize steps bug fixes in tables
* bugfix camera settings cam wont change
Authored-by: oriagranat9 <oriagranat9@gmail.com>
* disable pose print
* standardize on calibration in inches and add square size as a divisor
This should do the same thing as calibrating in the correct units but it doesnt like meeeeee
* updated pom for release, 3d calibration change, cosmetic changes
* WIP: 3D res filter
* bug fix in index parsing
* add good features to track point finder
* offset found corners by old tl position
* Add method to find most extreme per-quadrant points for solvePNP
* re did pipeline duplication
* Add release method to standard cv pipeline
* remove contor from tracked target
* Explicitly release all the intermediate results
* avoid creating new mats in group contors pipe
* removed mat release
* updated pom
* Actually cache the last target
We were trying to but i never hooked this part up
* Fix memory leak in sort contros pipe
* Fix memory leak in sort contors pipe
* Help more with memory usage in speckle reject and draw pipes
* minor bugfixes to the ui
* Start moving stream into vision process
Should probs move into a pipe to be idimatic
* justify bug fix
* Fix sort left/right bug
* Fix target grouping bug
* Avoid allocating a new mat in solvePNP, perf. increases in group contors pipe
* Event scripts (#36)
* Begin scripting work
* More scripting work
* Finalize scripting system
* Begin implementing script events
* Finalize script system
Co-authored-by: Banks T <btrout.dhrs@gmail.com>
Co-authored-by: OmerZ7 <zipory.omer@gmail.com>
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
Co-authored-by: Banks T <btrout.dhrs@gmail.com>
- added port hash map
- putting port into hashmap from camera class
- added sending port from full settings and camera change
- nt pipe change now sends pipe name to ui
