Team2890/PhotonVision
4
0
Fork 0
mirror of https://github.com/PhotonVision/photonvision synced 2026-06-23 01:21:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

V2.1 (#37)

Browse Source 
* Toggle mode

* RangeSlider able to be disabled

* Added another window from the press of the button

* removed warning

* Major changes to colorpicker page

* Major changes to threshold tab

* fix for BGR not RGB

* Reverted changes of colorpicker.vue

* Updated Readme.md

Mainly Development Setup

* Update readme

small changes

* updated readme.md

Authors credit update

* updated readme.md 

added usb camera to hardware

* Updated UI readme

added nodejs installation and explanation

* Unit test vision (and implement networktable sending)

* finished color picker in the ui

* bugfix for correct tab

* cleanup and commenting

* Reveted changes to range slider

* bug fix for color picker

* Pipeline mat copy fix

* Added crosshair pipe

* compilation fix

* removed unsued comment

* moved calibration stuff to static class under 2d pipeline settings

* Added debug print commented

* Reverted subclass "Calibration"
Fixed bug where single point calibration causes crash when no target is found

* disable dual point crosshair for now

* Proper no target found fix, and snackbar error now showing

* Removed java fix for crash bug

* fixed problem with clear points in single point calibration

* change compatator to sqared values to make caluclation faster (no need for sqare root function)

* fixed roated camera too heigh for the UI

* Revert "fixed roated camera too heigh for the UI"

This reverts commit 50c8ecd345.

* fixed rotated camera too heigh for UI and corrected scalling

* updated pom to fix the camera buffer issue

* added pom variables

* Created new CaptureStaticProperties when rotating cam

* File stucture change, moved Pipeline implementation and their settings to /impl folder, refracted imports

* ignore auto created iml file

* ignore auto created iml file

* Fix lack of access problems after moving pipleline impl out of the same package

* Removed commented lines

* Fixed bug when selecting the same 90 deg rotation twice causes crosshair to move

* cleaned up single point calibration and bug fixed isBinary in driverMode

* Enum fix for the UI

* typo

* fixed Stream divisor problems see issue 19 in github issues

* fixed compilation error for test class

* Fixed problem when rotating camera with stream divisor

* Spelling + button style change

* Add speed limit to UI Updates

* Added runtime arg to fix settings permissions, and on-first-init

* Added FileHelper class to manage setting permissions for all files.

* Removed unnecessary argument

* NT can set pipeline to out on bound indexes

* bugfix in platform check

* added pipeline popup

* Round steam divisor resolution

* added regex test for camera name and pipe name

* Fix for Renaming pipeline leaves old config file #14

* Version change

* adde ( ) - and . to regex

* Replace NT timestamp with latency

* added back NT flush

* Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35)

* Rebase solvePNP on master

* added 3D tab minimap and csv reader

* More solvePNP

* Create draw pipe for pnp data

* SolvePNP piping work

* Move sorting into solvepnppipe

* Create calibration pipeline

* Update CalibrateSolvePNPPipeline.java

* add camera tilt angle

* Add calibration slider and snapshot button to 3D view

* Mirror updates in the socket handler

* add 3d calibration mode to the pipeline manager

* created calibration functions in ui and backend

* Start plumbing calibration

* Add snapshot and other handling to the RequestHandler

* added select resolution before starting calibration

* Rename solvePNPPipe to bounding box solve pnp pipe

* Update BoundingBoxSolvePNPPipe.java

* Add Mat serializer and CameraCalibrationConfig

* Begun calibration saving, fixed UI/Backend snapshot count mismatch

* Add (unplumbed) option to set checkerboard size

This will allow users to change the units their calibration is in

* Create chessboard.png

* Fix calibration NPE

* changed string serialization to a json send

* bug fixed cancellation button

* Fix spelling of snapshot in 3d.vue

* Plumb resolution change

* Set resolution during config, start on config serialization

* Update .gitignore

* Config fixes

* Start transition away from cvpipeline3d

* fix NPE on uncalibrated cameras

* clear list on fail

* Fix video mode index error

* ignore getters in camera calibration config

* Create json constructor for jsonmat

* get solvePNP mostly returning sane values

* Fix solvePNP bug and add unit test

* FIx calibration mat truncation

* added capture amount model upload and minimap data

* Standardize on meters in calibration and bounding box

* fix json out of bounds and handle null calibration more gracefully

* don't put text on calibrate image, go back to inches

* convert distance to meters

this means calibration will need to be in inches

* Actually save raw contor

* Update GroupContoursPipe.java

* Add all calibration return to camera capture

* hard code 2019 target

* bugfixed draw2d added fail calib popup, merge end and cancel

added the res index to the calib start

* Clarify error message and draw more fancy rectangles

* Cleanup memory in solvepnp

* re did minimap component

* fix npe if left/right is null

* remove references to 2d

* try-catch running the current pipeline

* Add method to find corners using the harris corner detector

* Possibly fix left/right missmatch

* Fix 3D Tab error

* FIx file permissions, mat serializer adjustments

* fixed mini map for field coordinates

* mini map changes fov

* Update SolvePNPPipe.java

* get rid of target corners

* some memory leak fixes

* fixed mini map location

* added position under minimap

* changed player fov look

* put all targets in the web send

* re did target send to ui added target tables, bugfix calibration

* fixed y position

* Add tilt angle to capture properties

* maybe fix y axis in minimap

* Add square size to onCalibrationEnding

* Possibly add square size to UI

* fix NPE with pitch

* Fix bug with sending multiple targets

* Only instantiate 3d stuff if we are in 3d mode

* Fix array list exceptions

* Fix bug in sort contors

list was truncated too early

* added download chess, tilt setting and ordinal tilt,

* added square size connection

* removed unused code

* Update pom version to 2.1-RELEASE

* Send camera calibrations to UI

* Stream pose list to a LIst

* Only stream necessary parts of the aux list entry

* Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions

* added fps counter changed squaresize steps bug fixes in tables

* bugfix camera settings cam wont change

Authored-by: oriagranat9 <oriagranat9@gmail.com>

* disable pose print

* standardize on calibration in inches and add square size as a divisor

This should do the same thing as calibrating in the correct units but it doesnt like meeeeee

* updated pom for release, 3d calibration change, cosmetic changes

* WIP: 3D res filter

* bug fix in index parsing

* add good features to track point finder

* offset found corners by old tl position

* Add method to find most extreme per-quadrant points for solvePNP

* re did pipeline duplication

* Add release method to standard cv pipeline

* remove contor from tracked target

* Explicitly release all the intermediate results

* avoid creating new mats in group contors pipe

* removed mat release

* updated pom

* Actually cache the last target

We were trying to but i never hooked this part up

* Fix memory leak in sort contros pipe

* Fix memory leak in sort contors pipe

* Help more with memory usage in speckle reject and draw pipes

* minor bugfixes to the ui

* Start moving stream into vision process

Should probs move into a pipe to be idimatic

* justify bug fix

* Fix sort left/right bug

* Fix target grouping bug

* Avoid allocating a new mat in solvePNP, perf. increases in group contors pipe

* Event scripts (#36)

* Begin scripting work

* More scripting work

* Finalize scripting system

* Begin implementing script events

* Finalize script system

Co-authored-by: Banks T <btrout.dhrs@gmail.com>

Co-authored-by: OmerZ7 <zipory.omer@gmail.com>
Co-authored-by: Matt <matthew.morley.ca@gmail.com>
Co-authored-by: Banks T <btrout.dhrs@gmail.com>
This commit is contained in:
oriagranat9
2020-01-04 11:53:18 -08:00
committed by GitHub
parent 8a5052e0b3
commit 97a6e1419d
96 changed files with 4251 additions and 677 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
chameleon-server/src/main/java/com/chameleonvision/web/SocketHandler.java
View File

@@ -1,16 +1,21 @@
package com.chameleonvision.web;
import com.chameleonvision.config.CameraCalibrationConfig;
import com.chameleonvision.config.ConfigManager;
import com.chameleonvision.vision.VisionManager;
import com.chameleonvision.vision.VisionProcess;
import com.chameleonvision.vision.camera.CameraCapture;
import com.chameleonvision.vision.camera.CaptureStaticProperties;
import com.chameleonvision.vision.camera.USBCameraCapture;
import com.chameleonvision.vision.enums.ImageRotationMode;
import com.chameleonvision.vision.enums.StreamDivisor;
import com.chameleonvision.vision.pipeline.CVPipeline;
import com.chameleonvision.vision.pipeline.impl.StandardCVPipeline;
import com.chameleonvision.vision.pipeline.CVPipelineSettings;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import edu.wpi.cscore.VideoMode;
import io.javalin.websocket.WsBinaryMessageContext;
import io.javalin.websocket.WsCloseContext;
import io.javalin.websocket.WsConnectContext;
@@ -20,10 +25,9 @@ import org.msgpack.jackson.dataformat.MessagePackFactory;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.*;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
public class SocketHandler {
@@ -31,6 +35,8 @@ public class SocketHandler {
private static List<WsContext> users;
private static ObjectMapper objectMapper;
private static final Object broadcastLock = new Object();
SocketHandler() {
users = new ArrayList<>();
objectMapper = new ObjectMapper(new MessagePackFactory());
@@ -54,7 +60,7 @@ public class SocketHandler {
VisionProcess currentProcess = VisionManager.getCurrentUIVisionProcess();
CameraCapture currentCamera = currentProcess.getCamera();
CVPipeline currentPipeline = currentProcess.pipelineManager.getCurrentPipeline();
// System.out.println("entry.getKey()+entry.getValue()= " + entry.getKey() + entry.getValue());
switch (entry.getKey()) {
case "driverMode": {
HashMap<String, Object> data = (HashMap<String, Object>) entry.getValue();
@@ -77,39 +83,17 @@ public class SocketHandler {
VisionManager.saveCurrentCameraPipelines();
break;
}
case "duplicatePipeline": {
HashMap pipelineVals = (HashMap) entry.getValue();
int pipelineIndex = (int) pipelineVals.get("pipeline");
int cameraIndex = (int) pipelineVals.get("camera");
ObjectMapper mapper = new ObjectMapper();
CVPipelineSettings origPipeline = currentProcess.pipelineManager.getPipeline(pipelineIndex).settings;
String val = mapper.writeValueAsString(origPipeline);
CVPipelineSettings newPipeline = mapper.readValue(val, origPipeline.getClass());
if (cameraIndex != -1) {
VisionProcess newProcess = VisionManager.getVisionProcessByIndex(cameraIndex);
if (newProcess != null) {
currentProcess.pipelineManager.duplicatePipeline(newPipeline, newProcess);
} else {
System.err.println("Failed to get destination camera for pipeline duplication!");
}
} else {
currentProcess.pipelineManager.duplicatePipeline(newPipeline);
}
VisionManager.saveCurrentCameraPipelines();
case "addNewPipeline": {
// HashMap<String, Object> data = (HashMap<String, Object>) entry.getValue();
String pipeName = (String) entry.getValue();
// TODO: add to UI selection for new 2d/3d
currentProcess.pipelineManager.addNewPipeline(pipeName);
sendFullSettings();
VisionManager.saveCurrentCameraPipelines();
break;
}
case "command": {
switch ((String) entry.getValue()) {
case "addNewPipeline":
// TODO: add to UI selection for new 2d/3d
boolean is3d = false;
currentProcess.pipelineManager.addNewPipeline(is3d);
sendFullSettings();
VisionManager.saveCurrentCameraPipelines();
break;
case "deleteCurrentPipeline":
currentProcess.pipelineManager.deleteCurrentPipeline();
sendFullSettings();
@@ -129,20 +113,54 @@ public class SocketHandler {
sendFullSettings();
break;
}
case "is3D": {
VisionManager.getCurrentUIVisionProcess().setIs3d((Boolean) entry.getValue());
break;
}
case "currentPipeline": {
currentProcess.pipelineManager.setCurrentPipeline((Integer) entry.getValue());
sendFullSettings();
break;
}
case "isPNPCalibration": {
currentProcess.pipelineManager.setCalibrationMode((Boolean) entry.getValue());
break;
}
case "takeCalibrationSnapshot": {
currentProcess.pipelineManager.calib3dPipe.takeSnapshot();
}
default: {
// only change settings when we aren't in driver mode
if(currentProcess.pipelineManager.getDriverMode()) {
switch (entry.getKey()) {//Pre field value set
case "rotationMode": {//Create new CaptureStaticProperties with new width and height, reset crosshair calib
ImageRotationMode oldRot = currentPipeline.settings.rotationMode;
ImageRotationMode newRot = ImageRotationMode.class.getEnumConstants()[(Integer) entry.getValue()];
CaptureStaticProperties prop = currentCamera.getProperties().getStaticProperties();
int width, height;
if (oldRot.isRotated() != newRot.isRotated()) {
width = prop.mode.height;
height = prop.mode.width;
//Creates new video mode with new width and height to create new CaptureStaticProperties and applies it
currentCamera.getProperties().setStaticProperties(new CaptureStaticProperties(
new VideoMode(prop.mode.pixelFormat, width, height, prop.mode.fps), prop.fov));
}
prop = currentCamera.getProperties().getStaticProperties();
currentProcess.cameraStreamer.recalculateDivision();
if (currentPipeline instanceof StandardCVPipeline)
((StandardCVPipeline) currentPipeline).settings.point = Arrays.asList(prop.mode.width / 2, prop.mode.height / 2);//Reset Crosshair in single point calib
break;
}
}
if (currentProcess.pipelineManager.getDriverMode()) {
setField(currentProcess.pipelineManager.driverModePipeline.settings, entry.getKey(), entry.getValue());
} else {
setField(currentPipeline.settings, entry.getKey(), entry.getValue());
}
//Post field value set
switch (entry.getKey()) {
case "exposure": {
currentCamera.setExposure((Integer) entry.getValue());
@@ -152,11 +170,14 @@ public class SocketHandler {
currentCamera.setBrightness((Integer) entry.getValue());
break;
}
case "videoModeIndex":{
case "videoModeIndex": {
if (currentPipeline instanceof StandardCVPipeline)
((StandardCVPipeline) currentPipeline).settings.point = new ArrayList<>();//This will reset the calibration
currentCamera.setVideoMode((Integer) entry.getValue());
currentProcess.cameraStreamer.recalculateDivision();
break;
}
case "streamDivisor":{
case "streamDivisor": {
currentProcess.cameraStreamer.setDivisor(StreamDivisor.values()[(Integer) entry.getValue()], true);
break;
}
@@ -186,18 +207,22 @@ public class SocketHandler {
}
private static void broadcastMessage(Object obj, WsContext userToSkip) {
if (users != null)
for (WsContext user : users) {
if (userToSkip != null && user.getSessionId().equals(userToSkip.getSessionId())) {
continue;
}
try {
ByteBuffer b = ByteBuffer.wrap(objectMapper.writeValueAsBytes(obj));
user.send(b);
} catch (JsonProcessingException e) {
e.printStackTrace();
synchronized (broadcastLock) {
if (users != null) {
var userList = users;
for (WsContext user : userList) {
if (userToSkip != null && user.getSessionId().equals(userToSkip.getSessionId())) {
continue;
}
try {
ByteBuffer b = ByteBuffer.wrap(objectMapper.writeValueAsBytes(obj));
user.send(b);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
}
}
}
}
public static void broadcastMessage(Object obj) {
@@ -236,9 +261,16 @@ public class SocketHandler {
HashMap<String, Object> tmp = new HashMap<>();
VisionProcess currentVisionProcess = VisionManager.getCurrentUIVisionProcess();
USBCameraCapture currentCamera = VisionManager.getCurrentUIVisionProcess().getCamera();
tmp.put("fov", currentCamera.getProperties().getFOV());
tmp.put("streamDivisor", currentVisionProcess.cameraStreamer.getDivisor().ordinal());
tmp.put("resolution", currentVisionProcess.getCamera().getProperties().getCurrentVideoModeIndex());
tmp.put("tilt", currentVisionProcess.getCamera().getProperties().getTilt().getDegrees());
List<CameraCalibrationConfig.UICameraCalibrationConfig> calibrations = currentCamera.getAllCalibrationData().stream()
.map(CameraCalibrationConfig.UICameraCalibrationConfig::new).collect(Collectors.toList());
tmp.put("calibration", calibrations);
return tmp;
}