Files
PhotonVision/chameleon-server/src/main/java/com/chameleonvision/vision/camera/USBCameraCapture.java

132 lines
4.0 KiB
Java
Raw Normal View History

package com.chameleonvision.vision.camera;
Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35) * Rebase solvePNP on master * added 3D tab minimap and csv reader * More solvePNP * Create draw pipe for pnp data * SolvePNP piping work * Move sorting into solvepnppipe * Create calibration pipeline * Update CalibrateSolvePNPPipeline.java * add camera tilt angle * Add calibration slider and snapshot button to 3D view * Mirror updates in the socket handler * add 3d calibration mode to the pipeline manager * created calibration functions in ui and backend * Start plumbing calibration * Add snapshot and other handling to the RequestHandler * added select resolution before starting calibration * Rename solvePNPPipe to bounding box solve pnp pipe * Update BoundingBoxSolvePNPPipe.java * Add Mat serializer and CameraCalibrationConfig * Begun calibration saving, fixed UI/Backend snapshot count mismatch * Add (unplumbed) option to set checkerboard size This will allow users to change the units their calibration is in * Create chessboard.png * Fix calibration NPE * changed string serialization to a json send * bug fixed cancellation button * Fix spelling of snapshot in 3d.vue * Plumb resolution change * Set resolution during config, start on config serialization * Update .gitignore * Config fixes * Start transition away from cvpipeline3d * fix NPE on uncalibrated cameras * clear list on fail * Fix video mode index error * ignore getters in camera calibration config * Create json constructor for jsonmat * get solvePNP mostly returning sane values * Fix solvePNP bug and add unit test * FIx calibration mat truncation * added capture amount model upload and minimap data * Standardize on meters in calibration and bounding box * fix json out of bounds and handle null calibration more gracefully * don't put text on calibrate image, go back to inches * convert distance to meters this means calibration will need to be in inches * Actually save raw contor * Update GroupContoursPipe.java * Add all calibration return to camera capture * hard code 2019 target * bugfixed draw2d added fail calib popup, merge end and cancel added the res index to the calib start * Clarify error message and draw more fancy rectangles * Cleanup memory in solvepnp * re did minimap component * fix npe if left/right is null * remove references to 2d * try-catch running the current pipeline * Add method to find corners using the harris corner detector * Possibly fix left/right missmatch * Fix 3D Tab error * FIx file permissions, mat serializer adjustments * fixed mini map for field coordinates * mini map changes fov * Update SolvePNPPipe.java * get rid of target corners * some memory leak fixes * fixed mini map location * added position under minimap * changed player fov look * put all targets in the web send * re did target send to ui added target tables, bugfix calibration * fixed y position * Add tilt angle to capture properties * maybe fix y axis in minimap * Add square size to onCalibrationEnding * Possibly add square size to UI * fix NPE with pitch * Fix bug with sending multiple targets * Only instantiate 3d stuff if we are in 3d mode * Fix array list exceptions * Fix bug in sort contors list was truncated too early * added download chess, tilt setting and ordinal tilt, * added square size connection * removed unused code * Update pom version to 2.1-RELEASE * Send camera calibrations to UI * Stream pose list to a LIst * Only stream necessary parts of the aux list entry * Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions * added fps counter changed squaresize steps bug fixes in tables * bugfix camera settings cam wont change Authored-by: oriagranat9 <oriagranat9@gmail.com>
2019-12-31 04:53:20 -08:00
import com.chameleonvision.config.CameraCalibrationConfig;
import com.chameleonvision.config.FullCameraConfiguration;
import edu.wpi.cscore.CvSink;
import edu.wpi.cscore.UsbCamera;
import edu.wpi.cscore.VideoException;
import edu.wpi.cscore.VideoMode;
import edu.wpi.first.cameraserver.CameraServer;
import org.apache.commons.lang3.tuple.Pair;
import org.opencv.core.Mat;
Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35) * Rebase solvePNP on master * added 3D tab minimap and csv reader * More solvePNP * Create draw pipe for pnp data * SolvePNP piping work * Move sorting into solvepnppipe * Create calibration pipeline * Update CalibrateSolvePNPPipeline.java * add camera tilt angle * Add calibration slider and snapshot button to 3D view * Mirror updates in the socket handler * add 3d calibration mode to the pipeline manager * created calibration functions in ui and backend * Start plumbing calibration * Add snapshot and other handling to the RequestHandler * added select resolution before starting calibration * Rename solvePNPPipe to bounding box solve pnp pipe * Update BoundingBoxSolvePNPPipe.java * Add Mat serializer and CameraCalibrationConfig * Begun calibration saving, fixed UI/Backend snapshot count mismatch * Add (unplumbed) option to set checkerboard size This will allow users to change the units their calibration is in * Create chessboard.png * Fix calibration NPE * changed string serialization to a json send * bug fixed cancellation button * Fix spelling of snapshot in 3d.vue * Plumb resolution change * Set resolution during config, start on config serialization * Update .gitignore * Config fixes * Start transition away from cvpipeline3d * fix NPE on uncalibrated cameras * clear list on fail * Fix video mode index error * ignore getters in camera calibration config * Create json constructor for jsonmat * get solvePNP mostly returning sane values * Fix solvePNP bug and add unit test * FIx calibration mat truncation * added capture amount model upload and minimap data * Standardize on meters in calibration and bounding box * fix json out of bounds and handle null calibration more gracefully * don't put text on calibrate image, go back to inches * convert distance to meters this means calibration will need to be in inches * Actually save raw contor * Update GroupContoursPipe.java * Add all calibration return to camera capture * hard code 2019 target * bugfixed draw2d added fail calib popup, merge end and cancel added the res index to the calib start * Clarify error message and draw more fancy rectangles * Cleanup memory in solvepnp * re did minimap component * fix npe if left/right is null * remove references to 2d * try-catch running the current pipeline * Add method to find corners using the harris corner detector * Possibly fix left/right missmatch * Fix 3D Tab error * FIx file permissions, mat serializer adjustments * fixed mini map for field coordinates * mini map changes fov * Update SolvePNPPipe.java * get rid of target corners * some memory leak fixes * fixed mini map location * added position under minimap * changed player fov look * put all targets in the web send * re did target send to ui added target tables, bugfix calibration * fixed y position * Add tilt angle to capture properties * maybe fix y axis in minimap * Add square size to onCalibrationEnding * Possibly add square size to UI * fix NPE with pitch * Fix bug with sending multiple targets * Only instantiate 3d stuff if we are in 3d mode * Fix array list exceptions * Fix bug in sort contors list was truncated too early * added download chess, tilt setting and ordinal tilt, * added square size connection * removed unused code * Update pom version to 2.1-RELEASE * Send camera calibrations to UI * Stream pose list to a LIst * Only stream necessary parts of the aux list entry * Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions * added fps counter changed squaresize steps bug fixes in tables * bugfix camera settings cam wont change Authored-by: oriagranat9 <oriagranat9@gmail.com>
2019-12-31 04:53:20 -08:00
import org.opencv.core.Size;
import org.opencv.imgcodecs.Imgcodecs;
import java.util.ArrayList;
import java.util.List;
public class USBCameraCapture implements CameraCapture {
private final UsbCamera baseCamera;
private final CvSink cvSink;
Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35) * Rebase solvePNP on master * added 3D tab minimap and csv reader * More solvePNP * Create draw pipe for pnp data * SolvePNP piping work * Move sorting into solvepnppipe * Create calibration pipeline * Update CalibrateSolvePNPPipeline.java * add camera tilt angle * Add calibration slider and snapshot button to 3D view * Mirror updates in the socket handler * add 3d calibration mode to the pipeline manager * created calibration functions in ui and backend * Start plumbing calibration * Add snapshot and other handling to the RequestHandler * added select resolution before starting calibration * Rename solvePNPPipe to bounding box solve pnp pipe * Update BoundingBoxSolvePNPPipe.java * Add Mat serializer and CameraCalibrationConfig * Begun calibration saving, fixed UI/Backend snapshot count mismatch * Add (unplumbed) option to set checkerboard size This will allow users to change the units their calibration is in * Create chessboard.png * Fix calibration NPE * changed string serialization to a json send * bug fixed cancellation button * Fix spelling of snapshot in 3d.vue * Plumb resolution change * Set resolution during config, start on config serialization * Update .gitignore * Config fixes * Start transition away from cvpipeline3d * fix NPE on uncalibrated cameras * clear list on fail * Fix video mode index error * ignore getters in camera calibration config * Create json constructor for jsonmat * get solvePNP mostly returning sane values * Fix solvePNP bug and add unit test * FIx calibration mat truncation * added capture amount model upload and minimap data * Standardize on meters in calibration and bounding box * fix json out of bounds and handle null calibration more gracefully * don't put text on calibrate image, go back to inches * convert distance to meters this means calibration will need to be in inches * Actually save raw contor * Update GroupContoursPipe.java * Add all calibration return to camera capture * hard code 2019 target * bugfixed draw2d added fail calib popup, merge end and cancel added the res index to the calib start * Clarify error message and draw more fancy rectangles * Cleanup memory in solvepnp * re did minimap component * fix npe if left/right is null * remove references to 2d * try-catch running the current pipeline * Add method to find corners using the harris corner detector * Possibly fix left/right missmatch * Fix 3D Tab error * FIx file permissions, mat serializer adjustments * fixed mini map for field coordinates * mini map changes fov * Update SolvePNPPipe.java * get rid of target corners * some memory leak fixes * fixed mini map location * added position under minimap * changed player fov look * put all targets in the web send * re did target send to ui added target tables, bugfix calibration * fixed y position * Add tilt angle to capture properties * maybe fix y axis in minimap * Add square size to onCalibrationEnding * Possibly add square size to UI * fix NPE with pitch * Fix bug with sending multiple targets * Only instantiate 3d stuff if we are in 3d mode * Fix array list exceptions * Fix bug in sort contors list was truncated too early * added download chess, tilt setting and ordinal tilt, * added square size connection * removed unused code * Update pom version to 2.1-RELEASE * Send camera calibrations to UI * Stream pose list to a LIst * Only stream necessary parts of the aux list entry * Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions * added fps counter changed squaresize steps bug fixes in tables * bugfix camera settings cam wont change Authored-by: oriagranat9 <oriagranat9@gmail.com>
2019-12-31 04:53:20 -08:00
private List<CameraCalibrationConfig> calibrationList;
private Mat imageBuffer = new Mat();
private USBCaptureProperties properties;
Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35) * Rebase solvePNP on master * added 3D tab minimap and csv reader * More solvePNP * Create draw pipe for pnp data * SolvePNP piping work * Move sorting into solvepnppipe * Create calibration pipeline * Update CalibrateSolvePNPPipeline.java * add camera tilt angle * Add calibration slider and snapshot button to 3D view * Mirror updates in the socket handler * add 3d calibration mode to the pipeline manager * created calibration functions in ui and backend * Start plumbing calibration * Add snapshot and other handling to the RequestHandler * added select resolution before starting calibration * Rename solvePNPPipe to bounding box solve pnp pipe * Update BoundingBoxSolvePNPPipe.java * Add Mat serializer and CameraCalibrationConfig * Begun calibration saving, fixed UI/Backend snapshot count mismatch * Add (unplumbed) option to set checkerboard size This will allow users to change the units their calibration is in * Create chessboard.png * Fix calibration NPE * changed string serialization to a json send * bug fixed cancellation button * Fix spelling of snapshot in 3d.vue * Plumb resolution change * Set resolution during config, start on config serialization * Update .gitignore * Config fixes * Start transition away from cvpipeline3d * fix NPE on uncalibrated cameras * clear list on fail * Fix video mode index error * ignore getters in camera calibration config * Create json constructor for jsonmat * get solvePNP mostly returning sane values * Fix solvePNP bug and add unit test * FIx calibration mat truncation * added capture amount model upload and minimap data * Standardize on meters in calibration and bounding box * fix json out of bounds and handle null calibration more gracefully * don't put text on calibrate image, go back to inches * convert distance to meters this means calibration will need to be in inches * Actually save raw contor * Update GroupContoursPipe.java * Add all calibration return to camera capture * hard code 2019 target * bugfixed draw2d added fail calib popup, merge end and cancel added the res index to the calib start * Clarify error message and draw more fancy rectangles * Cleanup memory in solvepnp * re did minimap component * fix npe if left/right is null * remove references to 2d * try-catch running the current pipeline * Add method to find corners using the harris corner detector * Possibly fix left/right missmatch * Fix 3D Tab error * FIx file permissions, mat serializer adjustments * fixed mini map for field coordinates * mini map changes fov * Update SolvePNPPipe.java * get rid of target corners * some memory leak fixes * fixed mini map location * added position under minimap * changed player fov look * put all targets in the web send * re did target send to ui added target tables, bugfix calibration * fixed y position * Add tilt angle to capture properties * maybe fix y axis in minimap * Add square size to onCalibrationEnding * Possibly add square size to UI * fix NPE with pitch * Fix bug with sending multiple targets * Only instantiate 3d stuff if we are in 3d mode * Fix array list exceptions * Fix bug in sort contors list was truncated too early * added download chess, tilt setting and ordinal tilt, * added square size connection * removed unused code * Update pom version to 2.1-RELEASE * Send camera calibrations to UI * Stream pose list to a LIst * Only stream necessary parts of the aux list entry * Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions * added fps counter changed squaresize steps bug fixes in tables * bugfix camera settings cam wont change Authored-by: oriagranat9 <oriagranat9@gmail.com>
2019-12-31 04:53:20 -08:00
public USBCameraCapture(FullCameraConfiguration fullCameraConfiguration) {
var config = fullCameraConfiguration.cameraConfig;
this.calibrationList = new ArrayList<>(); //fullCameraConfiguration.calibration;
calibrationList.addAll(fullCameraConfiguration.calibration);
baseCamera = new UsbCamera(config.name, config.path);
cvSink = CameraServer.getInstance().getVideo(baseCamera);
properties = new USBCaptureProperties(baseCamera, config);
2019-11-29 23:51:58 -05:00
int videoMode = properties.videoModes.size() - 1 <= config.videomode ? config.videomode : 0;
setVideoMode(videoMode);
}
Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35) * Rebase solvePNP on master * added 3D tab minimap and csv reader * More solvePNP * Create draw pipe for pnp data * SolvePNP piping work * Move sorting into solvepnppipe * Create calibration pipeline * Update CalibrateSolvePNPPipeline.java * add camera tilt angle * Add calibration slider and snapshot button to 3D view * Mirror updates in the socket handler * add 3d calibration mode to the pipeline manager * created calibration functions in ui and backend * Start plumbing calibration * Add snapshot and other handling to the RequestHandler * added select resolution before starting calibration * Rename solvePNPPipe to bounding box solve pnp pipe * Update BoundingBoxSolvePNPPipe.java * Add Mat serializer and CameraCalibrationConfig * Begun calibration saving, fixed UI/Backend snapshot count mismatch * Add (unplumbed) option to set checkerboard size This will allow users to change the units their calibration is in * Create chessboard.png * Fix calibration NPE * changed string serialization to a json send * bug fixed cancellation button * Fix spelling of snapshot in 3d.vue * Plumb resolution change * Set resolution during config, start on config serialization * Update .gitignore * Config fixes * Start transition away from cvpipeline3d * fix NPE on uncalibrated cameras * clear list on fail * Fix video mode index error * ignore getters in camera calibration config * Create json constructor for jsonmat * get solvePNP mostly returning sane values * Fix solvePNP bug and add unit test * FIx calibration mat truncation * added capture amount model upload and minimap data * Standardize on meters in calibration and bounding box * fix json out of bounds and handle null calibration more gracefully * don't put text on calibrate image, go back to inches * convert distance to meters this means calibration will need to be in inches * Actually save raw contor * Update GroupContoursPipe.java * Add all calibration return to camera capture * hard code 2019 target * bugfixed draw2d added fail calib popup, merge end and cancel added the res index to the calib start * Clarify error message and draw more fancy rectangles * Cleanup memory in solvepnp * re did minimap component * fix npe if left/right is null * remove references to 2d * try-catch running the current pipeline * Add method to find corners using the harris corner detector * Possibly fix left/right missmatch * Fix 3D Tab error * FIx file permissions, mat serializer adjustments * fixed mini map for field coordinates * mini map changes fov * Update SolvePNPPipe.java * get rid of target corners * some memory leak fixes * fixed mini map location * added position under minimap * changed player fov look * put all targets in the web send * re did target send to ui added target tables, bugfix calibration * fixed y position * Add tilt angle to capture properties * maybe fix y axis in minimap * Add square size to onCalibrationEnding * Possibly add square size to UI * fix NPE with pitch * Fix bug with sending multiple targets * Only instantiate 3d stuff if we are in 3d mode * Fix array list exceptions * Fix bug in sort contors list was truncated too early * added download chess, tilt setting and ordinal tilt, * added square size connection * removed unused code * Update pom version to 2.1-RELEASE * Send camera calibrations to UI * Stream pose list to a LIst * Only stream necessary parts of the aux list entry * Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions * added fps counter changed squaresize steps bug fixes in tables * bugfix camera settings cam wont change Authored-by: oriagranat9 <oriagranat9@gmail.com>
2019-12-31 04:53:20 -08:00
public CameraCalibrationConfig getCalibration(Size size) {
for(var calibration: calibrationList) {
if(calibration.resolution.equals(size)) return calibration;
}
return null;
}
public CameraCalibrationConfig getCalibration(VideoMode mode) {
return getCalibration(new Size(mode.width, mode.height));
}
public void addCalibrationData(CameraCalibrationConfig newConfig) {
calibrationList.add(newConfig);
}
public List<CameraCalibrationConfig> getConfig() {
return calibrationList;
}
@Override
public USBCaptureProperties getProperties() {
return properties;
}
2019-11-27 21:11:05 -08:00
@Override
public VideoMode getCurrentVideoMode() {
return baseCamera.getVideoMode();
}
@Override
public Pair<Mat, Long> getFrame() {
Long deltaTime;
// TODO: Why multiply by 1000 here?
Mat tempMat = new Mat();
deltaTime = cvSink.grabFrame(tempMat) * 1000L;
tempMat.copyTo(imageBuffer);
tempMat.release();
return Pair.of(imageBuffer, deltaTime);
}
@Override
public void setExposure(int exposure) {
try {
baseCamera.setExposureManual(exposure);
} catch (VideoException e) {
System.err.println("Failed to change camera exposure!");
}
}
@Override
public void setBrightness(int brightness) {
try {
baseCamera.setBrightness(brightness);
} catch (VideoException e) {
System.err.println("Failed to change camera brightness!");
}
}
@Override
public void setVideoMode(VideoMode mode) {
try {
baseCamera.setVideoMode(mode);
properties.updateVideoMode(mode);
} catch (VideoException e) {
System.err.println("Failed to change camera video mode!");
}
}
public void setVideoMode(int index){
VideoMode mode = properties.getVideoModes().get(index);
setVideoMode(mode);
}
@Override
public void setGain(int gain) {
if (properties.isPS3Eye) {
try {
baseCamera.getProperty("gain_automatic").set(0);
baseCamera.getProperty("gain").set(gain);
} catch (Exception e) {
System.err.println("Failed to change camera gain!");
}
}
}
Add solvePNP, 3d tab on the UI, and some other misc bug fixes (#35) * Rebase solvePNP on master * added 3D tab minimap and csv reader * More solvePNP * Create draw pipe for pnp data * SolvePNP piping work * Move sorting into solvepnppipe * Create calibration pipeline * Update CalibrateSolvePNPPipeline.java * add camera tilt angle * Add calibration slider and snapshot button to 3D view * Mirror updates in the socket handler * add 3d calibration mode to the pipeline manager * created calibration functions in ui and backend * Start plumbing calibration * Add snapshot and other handling to the RequestHandler * added select resolution before starting calibration * Rename solvePNPPipe to bounding box solve pnp pipe * Update BoundingBoxSolvePNPPipe.java * Add Mat serializer and CameraCalibrationConfig * Begun calibration saving, fixed UI/Backend snapshot count mismatch * Add (unplumbed) option to set checkerboard size This will allow users to change the units their calibration is in * Create chessboard.png * Fix calibration NPE * changed string serialization to a json send * bug fixed cancellation button * Fix spelling of snapshot in 3d.vue * Plumb resolution change * Set resolution during config, start on config serialization * Update .gitignore * Config fixes * Start transition away from cvpipeline3d * fix NPE on uncalibrated cameras * clear list on fail * Fix video mode index error * ignore getters in camera calibration config * Create json constructor for jsonmat * get solvePNP mostly returning sane values * Fix solvePNP bug and add unit test * FIx calibration mat truncation * added capture amount model upload and minimap data * Standardize on meters in calibration and bounding box * fix json out of bounds and handle null calibration more gracefully * don't put text on calibrate image, go back to inches * convert distance to meters this means calibration will need to be in inches * Actually save raw contor * Update GroupContoursPipe.java * Add all calibration return to camera capture * hard code 2019 target * bugfixed draw2d added fail calib popup, merge end and cancel added the res index to the calib start * Clarify error message and draw more fancy rectangles * Cleanup memory in solvepnp * re did minimap component * fix npe if left/right is null * remove references to 2d * try-catch running the current pipeline * Add method to find corners using the harris corner detector * Possibly fix left/right missmatch * Fix 3D Tab error * FIx file permissions, mat serializer adjustments * fixed mini map for field coordinates * mini map changes fov * Update SolvePNPPipe.java * get rid of target corners * some memory leak fixes * fixed mini map location * added position under minimap * changed player fov look * put all targets in the web send * re did target send to ui added target tables, bugfix calibration * fixed y position * Add tilt angle to capture properties * maybe fix y axis in minimap * Add square size to onCalibrationEnding * Possibly add square size to UI * fix NPE with pitch * Fix bug with sending multiple targets * Only instantiate 3d stuff if we are in 3d mode * Fix array list exceptions * Fix bug in sort contors list was truncated too early * added download chess, tilt setting and ordinal tilt, * added square size connection * removed unused code * Update pom version to 2.1-RELEASE * Send camera calibrations to UI * Stream pose list to a LIst * Only stream necessary parts of the aux list entry * Make broadcastMessage synchronized to prevent ConcurrentModificationExceptions * added fps counter changed squaresize steps bug fixes in tables * bugfix camera settings cam wont change Authored-by: oriagranat9 <oriagranat9@gmail.com>
2019-12-31 04:53:20 -08:00
@Override
public CameraCalibrationConfig getCurrentCalibrationData() {
return getCalibration(getCurrentVideoMode());
}
@Override
public List<CameraCalibrationConfig> getAllCalibrationData() {
return calibrationList;
}
}