[WIP] Simulation Overhaul (#742)

### What does this do?

- Deprecates previous sim classes
- Has a `CameraProperties` class for describing a camera's basic/calibration info, and performance values for simulation. Calibration values can be loaded from the `config.json` in the settings exported by photonvision.
- `OpenCVHelp` provides convenience functions for using opencv methods with wpilib/photonvision classes, mainly to project 3d points to a camera's 2d image and perform solvePnP with the above camera calibration info.
  - `TargetModel`s describe the 3d shape of a target, both for projecting into the camera's 2d image and use in solvePnP.
- `PhotonCameraSim` uses camera properties to simulate how 3d targets would appear in its view, and has simulated noise, latency, and FPS. For apriltags, the best/alternate camera-to-target transform is also estimated with solvePnP.
  - `VideoSimUtil` has helper functions for drawing apriltags to a simulated raw and processed MJPEG stream for each camera using the projected tag corners.
- `VisionSystemSim` stores `VisionTargetSim`s and `PhotonCameraSim`s, and is periodically updated with the robot's simulated pose. When updating, camera sims are automatically processed and published with their visible targets from their respective poses with proper latency.

### What's still not working?

- Mac Arm builds are broken
- More examples
- Update website/docs
This commit is contained in:
amquake
2023-06-18 15:54:12 -07:00
committed by GitHub
parent 4a94775639
commit f1cadc1e1e
56 changed files with 2471 additions and 199 deletions

View File

@@ -51,10 +51,10 @@ import org.photonvision.common.hardware.VisionLEDMode;
import org.photonvision.targeting.PhotonPipelineResult;
/** Represents a camera that is connected to PhotonVision. */
public class PhotonCamera {
static final String kTableName = "photonvision";
public class PhotonCamera implements AutoCloseable {
public static final String kTableName = "photonvision";
protected final NetworkTable cameraTable;
private final NetworkTable cameraTable;
RawSubscriber rawBytesEntry;
BooleanPublisher driverModePublisher;
BooleanSubscriber driverModeSubscriber;
@@ -73,6 +73,7 @@ public class PhotonCamera {
private DoubleArraySubscriber cameraIntrinsicsSubscriber;
private DoubleArraySubscriber cameraDistortionSubscriber;
@Override
public void close() {
rawBytesEntry.close();
driverModePublisher.close();
@@ -151,9 +152,7 @@ public class PhotonCamera {
m_topicNameSubscriber =
new MultiSubscriber(
instance,
new String[] {"/photonvision/"},
new PubSubOption[] {PubSubOption.topicsOnly(true)});
instance, new String[] {"/photonvision/"}, PubSubOption.topicsOnly(true));
}
/**
@@ -186,7 +185,7 @@ public class PhotonCamera {
ret.createFromPacket(packet);
// Set the timestamp of the result.
// getLatestChange returns in microseconds so we divide by 1e6 to convert to seconds.
// getLatestChange returns in microseconds, so we divide by 1e6 to convert to seconds.
ret.setTimestampSeconds((rawBytesEntry.getLastChange() / 1e6) - ret.getLatencyMillis() / 1e3);
// Return result.
@@ -334,6 +333,14 @@ public class PhotonCamera {
} else return Optional.empty();
}
/**
* Gets the NetworkTable representing this camera's subtable. You probably don't ever need to call
* this.
*/
public final NetworkTable getCameraTable() {
return cameraTable;
}
private void verifyVersion() {
if (!VERSION_CHECK_ENABLED) return;