# Camera Matching

Diagrams generated by the [PlantUML UML editor](https://www.plantuml.com/plantuml/). Copy the image URLs below and decode in the editor to make changes.

## Initial Setup

When PhotonVision first starts, settings are loaded from disk and [VisionSources](https://javadocs.photonvision.org/release/org/photonvision/vision/processes/VisionSource.html) are created for every serialized & active [Camera Configuration](https://javadocs.photonvision.org/release/org/photonvision/common/configuration/CameraConfiguration.html)

![](https://www.plantuml.com/plantuml/png/VP5FQnin4CNl-XI3JotK-DAJAI6fIw6GfOMbFkKoramSqTKVfF6MVFkETfKsei6trVpUldbwkYs2MIv-CeI29omCcn5d9XXPn8LpsG0MAErWaggTTGc3m6P05nRizQD7HrTS3336IxOC0mOySrwqS_5lIeT8bubxgVTNN9jRhpYCXvXNP8lLpokxsWvZNcwtlQaNsSDzH8B773sGAxzC7MvlDFSUxeXWKie4DeP7futelC8z73AZCDnPSJD35xKOh5F5DR31IU3d-1aiUive06PTlSRTm_V4eH4uFJ-4Aamn2xmxFMyJojDx0x2AjtNn-WSJ73_UltRyzC_o2mjRQH1IZecpE4t5WPOmX_5R7sPof_NyVvwghNbK-LVL1sbErTneFLqxNxF27pdEZZXNs8gjbJFrhHdYLxMredrx1Obm70QZvnUBtKxdJE2NnosxNVj3qIYO1GB_Rb3DEZAlQxKPowMuS7u8oIMUNE0F84-PaOgvvK0NF_q1)

## UI Workflow

A [background thread](https://javadocs.photonvision.org/org/photonvision/common/util/TimedTaskManager.html) will periodically query CSCore and Libcamera for what cameras we currently see connected. This list is provided to the web UI for display.

![](https://www.plantuml.com/plantuml/svg/POvDJyCm343l-HLMxnFt7j14uJ099AHkSCvSCopoCSLE-FjaxQW8kpbwpy_PYjgasJk3qJb2vHW4kZrxcc1lvGjURB0dIXrO0LLlpBakCFBP1eNkZQLkm1XpGchS8hvLXt68YMQ6WdLiyJCVqNfATZRSxwkLtka8XzriP3P6rM_kww4U7hac2oK8z0qJ5KOIKwJYvLOFJo5VUafm61zWYOjPwEPQ6M88X4fJuyoPzKD_IyEuMwrLk8rLhOrbxk4rooVWwbmvE1Rz9rbKBdJ7OHakInzy4hEbC6NlVW00)

![](images/matching_ui.png)

This UI allows users to "Activate" a camera that's never been seen before, or activate a CameraConfiguration we've seen before but was disabled. Allowing camera configurations to be saved but not loaded by default lets us support temporarily disabling/unplugging a camera without flooding log files.

Since our backend logic intentionally does not protect users from plugging camera B into the port that camera A was active on, the UI shall show a warning but vision processing will (attempt to) continue like normal.

### Activate New Camera

When a new camera (ie, one we can't match by-path to a deserialized CameraConfiguration) is activated, we'll create a spin up a new Vision Module for it

![](https://www.plantuml.com/plantuml/svg/VL7B3jCm4BpxArOzWKIK2wSALOKWf4gDG8fQBhquzggrY1_u4TI_PvCufGRKK-ATySpiU1yYzp7fWJdwAg4SDn4stx67qs43F41I9NHMGLa3dKrU8BJSy2lwcJa6_LzgQsKQ_g9g_K8rgvMCfckiNo0H1FsMy57rWclqV6OCw-b5e1o4iQIg7MNVmaSfeRz3CkfdGZ0am6YUmOuR5UyWRYX-X7M-XSOZZmX5_i2uY6ga-RG5uqE4K_S9SYAWORLRTjZ2LuSc8-HzCHFHMH_XJN-l78-tjmpomjNakDn02UVtnrKHZPnDckvGcZng-DU7kBCFCH-imk1PdDRzy2VoPumeuYhcl7L87UDKIj795q-CRzwEIgAVmDpaqNA9igoCINpgBDUhyvj42-UsPNHU9UgQvgIXvvSCTRtUe7UAt4Sm-2k395OWus9BiGM6eCprOfnoE2Y3xo3UF78Ps1wDJ7hu3G00)

### Deactivate Camera

Deactivating a camera will release the native resources it owns, and return the CameraConfiguration to the pool of currently disabled cameras we can re-enable later.

![](https://www.plantuml.com/plantuml/svg/ROxBJiCm44Nt_efHLtIH7yWYgWWB8gYeI0eRDXDdW97yABQd5N-FvV1G8bOUppdNrxkOC2InHftooPfFw19idcc4OxS1Z22yH4ySsJlelGHDi4U7RnIAUOxsNtNl9p4hrQxKjczzeC9qr7bSudiUDLeAM0ppSrDAk6foRmqtX3hn6HD16GXcvSMDdo2EFuJ0vOtATexO77aawxDdo_TKNbLLCvVNq1eV_vwuwbxXs5zllwNV_Xe6mZ3vYrkeRTzjvvv6k8Q3n7TmT86OC541LG6tmt20Xpkr8pU9DLy0)

### Reactivate a CameraConfig

When a new camera (ie, one we can't match by-path to a deserialized CameraConfiguration) is activated, we'll create and spin up a new Vision Module for it.

![](https://www.plantuml.com/plantuml/svg/VP9BYnD158NtzIikirAmoSPL8s5YH1n8SB1duYQRsrtNcGlrAElHadzlLVgXXP9LACvNvvmwwViGqSUabN3vbmTsQ2BSVQSUdX_k00CahgKJ1xO6EflyG714Wo_ah-GOz7_HevL9KOrgVSDrTgk9VRUtVfA6C5XFjNpWVa1D7g-4Maut2ir5X4ZSR7Ft5huH3f57Z0II0_QA94msPzDV81d-cGWCQX82LOJdxYCuwoEmWHH8G9cWsIPkuSlJqoFyG5R9ao0ZXIXIZcbXxwaax4eKGVNm8DO2OrWpvWvN-sOxFRw5huxCh41_EPkrp9l-qZYChsy5m0GtKt2vGH9Exm-BOobMGlRTGnsoxlTlJc5BJYPNgWgOuUNL7_vK_aIHXhYOEMyT-SWKCbLDyzbduj7RaINv8ix_py6Y95bF9YJzjTcyiixmJag85ax7eyZdnMApsSdYeQ-VGDXibXijT15z14E_5b6CbJ9EiRdsG26mUJaRnuuK6te7yTKJoY3koSYarMy0)

# Camera Matching Requirements

## Definitions
- VALID USB PATH: a path in the form `/dev/v4l/by-path/[UUID]`
- VIDEO DEVICE PATH: a CSCore-provided identifier derived from the V4L path `/dev/video[N]` on Linux, or an opaque string on Windows
- UNIQUE NAME: an identifier that is unique within the set of all deserialized CameraConfigurations and unmatched USB cameras
  - I don't love this, it means that a USB camera matched to a VisionModule will share a UNIQUE NAME, right?
- DESERIALIZED CAMERA CONFIGURATIONS: The set of camera configurations loaded from disk and provided to the VisionSourceManager. This configuration data structure includes the UNIQUE NAME
- CURRENTLY ACTIVE CAMERAS: The set of VisionModules currently active and processing vision data, and associated metadata

## Startup:

- GIVEN An empty set of deserialized Camera Configurations
<br>WHEN PhotonVision starts
<br>THEN no VisionModules will be started

- GIVEN A valid set of deserialized Camera Configurations
<br>WHEN PhotonVision starts
<br>THEN VisionModules will be started FOR EACH un-DISABLED config

- GIVEN A valid set of deserialized Camera Configurations
<br>WHEN PhotonVision starts
<br>THEN VisionModules will NOT be started FOR EACH DISABLED config

- GIVEN A CameraConfiguration with a VALID USB PATH
<br>WHEN a VisionModule is created
<br>THEN The VisionModule shall open the camera using the USB path

- GIVEN A CameraConfiguration without a valid USB path
<br>WHEN a VisionModule is created
<br>THEN The VisionModule shall open the camera using the VIDEO DEVICE PATH

## Camera (re)enumeration:

- GIVEN a NEW USB CAMERA is available for enumeration
<br>WHEN a USB camera is discovered by VisionSourceManager
<br>AND the USB camera's VIDEO DEVICE PATH is not in the set of DESERIALIZED CAMERA CONFIGURATIONS
<br>THEN a UNIQUE NAME will be assigned to the camera info

- GIVEN a NEW USB CAMERA is available for enumeration
<br>WHEN a USB camera is discovered by VisionSourceManager
<br>AND the USB camera's VIDEO DEVICE PATH is in the set of DESERIALIZED CAMERA CONFIGURATIONS
<br>THEN a UNIQUE NAME equal to the matching DESERIALIZED CAMERA CONFIGURATION will be assigned to the camera info
  - This is a weird case. How -should- we handle this? see above

## Creating from a new camera

- Given: A UNIQUE NAME from a NEW USB CAMERA
<br>WHEN I request a new VisionModule is created for this NEW USB CAMERA
<br>AND the camera has a VALID USB PATH
<br>AND the camera's VALID USB PATH is not in use by any CURRENTLY ACTIVE CAMERAS
<br>THEN a NEW VisionModule will be started for the NEW USB CAMERA using the VALID USB PATH

- Given: A UNIQUE NAME from a NEW USB CAMERA
<br>WHEN I request a new VisionModule is created for this NEW USB CAMERA
<br>AND the camera does not have a VALID USB PATH
<br>AND the camera's VIDEO DEVICE PATH is not in use by any CURRENTLY ACTIVE CAMERAS
<br>THEN a NEW VisionModule will be started for the NEW USB CAMERA using the VIDEO DEVICE PATH

## Deactivate

- Given: A UNIQUE NAME from a CURRENTLY ACTIVE CAMERA
<br>WHEN I request the VisionModule be DEACTIVATED
<br>THEN the VisionModule will be stopped for the given CURRENTLY ACTIVE CAMERA
<br>AND the CameraConfiguration DISABLED flag will be set to TRUE

## Reactivate

- Given: A UNIQUE NAME from a DESERIALIZED CAMERA CONFIGURATIONS
<br>WHEN I request the VisionModule be ACTIVATED
<br>AND the CameraConfiguration's DISABLED flag is TRUE
<br>THEN a VisionModule will be created and started for the camera