Team2890/PhotonVision
4
0
Fork 0
mirror of https://github.com/PhotonVision/photonvision synced 2026-06-28 02:11:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add GPU Acceleration on the Raspberry Pi (#140)

Browse Source 
* Add native stuff

* use runtimeloader

* add more native methods

* more stuff

* Switch JNI methods to static

* Remove non-java classes from the picam jni

* Add gradle task for JNI generation

* Migrate my previous GPU accel work

* Initial work on defining JNI interface

* Change libpicam to a symlink for now

* Initial work on adding no-copy OMX GPU accel on the pi

* Make DIRECT_OMX GPU accel mode not crash

* Clean up OMX changes (still not getting valid data back)

* Re-add GPU unit test

* A couple debugging tweaks/notes

* Add temporary special cases to get RGB out of ProcessingMode.NONE

* Code clarity improvements; fix possible VBO bug

* Get DIRECT_OMX working

* Remove some debugging switches in GPUAccelerator

* Pipe in VCSM stuff to read out pixels FAST

* Apply Spotless

* Revert versioningHelper changes

* Add missing import

* Convert to MMAL and move everything to native

* Re-add shared object

* Rework to use MMAL and do everything natively

* Condense pipeline settings classes

* Add OutputStreamPipeline

* Apply spotless

* Fix duplicate variable inits and add more video modes

* Integrate color frames and latency measurements for GPU

* Fix camera detection on pi and other platforms

* Add proper color copy disabling and camera settings calls

* Fix things that were broken by rebase

* Fix spotless issues and remove uneeded prints

* Remove libpicam symlink

* Fix stream resolution limiting

* Remove testing code in GPUAcceleratedHSVPipe

* Make profiling options general to all computers

* Make PicamJNI load from resources

* run spotlessApply

* Address review comments

* Update Maven repo for JOGL

* Fix release race condition

* Only run GPU accel test on the pi

* Lint fix and merge conflict accident fixes

* Make Jackson ignore extra fields when unmarshalling HardwareConfig

* Fix Mat releasing data race

* Spotless apply

* Remove broken header generation task

* Fix shared library loading typo

* Add a ZeroCopyPicam quirk to allow setting gain with the MMAL backend

* Make sure that exposure/brightness/gain get set after res changes

* Make rawInputMat properly local

* Remove bogus set of shouldRun flag

* Clean up small GPUHSVPipe print

* Add in some things that missed the ZeroCopyPiCameraSource rename

* Fix incorrect scoping introduced in past rebase

* Don't filter out too-low resolutions

* Only show latency when GPU accel is enabled

* Don't free Mats in stream thread before we use them

* Fix use-after-free and latency caluclation bugs on USB camera source

* Update libpicam

* Remove unwanted print

* Add libpicam forceLoad in unit test

* Fix streaming during camera calibration

* Fix zerocopy Picam calculation

* Use logger trace method instead of raw prints

* Fix calibration and driver mode pipes with the Picam

Co-authored-by: Matt <matthew.morley.ca@gmail.com>
Co-authored-by: Banks Troutman <btrout.dhrs@gmail.com>
This commit is contained in:
Declan Freeman-Gleason
2020-12-08 02:34:21 -05:00
committed by GitHub
parent 15d21b7841
commit c3dbd45716
50 changed files with 1705 additions and 397 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
photon-client/src/views/PipelineViews/InputTab.vue
View File

@@ -56,7 +56,6 @@
tooltip="Resolution to which camera frames are downscaled for streaming to the dashboard"
:list="streamResolutionList"
:select-cols="largeBox"
@input="handlePipelineData('streamingFrameDivisor')"
@rollback="e => rollback('streamingFrameDivisor', e)"
/>
</div>
@@ -66,6 +65,8 @@
import CVslider from '../../components/common/cv-slider'
import CVselect from '../../components/common/cv-select'
const unfilteredStreamDivisors = [1, 2, 4, 6];
export default {
name: 'Input',
components: {
@@ -75,7 +76,9 @@
// eslint-disable-next-line vue/require-prop-types
props: ['value'],
data() {
return {}
return {
rawStreamDivisorIndex: 0,
}
},
computed: {
largeBox: {
@@ -120,18 +123,22 @@
},
cameraVideoModeIndex: {
get() {
return this.$store.getters.currentPipelineSettings.cameraVideoModeIndex
return this.$store.getters.currentPipelineSettings.cameraVideoModeIndex;
},
set(val) {
this.$store.commit("mutatePipeline", {"cameraVideoModeIndex": val});
this.handlePipelineUpdate("streamingFrameDivisor", this.getNumSkippedStreamDivisors());
this.rawStreamDivisorIndex = 0;
}
},
streamingFrameDivisor: {
get() {
return this.$store.getters.currentPipelineSettings.streamingFrameDivisor
return this.rawStreamDivisorIndex;
},
set(val) {
this.$store.commit("mutatePipeline", {"streamingFrameDivisor": val});
this.rawStreamDivisorIndex = val;
this.handlePipelineUpdate("streamingFrameDivisor", this.getNumSkippedStreamDivisors() + val);
}
},
@@ -147,21 +154,31 @@
streamResolutionList: {
get() {
let cam_res = this.$store.getters.videoFormatList[
this.$store.getters.currentCameraSettings.currentPipelineSettings.cameraVideoModeIndex]
const cam_res = this.$store.getters.videoFormatList[
this.$store.getters.currentCameraSettings.currentPipelineSettings.cameraVideoModeIndex];
let tmp_list = [];
tmp_list.push(`${Math.floor(cam_res['width'])} X ${Math.floor(cam_res['height'])}`);
for (let x = 2; x <= 6; x += 2) {
for (const x of this.getRawStreamDivisors()) {
tmp_list.push(`${Math.floor(cam_res['width'] / x)} X ${Math.floor(cam_res['height'] / x)}`);
}
return tmp_list;
}
}
},
methods: {}
methods: {
getRawStreamDivisors() {
// Limit stream res when GPU acceleration is enabled because we *know* that we won't be able to get smooth streams above ~640x480
// It would probably be cleaner if this checked that we're on the Raspi 3 instead of checking for GPU accel status
const width = this.$store.getters.videoFormatList[
this.$store.getters.currentCameraSettings.currentPipelineSettings.cameraVideoModeIndex]['width'];
return unfilteredStreamDivisors.filter((x) => !this.$store.state.settings.general.gpuAcceleration || width / x < 400);
},
getNumSkippedStreamDivisors() {
return unfilteredStreamDivisors.length - this.getRawStreamDivisors().length;
}
}
}
</script>
<style scoped>
</style>
</style>