Hi, I’m working on a remote-controlled sidewalk mobile robot for a Japanese manufacturer and I’m trying to cut down camera latency. We have 8 remote control cameras installed on the robot and right now the latency is around 300-400 ms, which is too much for remote driving. I need to get it down by more than half while still keeping the robot affordable.
Current setup:
Camera kit: B0331 (OV9782) — the 1MP*4 Quadrascopic Camera Bundle Kit
Two Jetson Nanos, each with 4 x 120-degree cameras installed
Resolution/frame rate: 800x600 @ 15fps
FoV: 120°
We’d like the 8 cameras to work via WebRTC for remote control; the sensor interface is flexible as long as all 8 can run that way.
For reference, our existing cameras are ELP units with 1/2.8" Sony IMX291 sensors, output 800x600 @ 15fps over USB 3.0.
We’ll probably need to try several prototypes. Delivery target is end of November but earlier is better. Where should we start?
Thanks for the detailed write-up. Before we chase latency, let’s make sure the capture path itself is healthy. On a Jetson Nano the usual first step is to test the camera straight through GStreamer.
Do you have any update on this? We started trying to build the 4 Quadrascopic Camera with the Jetson Nano last week and got stuck.
We tried testing with GStreamer on the Jetson Nano but the test failed. This is the pipeline we used:
gst-launch-1.0 v4l2src device=“/dev/video0” ! videoconvert ! ximagesink
The console just floods with “A lot of buffers are being dropped.” and we never get a stable image.
Thanks — that tells us the capture stream isn’t stable yet, so let’s not worry about latency until we can get a clean frame out.
Can you run the bug report tool from my earlier message and attach arducam-bug-report.txt? That will show us which kernel and driver build you’re running for the OV9782 on the Nano.
Ran it — arducam-bug-report.txt is attached. Also, could you share sample code for screen output using GStreamer? We mainly want to prove we can get an image on screen first. I’ve also sent the signed NDA and our inquiry details through private message, let me know if you received them.
Got the report and your message, thanks. Here’s the key thing for this kit: the jevariety camera is reported to be incompatible with GStreamer. That’s why your v4l2src pipeline is throwing all those dropped buffers.
So if I’m reading that right, the jevariety camera just can’t be used with GStreamer at all, and I should use step 3.B instead to verify the sensor? We’ll try that now. Once we get a clean image we’ll come back to the WebRTC/latency question for the 8-camera robot setup.