Arducam 16 MP Raspberry Pi with OpenCV

Aros · August 14, 2023, 3:29pm

Hi,
I am trying to make the 16 MP Arducam work on Raspi4 with OpenCV. I have followed the Arducam tutorial to make it working with libcamera. Which it does. At least libcamera-hello does it’s job.

Now what I need to achieve is a simple camera feed on the screen similar to what libcamera-hello -f would do, but I need to implement keyboard controlled Zoom in / out. Which is effectively realtime adjustment of the ROI.

Since I know my way around OpenCV, my first bet is to use that, but for some reason, I cannot open the camera in OpenCV (I’m trying C++, not Python). I installed the latest OpenCV (4.8), but it fails when trying to open the camera. When I call cv::VideoCapture::open, it fails with the following console output:

[ WARN:[email protected]] global cap_gstreamer.cpp:2784 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Failed to allocate required memory.
[ WARN:[email protected]] global cap_gstreamer.cpp:1679 open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:[email protected]] global cap_gstreamer.cpp:1164 isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

Which is weird as I use Rpi4 with 4 GB RAM. I also tried increasing the video memory to 256 MB, but I don’t think that’s the problem given what the message says. And it of course didn’t help.

I wasn’t able to find anything useful regarding arducam and C++ OpenCV. Is there anything else I need to do?

Also, I assume I should be able to achieve the above mentioned functionality using libcamera only, but first, I’m trying to get it working with OpenCV since that would be waaaay easier for me not having to learn a new API…

Edward · August 17, 2023, 1:03am

@Aros

You can look at the example I wrote earlier

github.com

edward-ardu/libcamera-cpp-demo/blob/main/main.cpp

#include <opencv2/imgproc.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>

#include "LibCamera.h"

using namespace cv;

int main() {
    time_t start_time = time(0);
    int frame_count = 0;
    float lens_position = 100;
    float focus_step = 50;
    LibCamera cam;
    uint32_t width = 1920;
    uint32_t height = 1080;
    uint32_t stride;
    char key;
    int window_width = 1920;

This file has been truncated. show original

Aros · August 21, 2023, 8:37pm

Hi, I really appreciate the suggestion.

I have tried, but it throws this output into console:

[0:15:16.818221704] [1932]  INFO Camera camera_manager.cpp:297 libcamera v0.0.5+83-bde9b04f
[0:15:16.906931371] [1933]  INFO RPI vc4.cpp:437 Registered camera /base/soc/i2c0mux/i2c@1/imx519@1a to Unicam device /dev/media2 and ISP device /dev/media0
Configuring still capture...
Stream configuration adjusted
Still capture setup complete
[0:15:16.908277691] [1932]  INFO Camera camera.cpp:1033 configuring streams: (0) 1920x1080-RGB888
[0:15:16.909406587] [1933]  INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/imx519@1a - Selected sensor format: 1920x1080-SRGGB10_1X10 - Selected unicam format: 1920x1080-pRAA
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.8.0) /home/pi/opencv-src/modules/highgui/src/window.cpp:1272: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage'

Aborted

Do I need to tweak some params in main.cpp for this specific sensor?

// I cannot seem to figure out where formats namespace / class used in the call to configureStill come from…