Arducam 16MP / IMX519 / Rapsberry Pi: ffmpeg cannot record from camera

  1. Where did you get the camera module(s)?
    Kickstarter

  2. Model number of the product(s)?
    N/A

  3. What hardware/platform were you working on?
    Raspberry Pi Zero 2 W

  4. Instructions you have followed. (link/manual/etc.)
    I followed all instructions regarding the installation of the module. Tests with libcamera-still and libcamera-vid are successful.

  5. Problems you were having?
    I can’t get ffmpeg to record video. I tried using /dev/video0 with v4l2 but had no success. In the most simple form, the record command looks like this:

ffmpeg -f v4l2 -i /dev/video0 test.avi

Result:

libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
[video4linux2,v4l2 @ 0x881cd0] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0x881cd0] Time per frame unknown
[video4linux2,v4l2 @ 0x881cd0] ioctl(VIDIOC_STREAMON): Invalid argument
/dev/video0: Invalid argument

In dmesg I get the following entry every time I try to start ffmpeg:
unicam 3f801000.csi: Failed to start media pipeline: -22

  1. The dmesg log from your hardware?
    imx519 10-001a: Device found is imx519
    imx519 10-001a: Consider updating driver imx519 to match on endpoints

and in case of an ffmpeg start:
unicam 3f801000.csi: Failed to start media pipeline: -22

  1. Troubleshooting attempts you’ve made?
    I tried all sorts of additional parameters, other video devices, different configurations in /boot/config.json, but nothing fixed the issue.

I also used v4l2-compliant to check if /dev/video0 should works, the result is good:
Total for unicam device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0

  1. What help do you need?
    I’m using a third party software that supports ffmpeg video streams as an input. So I need to get the camera running with ffmpeg. All my tests via v4l2 where unsuccessful so far.

Best regards

1024kilobyte

Unfortunately, the video0 node outputs Bayer RAW format, which ffmpeg doesn’t support, and it doesn’t have ISP support either.
What libcamera does is get RAW image from video0, send it to ISP for processing, and output the result. If you don’t have special requirements, you can try V4l2loopback, check Create a virtual video node (Access by Python):

Thanks for the explanation.

I now created a v4l2 loopback device and tried to convert the camera stream to a ffmpeg compatible one using v4l2tools (https://github.com/mpromonet/v4l2tools). Unfortunately without any luck. Using

v4l2compress /dev/video0 loopbackDevice

I get the same error message, as with ffmpeg:

VIDIOC_STREAMON: Invalid argument

Isn’t there any practical way to convert the camera stream to a more usable format? (Without the need to write your own code)

You can refer to the method in the documentation I provided to use v4l2loopback:

sudo modprobe v4l2loopback video_nr=3
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080' ! videoconvert ! tee ! v4l2sink device=/dev/video3

If you just want to simply let ffmpeg receive the data output by libcamera, you can use pipe, for example this is an example of using libcamera to encode h264 and use ffmpeg to pack as mp4:

libcamera-vid -t 0 -o - | ffmpeg -i - -c:v copy test.mp4