Libcamera-vid fails to produce h264 with IMX462 camera

  1. Where did you get the camera module(s)? uctronics

  2. Model number of the product(s)? imx462 low light camera

  3. What hardware/platform were you working on? rPi 3A+

  4. Instructions you have followed. (link/manual/etc.) How to install and build Raspberry Pi Kernel Driver for Arducam Pivariety Camera

  5. Problems you were having? libcamera-vid ... -o test.h264 hangs

  6. The dmesg log from your hardware? – I can post the full log, there are no unusual error messages. And:

$ dmesg|head
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 5.10.60-v7+ ([email protected]) (arm-linux-gnueabihf-gcc-8 (Ubuntu/Linaro 8.4.0-3ubuntu1) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1449 SMP Wed Aug 25 15:00:01 BST 2021
[    0.000000] CPU: ARMv7 Processor [410fd034] revision 4 (ARMv7), cr=10c5383d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] OF: fdt: Machine model: Raspberry Pi 3 Model A Plus Rev 1.0
[    0.000000] random: fast init done
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] Reserved memory: created CMA memory pool at 0x07c00000, size 256 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool

$ dmesg|egrep arducam
[    8.393648] arducam: loading out-of-tree module taints kernel.
[    8.410354] arducam-pivariety 10-000c: firmware version: 0x10002
[   33.754466] arducam_vddl: disabling
[   33.754486] arducam_vdig: disabling
[   33.754507] arducam_vana: disabling
  1. Troubleshooting attempts you’ve made? see below

  2. What help do you need? Information on how to make it work’


I’m using the arducam imx462 low light camera and cannot get h264 to work. I installed everything per instructions (multiple times). The one thing that I have different from the instructions is that after an apt upgrade I now have the latest kernel. But I have reinstalled the arducam kernel driver after upgrading the kernel.

$ uname -a
Linux rpi-cam2 5.10.60-v7+ #1449 SMP Wed Aug 25 15:00:01 BST 2021 armv7l GNU/Linux

What happens is the following (taken from the instructions) hangs:

$ libcamera-vid -t 10000 --width 1920 --height 1080 -n -v -o test.h264
Options:
    verbose: 1
    info_text:#%frame (%fps fps) exp %exp ag %ag dg %dg
    timeout: 10000
    width: 1920
    height: 1080
    output: test.h264
    rawfull: 0
    preview: none
    transform: identity
    roi: all
    metering: centre
    exposure: normal
    ev: 0
    awb: auto
    flush: false
    wrap: 0
    brightness: 0
    contrast: 1
    saturation: 1
    sharpness: 1
    framerate: 30
    denoise: auto
    viewfinder-width: 0
    viewfinder-height: 0
    bitrate: 0
    profile:
    level:
    intra: 0
    inline: 0
    save-pts:
    codec: h264
    quality (for MJPEG): 50
    keypress: 0
    signal: 0
    initial: record
    split: 0
    segment: 0
    circular: 0
Opened H264Encoder on /dev/video11 as fd 3
Got 6 output buffers
Got 12 capture buffers
Codec streaming started
Running without preview window
Opening camera...
[2:54:55.418254864] [1041]  INFO Camera camera_manager.cpp:296 libcamera v0.0.0+2736-579f55b1-dirty
(2021-07-01T03:47:30-04:00)
[2:54:55.433443946] [1046]  WARN CameraSensor camera_sensor.cpp:197 'arducam-pivariety 10-000c': Rec
ommended V4L2 control 0x009a0922 not supported
[2:54:55.447397721] [1046]  WARN CameraSensor camera_sensor.cpp:249 'arducam-pivariety 10-000c': The
 sensor kernel driver needs to be fixed
[2:54:55.447499856] [1046]  WARN CameraSensor camera_sensor.cpp:251 'arducam-pivariety 10-000c': See
 Documentation/sensor_driver_requirements.rst in the libcamera sources for more information
[2:54:55.454658566] [1046]  WARN CameraSensorProperties camera_sensor_properties.cpp:109 No static p
roperties available for 'arducam-pivariety'
[2:54:55.454788305] [1046]  WARN CameraSensorProperties camera_sensor_properties.cpp:111 Please cons
ider updating the camera sensor properties database
[2:54:55.454876273] [1046]  WARN CameraSensor camera_sensor.cpp:403 'arducam-pivariety 10-000c': Fai
led to retrieve the camera location
[2:54:55.695713045] [1046] ERROR IPAProxy ipa_proxy.cpp:149 Configuration file 'arducam-pivariety.js
on' not found for IPA module 'raspberrypi'
[2:54:56.115404801] [1046] ERROR DelayedControls delayed_controls.cpp:87 Delay request for control i
d 0x009a090a but control is not exposed by device /dev/video0
Acquired camera /base/soc/i2c0mux/[email protected]/[email protected]
Configuring video...
[2:54:56.124474698] [1041]  WARN V4L2 v4l2_pixelformat.cpp:181 Unsupported V4L2 pixel format XB24
[2:54:56.126173491] [1041]  INFO Camera camera.cpp:906 configuring streams: (0) 1920x1080-YUV420
[2:54:56.127162235] [1046]  INFO RPI raspberrypi.cpp:128 Mode: 1920x1080 fmt RG10 Score: 1500 (best
1500)
[2:54:56.127328797] [1046]  INFO RPI raspberrypi.cpp:128 Mode: 1920x1080 fmt pRAA Score: 1000 (best
1000)
[2:54:56.194027849] [1046]  INFO RPI raspberrypi.cpp:625 Sensor: /base/soc/i2c0mux/[email protected]/[email protected]
- Selected mode: 1920x1080-pRAA
Camera streams configured
Buffers allocated and mapped
Video setup complete
Requests created
[2:54:56.331913012] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: Sharpness = 1.000000
[2:54:56.332288739] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: Saturation = 1.000000
[2:54:56.332438634] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: Contrast = 1.000000
[2:54:56.332596862] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: Brightness = 0.000000
[2:54:56.332724726] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: FrameDurationLimits = [ 33
333, 33333 ]
[2:54:56.332984985] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: NoiseReductionMode = 1
[2:54:56.333302067] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: AeMeteringMode = 0
[2:54:56.333698314] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: AeExposureMode = 0
[2:54:56.333874095] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: ExposureValue = 0.000000
[2:54:56.334065343] [1050]  INFO IPARPI raspberrypi.cpp:637 Request ctrl: AwbMode = 0
[2:54:56.343128522] [1046]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[2:54:56.343304354] [1046]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
Camera started!

After that… nothing…forever… And the output file test.h264 remains empty.
I tried producing mjpeg instead and that works:

$ libcamera-vid -t 10000 --width 1920 --height 1080 --codec mjpeg -n -v -o test.mjpeg
[... lots of line omitted ...]
[2:49:43.503900436] [985]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (1
 left)
[2:49:43.530779400] [985]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (0
 left)
Viewfinder frame 0
Viewfinder frame 1
Viewfinder frame 2
Viewfinder frame 3
Viewfinder frame 4
Viewfinder frame 5
FileOutput: opened output file test.mjpeg
FileOutput: output buffer 0x732df008 size 96449
FileOutput: output buffer 0x7329e008 size 165940
Viewfinder frame 6
FileOutput: output buffer 0x727243d8 size 99164
[... etc etc]

The video stuff in my /boot/config.txt are:

[all]
start_x=1
gpu_mem=128
dtoverlay=arducam
dtoverlay=vc4-fkms-v3d
disable_touchscreen=1

Hi @tve
This is a bit weird.
I’m not sure about the specific reason.
Can you try to set a smaller resolution?
For example: -w 1280 -h 720

Aha, so 1280x720 doesn’t work either, but 640x480 does work. Libcamera uses the same full-frame src, it seems:

[8:17:45.554113609] [1403]  WARN V4L2 v4l2_pixelformat.cpp:181 Unsupported V4L2 pixel format XB24
[8:17:45.555798962] [1403]  INFO Camera camera.cpp:906 configuring streams: (0) 640x480-YUV420
[8:17:45.556837184] [1408]  INFO RPI raspberrypi.cpp:128 Mode: 1920x1080 fmt RG10 Score: 2136.67 (be
st 2136.67)
[8:17:45.557063849] [1408]  INFO RPI raspberrypi.cpp:128 Mode: 1920x1080 fmt pRAA Score: 1636.67 (be
st 1636.67)
[8:17:45.626653161] [1408]  INFO RPI raspberrypi.cpp:625 Sensor: /base/soc/i2c0mux/[email protected]/[email protected]
- Selected mode: 1920x1080-pRAA

but then it actually does process frames:

Camera started!
[8:17:46.147852648] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (7 left)
[8:17:46.173763251] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (6 left)
[8:17:46.201737902] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (
5 left)
[8:17:46.226070183] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (4 left)
[8:17:46.251561414] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (3 left)
[8:17:46.276058797] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (2 left)
[8:17:46.300838835] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (1 left)
[8:17:46.326424128] [1408]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (0 left)
Viewfinder frame 0
FileOutput: opened output file test.h264
FileOutput: output buffer 0x75487000 size 18974
Viewfinder frame 1
FileOutput: output buffer 0x75407000 size 13562
Viewfinder frame 2
FileOutput: output buffer 0x75387000 size 7745
Viewfinder frame 3
FileOutput: output buffer 0x75307000 size 18700
etc...

1024x768 works as well…
This is what I have in terms of reserved memory:

$ egrep mem /boot/config.txt
gpu_mem=128
$ dmesg|egrep cma
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Memory: 109728K/393216K available (10240K kernel code, 1312K rwdata, 2948K rodata, 1024K init, 863K bss, 21344K reserved, 262144K cma-reserved)
[    6.816200] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[    6.822058] bcm2835_vc_sm_cma_probe: Videocore shared memory driver

It’s strange, it doesn’t have any related error logs.
Is there any related output in the complete dmesg log?
If not, I think I need to perform some tests.

$ dmesg|tail
[   16.996478] Bluetooth: HCI UART protocol H4 registered
[   16.996574] Bluetooth: HCI UART protocol Three-wire (H5) registered
[   16.996835] Bluetooth: HCI UART protocol Broadcom registered
[   17.289139] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   17.289152] Bluetooth: BNEP filters: protocol multicast
[   17.289169] Bluetooth: BNEP socket layer initialized
[   21.277684] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save disabled
[   33.754466] arducam_vddl: disabling
[   33.754486] arducam_vdig: disabling
[   33.754507] arducam_vana: disabling

i.e., nothing here since boot… I have a rpi 3B (+?) that I can test with as well…

If you can use 3B to test it will be the best, which can confirm whether it is related to the hardware.

Some progress. I had forgotten the 3A+ only has 512MB memory, so I tweaked the config.txt to reduce gpu_mem to 64MB and disable framebuffers. Now 1280x1024 works and 1920x1080 sometimes works and sometimes doesn’t. Here’s top (it doesn’t look different when it works vs. doesn’t):

top - 20:35:13 up 11 min,  3 users,  load average: 0.08, 0.06, 0.01
Tasks: 115 total,   1 running, 114 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.2 sy,  0.0 ni, 99.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :    427.6 total,    176.0 free,    105.4 used,    146.1 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.    269.6 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  733 tve       20   0  108588  13872  12724 S   0.0   3.2   0:00.47 libcamera-vid

Do you have any suggestions for figuring out what specifically is failing?

Any idea on troubleshooting this?

I have the same issue on a rPi 3B w/1GB memory using gstreamer. The following works:

gst-launch-1.0 -v libcamerasrc ! video/x-raw,format=NV21,width=640,height=480,framerate=4/1 ! \
  filesink location=test.nv21

But this fails, i.e., it just hangs:

gst-launch-1.0 -v libcamerasrc ! video/x-raw,format=NV21,width=640,height=480,framerate=4/1 ! \
  v4l2h264enc ! filesink location=test.h264

Dmesg shows no error. Syslog shows no error. Gstream shows no unusual error (just all the noise from libcamera). So the gstreamer output ends with

...
[0:06:40.467781955] [680]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (2 left)
[0:06:40.506089521] [680]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (1 left)
[0:06:40.539310783] [680]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (0 left)

I.e. just the same as where libcamera-vid ended

Today I will conduct some tests, and I am not yet sure about the cause of the problem.

1 Like

Use the following command to make it output normally, but the output h264 file is not correct, even if I use videotestsrc it is the same, this may be a problem with the encoding component.

 gst-launch-1.0 -v libcamerasrc ! video/x-raw,format=NV21,width=640,height=480 ! videoconvert ! tee ! v4l2h264enc ! filesink location=test.h264

videotestsrc

gst-launch-1.0 -v videotestsrc ! video/x-raw,format=NV21,width=640,height=480,framerate=4/1 !  v4l2h264enc ! filesink location=test.h264

Use the RGB format to make it work properly:

gst-launch-1.0 -v libcamerasrc ! video/x-raw,format=RGB,width=640,height=480 ! videoconvert ! tee ! v4l2h264enc ! filesink location=test.h264

But this is using software coding, and the coding speed is slow.

In your first example (gst-launch-1.0 -v libcamerasrc ! video/x-raw,format=NV21,width=640,height=480 ! videoconvert ! tee ! v4l2h264enc !…) you use videoconvert, which means software processing of the image stream. THat doesn’t really work… (Maybe for 640x480, but that’s not the goal here, I’m just using that video size to avoid hitting obscure memory issues while I’m trying to get a HW pipeline to work.)

Your second example also uses videoconvert and must be similarly slow as molasses…

I’m not sure why it doesn’t work when I don’t use videoconvert, maybe I can submit an issue to libcamera, but I can’t fix it. .

Unfortunately, I cannot find 3A+ for testing temporarily, but you should not need to set gpu_mem, you can try to comment out start_x=1 and gpu_mem=xxx.

[all]
# start_x=1
# gpu_mem=128
dtoverlay=arducam
dtoverlay=vc4-fkms-v3d
disable_touchscreen=1

maybe I can submit an issue to libcamera, but I can’t fix it. .

That would be awesome!

I cannot find 3A+ for testing temporarily

On which rPi model does it work for you? Any?

I mainly test on Pi4B and Pi3B

I have a Pi 3B here and it’s not working any better. The last examples I posted were from the 3B… Can you show a commandline that produces h264 all in HW with gstreamer on the 3B?

Using libcamera-vid on Pi3B can work normally.

So I did a bunch more testing… and it turns out I have a bad camera.

I had purchased two imx462 cameras (from uctronics) and had both assembled. Then, unbeknownst to me, depending on which camera I happened to connect to the rPi in the morning things would work or not. So on friday I was lucky and connected the good camera and got libcamera-vid piped into gst-launch to work nicely. I then proceeded to configure a second identical rPi for the other camera and I just couldn’t get it to work. I thought I installed the software incorrectly, and spent hours troubleshooting, etc, etc.

Today I finally swapped cameras between the rPis and discovered that one camera works and the other doesn’t. Then I stripped everything down and swapped one part at a time: power supply, SFC cables, adapter board, etc). The bottom line is that one of the UC-755 camera boards works and the other doesn’t. The camera initializes fine, libcamera-still -t 5000 -o test.jpg works, but libcamera-vid at full resolution just hangs there and never produces data. Specifically:

$ libcamera-vid -t 0000 --width 1920 --height 1080 --framerate 5 -n --codec h264 --pro
file high --intra 20 -o test.h264
[0:01:11.789053510] [723]  INFO Camera camera_manager.cpp:296 libcamera v0.0.0+2736-579f55b1-dirty (2021-07-01T03:47:30-04:00)
...
[0:01:12.588632181] [727]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[0:01:12.588727029] [727]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[...libcamera-vid hangs here...]

With the good camera, the output continues with:

[0:00:31.139760820] [560]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (7
 left)
[0:00:31.165444934] [560]  INFO RPI raspberrypi.cpp:1671 Dropping frame at the request of the IPA (6
 left)
[etc.]

I suppose I need to go through uctronics to get the bad camera replaced?

Hi @tve

Sorry for your bad manager,
Can you take a detailed photo of a non-working UC-755?
All modules should be tested before they leave the factory, and we want to confirm whether there is any obvious damage on the modules.