16MP Autofocus Camera - cannot control focus and high cpu usage on mjpg-streamer-experimental

Raspberry Pi Cameras
,
1

  1. Where did you get the camera module(s)?
    Kickstarter

  2. Model number of the product(s)?
    IMX519 AF Camera

  3. What hardware/platform were you working on?
    Raspberry Pi 3 Model B Rev 1.2

  4. Instructions you have followed. (link/manual/etc.)
    Official arducam documentation for driver and utilities

  5. Problems you were having?
    I am using the recently released IMX519 AF camera on a pi 3B. My use case is headless as a camera for klipper (a pi controlled 3d printer firmware). I followed the instructions on the product page, and I have the driver installed, and the arducam fork of jpg-streamer installed. The basic setup is working, but mjpg-streamer hits a constant 100% cpu and over time leads the system to a thermal throttle. I am trying to troubleshoot this, and I am noticing some quirks from every other install out there. These are a few issues I see:

  6. Every blog/tutorial refers to /dev/v4l-subdev1 as the subdevice to control focus, but I don’t have that on my system. What I have is /dev/v4l-subdev0, and while it exposes several camera parameters, it does not show any focus settings

  7. I cannot get libcamera-vid to work at all to stream the video. Every attempt ends in ERROR: *** failed to allocate capture buffers ***, and I have tried a few combinations of the command.

  8. The dmesg log from your hardware?

pi@ender3:~ $ dmesg | grep -E 'vc4|bcm|imx'
[    0.000000] Kernel command line: coherent_pool=1M 8250.nr_uarts=1 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 video=Composite-1:720x480@60i,margin_left=32,margin_right=32,margin_top=32,margin_bottom=32 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=tty1 root=PARTUUID=36a7c549-02 rootfstype=ext4 fsck.repair=yes rootwait
[    0.061645] bcm2835-mbox 3f00b880.mailbox: mailbox enabled
[    0.154397] bcm2835-dma 3f007000.dma: DMA legacy API manager, dmachans=0x1
[    1.980227] bcm2835-rng 3f104000.rng: hwrng registered
[    1.990003] gpiomem-bcm2835 3f200000.gpiomem: Initialised: Registers at 0x3f200000
[    3.135366] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    3.138286] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    3.142636] mmc-bcm2835 3f300000.mmcnr: mmc_debug:0 mmc_debug2:0
[    3.145179] mmc-bcm2835 3f300000.mmcnr: DMA channel allocated
[    3.284418] mmc0: sdhost-bcm2835 loaded - DMA enabled (>1)
[    8.176694] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
[    8.284747] snd_bcm2835: module is from the staging directory, the quality is unknown, you have been warned.
[    8.295103] bcm2835_audio bcm2835_audio: card created with 8 channels
[    8.303791] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[    8.336744] bcm2835_isp: module is from the staging directory, the quality is unknown, you have been warned.
[    8.345347] bcm2835-isp bcm2835-isp: Device node output[0] registered as /dev/video13
[    8.346035] bcm2835-isp bcm2835-isp: Device node capture[0] registered as /dev/video14
[    8.346595] bcm2835-isp bcm2835-isp: Device node capture[1] registered as /dev/video15
[    8.347074] bcm2835-isp bcm2835-isp: Device node stats[2] registered as /dev/video16
[    8.347108] bcm2835-isp bcm2835-isp: Register output node 0 with media controller
[    8.347136] bcm2835-isp bcm2835-isp: Register capture node 1 with media controller
[    8.347158] bcm2835-isp bcm2835-isp: Register capture node 2 with media controller
[    8.347180] bcm2835-isp bcm2835-isp: Register capture node 3 with media controller
[    8.347477] bcm2835_v4l2: module is from the staging directory, the quality is unknown, you have been warned.
[    8.376268] bcm2835-isp bcm2835-isp: Device node output[0] registered as /dev/video20
[    8.403864] bcm2835_codec: module is from the staging directory, the quality is unknown, you have been warned.
[    8.441129] bcm2835-isp bcm2835-isp: Device node capture[0] registered as /dev/video21
[    8.478121] bcm2835-isp bcm2835-isp: Device node capture[1] registered as /dev/video22
[    8.480740] bcm2835-isp bcm2835-isp: Device node stats[2] registered as /dev/video23
[    8.480795] bcm2835-isp bcm2835-isp: Register output node 0 with media controller
[    8.480824] bcm2835-isp bcm2835-isp: Register capture node 1 with media controller
[    8.480847] bcm2835-isp bcm2835-isp: Register capture node 2 with media controller
[    8.480893] bcm2835-isp bcm2835-isp: Register capture node 3 with media controller
[    8.481319] bcm2835-isp bcm2835-isp: Loaded V4L2 bcm2835-isp
[    8.512988] bcm2835-codec bcm2835-codec: Device registered as /dev/video10
[    8.513067] bcm2835-codec bcm2835-codec: Loaded V4L2 decode
[    8.521099] bcm2835-codec bcm2835-codec: Device registered as /dev/video11
[    8.521160] bcm2835-codec bcm2835-codec: Loaded V4L2 encode
[    8.531661] bcm2835-codec bcm2835-codec: Device registered as /dev/video12
[    8.531723] bcm2835-codec bcm2835-codec: Loaded V4L2 isp
[    8.534573] bcm2835-codec bcm2835-codec: Device registered as /dev/video18
[    8.534634] bcm2835-codec bcm2835-codec: Loaded V4L2 image_fx
[    8.537697] bcm2835-codec bcm2835-codec: Device registered as /dev/video31
[    8.537765] bcm2835-codec bcm2835-codec: Loaded V4L2 encode_image
[    9.135505] fb0: switching to vc4 from simple
[    9.142059] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[    9.148044] rc rc0: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0
[    9.148376] input: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0/input0
[    9.467312] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[    9.495082] rc rc0: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0
[    9.495361] input: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0/input1
[    9.610710] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[    9.631660] rc rc0: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0
[    9.631946] input: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0/input2
[   10.460383] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[   10.470034] rc rc0: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0
[   10.470345] input: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0/input3
[   10.588272] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[   10.592013] rc rc0: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0
[   10.592317] input: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0/input4
[   10.599016] vc4-drm soc:gpu: bound 3f902000.hdmi (ops vc4_hdmi_ops [vc4])
[   10.599601] vc4-drm soc:gpu: bound 3f004000.txp (ops vc4_txp_ops [vc4])
[   10.600070] vc4-drm soc:gpu: bound 3f206000.pixelvalve (ops vc4_crtc_ops [vc4])
[   10.601685] vc4-drm soc:gpu: bound 3f207000.pixelvalve (ops vc4_crtc_ops [vc4])
[   10.602352] vc4-drm soc:gpu: bound 3f807000.pixelvalve (ops vc4_crtc_ops [vc4])
[   10.602838] vc4-drm soc:gpu: bound 3fc00000.v3d (ops vc4_v3d_ops [vc4])
[   10.607463] [drm] Initialized vc4 0.0.0 20140616 for soc:gpu on minor 0
[   10.608102] vc4-drm soc:gpu: [drm] Cannot find any crtc or sizes
[   10.723578] imx519 10-001a: Device found is imx519
[   10.724136] imx519 10-001a: Consider updating driver imx519 to match on endpoints
[   20.951370] vc4-drm soc:gpu: [drm] Cannot find any crtc or sizes
  1. Troubleshooting attempts you’ve made?
    I have checked that the driver is loaded, and the camera is active. This is the state of my system
Hardware	: BCM2835
Revision	: a02082
Serial		: 00000000991463c0
Model		: Raspberry Pi 3 Model B Rev 1.2

pi@ender3:~ $ grep -vE '^$|^#' /boot/config.txt
dtparam=spi=on
dtparam=audio=on
[pi4]
dtoverlay=vc4-fkms-v3d
max_framebuffers=2
gpu_mem=256
[all]
dtoverlay=vc4-kms-v3d,cma-128
dtoverlay=imx519
enable_uart=1
dtoverlay=disable-bt
start_x=1
[pi0w]
dtoverlay=pi3-disable-bt
gpu_mem=128
[pi02]
gpu_mem=128
dtoverlay=imx519

pi@ender3:~ $ cat /etc/debian_version
11.3

pi@ender3:~ $ v4l2-ctl -d /dev/v4l-subdev0 -L

User Controls

                       exposure 0x00980911 (int)    : min=1 max=3248 step=1 default=788 value=1968
                horizontal_flip 0x00980914 (bool)   : default=0 value=0 flags=modify-layout
                  vertical_flip 0x00980915 (bool)   : default=0 value=1 flags=modify-layout

Camera Controls

             camera_orientation 0x009a0922 (menu)   : min=0 max=2 default=2 value=2 flags=read-only
				0: Front
				1: Back
				2: External
         camera_sensor_rotation 0x009a0923 (int)    : min=0 max=0 step=1 default=0 value=0 flags=read-only

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=100 max=8383280 step=1 default=100 value=2560
            horizontal_blanking 0x009e0902 (int)    : min=5691 max=5691 step=1 default=5691 value=5691 flags=read-only
                  analogue_gain 0x009e0903 (int)    : min=0 max=960 step=1 default=0 value=0
                red_pixel_value 0x009e0904 (int)    : min=0 max=4095 step=1 default=4095 value=4095
          green_red_pixel_value 0x009e0905 (int)    : min=0 max=4095 step=1 default=4095 value=4095
               blue_pixel_value 0x009e0906 (int)    : min=0 max=4095 step=1 default=4095 value=4095
         green_blue_pixel_value 0x009e0907 (int)    : min=0 max=4095 step=1 default=4095 value=4095

Image Processing Controls

                     pixel_rate 0x009f0902 (int64)  : min=686000000 max=686000000 step=1 default=686000000 value=686000000 flags=read-only
                   test_pattern 0x009f0903 (menu)   : min=0 max=4 default=0 value=0
				0: Disabled
				1: Color Bars
				2: Solid Color
				3: Grey Color Bars
				4: PN9
                   digital_gain 0x009f0905 (int)    : min=256 max=65535 step=1 default=256 value=256

pi@ender3:~ $ ls -l /dev/video* /dev/v4l-* /dev/v4l/by-path/*
lrwxrwxrwx 1 root root      12 Apr  3 20:11 /dev/v4l/by-path/platform-3f801000.csi-video-index0 -> ../../video0
lrwxrwxrwx 1 root root      12 Apr  3 20:11 /dev/v4l/by-path/platform-3f801000.csi-video-index1 -> ../../video1
lrwxrwxrwx 1 root root      13 Apr  3 20:11 /dev/v4l/by-path/platform-bcm2835-codec-video-index0 -> ../../video18
lrwxrwxrwx 1 root root      13 Apr  3 20:11 /dev/v4l/by-path/platform-bcm2835-isp-video-index0 -> ../../video13
lrwxrwxrwx 1 root root      13 Apr  3 20:11 /dev/v4l/by-path/platform-bcm2835-isp-video-index1 -> ../../video21
lrwxrwxrwx 1 root root      13 Apr  3 20:11 /dev/v4l/by-path/platform-bcm2835-isp-video-index2 -> ../../video22
lrwxrwxrwx 1 root root      13 Apr  3 20:11 /dev/v4l/by-path/platform-bcm2835-isp-video-index3 -> ../../video23
crw-rw---- 1 root video 81, 15 Apr  3 20:11 /dev/v4l-subdev0
crw-rw---- 1 root video 81, 13 Apr  3 20:11 /dev/video0
crw-rw---- 1 root video 81, 14 Apr  3 20:11 /dev/video1
crw-rw---- 1 root video 81,  8 Apr  3 20:11 /dev/video10
crw-rw---- 1 root video 81,  9 Apr  3 20:11 /dev/video11
crw-rw---- 1 root video 81, 10 Apr  3 20:11 /dev/video12
crw-rw---- 1 root video 81,  0 Apr  3 20:11 /dev/video13
crw-rw---- 1 root video 81,  1 Apr  3 20:11 /dev/video14
crw-rw---- 1 root video 81,  2 Apr  3 20:11 /dev/video15
crw-rw---- 1 root video 81,  3 Apr  3 20:11 /dev/video16
crw-rw---- 1 root video 81, 11 Apr  3 20:11 /dev/video18
crw-rw---- 1 root video 81,  4 Apr  3 20:11 /dev/video20
crw-rw---- 1 root video 81,  5 Apr  3 20:11 /dev/video21
crw-rw---- 1 root video 81,  6 Apr  3 20:11 /dev/video22
crw-rw---- 1 root video 81,  7 Apr  3 20:11 /dev/video23
crw-rw---- 1 root video 81, 12 Apr  3 20:11 /dev/video31
pi@ender3:~ $ uname -a
Linux ender3 5.15.32-v7+ #1539 SMP Fri Apr 1 11:38:09 BST 2022 armv7l GNU/Linux
  1. What help do you need?
    I have a feeling my setup is missing a driver or a configuration settings, but I am not sure what. This is on a 32bit bullseye system.
2

libcamera-vid streaming error:

pi@ender3:~ $ libcamera-vid -t 0 --inline --listen -o tcp://0.0.0.0:8080 --codec h264 -n
[41:06:35.844498022] [24869]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0
[41:06:35.855826787] [24873]  WARN CameraSensorProperties camera_sensor_properties.cpp:141 No static properties available for 'imx519'
[41:06:35.855967168] [24873]  WARN CameraSensorProperties camera_sensor_properties.cpp:143 Please consider updating the camera sensor properties database
[41:06:35.856085002] [24873] ERROR CameraSensor camera_sensor.cpp:551 'imx519 10-001a': Camera sensor does not support test pattern modes.
[41:06:35.926585486] [24873]  WARN RPI raspberrypi.cpp:1233 Mismatch between Unicam and CamHelper for embedded data usage!
[41:06:35.927895827] [24873] ERROR DelayedControls delayed_controls.cpp:87 Delay request for control id 0x009a090a but control is not exposed by device /dev/v4l-subdev0
[41:06:35.928675003] [24873]  INFO RPI raspberrypi.cpp:1356 Registered camera /base/soc/i2c0mux/i2c@1/imx519@1a to Unicam device /dev/media3 and ISP device /dev/media0
[41:06:35.930181388] [24869]  INFO Camera camera.cpp:1028 configuring streams: (0) 640x480-YUV420
[41:06:35.931050905] [24873]  INFO RPI raspberrypi.cpp:751 Sensor: /base/soc/i2c0mux/i2c@1/imx519@1a - Selected sensor format: 1280x720-SRGGB10_1X10 - Selected unicam format: 1280x720-pRAA
[41:06:35.934109600] [24869] ERROR Allocator framebuffer_allocator.cpp:98 Stream is not part of /base/soc/i2c0mux/i2c@1/imx519@1a active configuration
ERROR: *** failed to allocate capture buffers ***

I get the same errors if I try a subset of the command, for example: libcamera-vid -t 0 --inline --listen -o tcp://0.0.0.0:8080

3

For me, ./mjpeg_streamer -o output_http.so -w ./www -i input_libcamera.so consume 1 core for 100% all time. Also Stream in OctoPrint often hang.

1 Like
4

Can you try turning off the preview?
libcamera-vid -t 0 --inline --listen -o tcp://0.0.0.0:8080 -n

5

Now we use software to encode mjpg, which may be the cause of CPU usage… But currently there is no hardware encoding support.

6

I get the same error. I had tried a few variations of the command to see if I can find a winning combination, but every time it fails at the same spot.

pi@ender3:~ $ libcamera-vid -t 0 --inline --listen -o tcp://0.0.0.0:8080 -n
[66:28:47.054357496] [16159]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0
[66:28:47.059919312] [16163]  WARN CameraSensorProperties camera_sensor_properties.cpp:141 No static properties available for 'imx519'
[66:28:47.060060562] [16163]  WARN CameraSensorProperties camera_sensor_properties.cpp:143 Please consider updating the camera sensor properties database
[66:28:47.060125978] [16163] ERROR CameraSensor camera_sensor.cpp:551 'imx519 10-001a': Camera sensor does not support test pattern modes.
[66:28:47.103260875] [16163]  WARN RPI raspberrypi.cpp:1233 Mismatch between Unicam and CamHelper for embedded data usage!
[66:28:47.104393374] [16163] ERROR DelayedControls delayed_controls.cpp:87 Delay request for control id 0x009a090a but control is not exposed by device /dev/v4l-subdev0
[66:28:47.105230821] [16163]  INFO RPI raspberrypi.cpp:1356 Registered camera /base/soc/i2c0mux/i2c@1/imx519@1a to Unicam device /dev/media3 and ISP device /dev/media0
[66:28:47.106623059] [16159]  INFO Camera camera.cpp:1028 configuring streams: (0) 640x480-YUV420
[66:28:47.107709516] [16163]  INFO RPI raspberrypi.cpp:751 Sensor: /base/soc/i2c0mux/i2c@1/imx519@1a - Selected sensor format: 1280x720-SRGGB10_1X10 - Selected unicam format: 1280x720-pRAA
[66:28:47.110497221] [16159] ERROR Allocator framebuffer_allocator.cpp:98 Stream is not part of /base/soc/i2c0mux/i2c@1/imx519@1a active configuration
ERROR: *** failed to allocate capture buffers ***

I’ve tried stracing the command, but I don’t see an obvious place where it fails.

I suspect my problem is partly because I have some misconfiguration somewhere. For instance, as I mentioned in the OP, I have a /dev/v4l-subdev0 and no /dev/v4l-subdev1, and my subdev does not expose the focus controls. This seems different to what everyone else reports. Is it perhaps related?

7

I understand, and to be clear, I am not complaining about the lack of hardware accelerated mjpg encoding. I’d like to get a native x264 stream and ditch mjpeg - h264 is a much better format for streaming and all clients can natively process it.

8

I think an improvement would be to only capture and encode when there is a client to send to. I thought that was what the original mjpegstreamer did with an rpi camera. At least I do not see 100% CPU usage on my other systems running legacy mmal mjpegstreamer (only about 4% when streaming, less when not), only the rpi that is using libcamera seems to use 100%.
I have ended up using this to stream h264 and it only uses about 40% of CPU when there is a connection.

libcamera-vid --autofocus -n -t 0 --width=1280 --height=1024 --framerate=10 --inline --listen -o tcp://0.0.
0.0:1234

But I can’t use a web browser to view the stream I use…

ffplay tcp://192.168.1.123:1234 -f h264 -flags low_delay -probesize 32 -analyzeduration 0 -v error -an -x 128
0 -y 1024 -fast -vf "setpts=N/30"

This gives me minimal latency as well.

9

We tested and didn’t find this problem,
Maybe you can comment out gpu_mem in /boot/config.txt and increase dtoverlay=vc4-kms-v3d,cma-128 appropriately
E.g:
dtoverlay=vc4-kms-v3d,cma-192

10

This is my config now. I removed the sections for pi0 and pi2 since those were not relevant to my hardware:

pi@ender3:~ $ grep -vE '^$|^#' /boot/config.txt
dtparam=spi=on
dtparam=audio=on
[pi4]
dtoverlay=vc4-fkms-v3d
max_framebuffers=2
gpu_mem=256
[all]
dtoverlay=vc4-kms-v3d,cma-192
dtoverlay=imx519
enable_uart=1
dtoverlay=disable-bt
start_x=1

I get more or less the same error as before:

pi@ender3:~ $ libcamera-vid --autofocus -n -t 0 --width=1280 --height=1024 --framerate=10 --inline  --listen -o tcp://0.0.0.0:8080
[0:02:20.863280258] [937]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0
[0:02:20.868737261] [941]  WARN CameraSensorProperties camera_sensor_properties.cpp:141 No static properties available for 'imx519'
[0:02:20.868887987] [941]  WARN CameraSensorProperties camera_sensor_properties.cpp:143 Please consider updating the camera sensor properties database
[0:02:20.868983245] [941] ERROR CameraSensor camera_sensor.cpp:551 'imx519 10-001a': Camera sensor does not support test pattern modes.
[0:02:20.911293000] [941]  WARN RPI raspberrypi.cpp:1233 Mismatch between Unicam and CamHelper for embedded data usage!
[0:02:20.912207769] [941] ERROR DelayedControls delayed_controls.cpp:87 Delay request for control id 0x009a090a but control is not exposed by device /dev/v4l-subdev0
[0:02:20.912823327] [941]  INFO RPI raspberrypi.cpp:1356 Registered camera /base/soc/i2c0mux/i2c@1/imx519@1a to Unicam device /dev/media3 and ISP device /dev/media0
[0:02:20.913918561] [937]  INFO Camera camera.cpp:1028 configuring streams: (0) 1280x1024-YUV420
[0:02:20.914452298] [941]  INFO RPI raspberrypi.cpp:751 Sensor: /base/soc/i2c0mux/i2c@1/imx519@1a - Selected sensor format: 1920x1080-SRGGB10_1X10 - Selected unicam format: 1920x1080-pRAA
[0:02:20.916657089] [937] ERROR Allocator framebuffer_allocator.cpp:98 Stream is not part of /base/soc/i2c0mux/i2c@1/imx519@1a active configuration
ERROR: *** failed to allocate capture buffers ***

I have a longer length cable (75cms) because my mount on the 3d printer requires a longer cable. This cable has worked well for over 100 hours on the picam v2 and it works for capturing video with the arducam fork of mjpg-streamer. I don’t think the cable is the fault, I am bringing it up since it’s probably the only variable that might be different between my setup and your lab.

11

These are from my latest dmesg after a reboot with the new config.txt. After glancing through, I see some errors with the wifi chipset (which should be unrelated), and a couple of instances of

vc4-drm soc:gpu: [drm] Cannot find any crtc or sizes

A quick google search tells me I need vc4-kms-v3d for a 32bit OS, I’ll make this change, reboot and test.

Full dmesg:

[    6.905661] systemd-journald[132]: File /var/log/journal/333c0c782d4679d8674440ae61f83757/system.journal corrupted or uncleanly shut down, renaming and replacing.
[    8.100883] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[    8.103487] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
[    8.103526] [vc_sm_connected_init]: start
[    8.108863] mc: Linux media interface: v0.10
[    8.118962] [vc_sm_connected_init]: installed successfully
[    8.158435] videodev: Linux video capture interface: v2.00
[    8.203071] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[    8.213898] bcm2835_v4l2: module is from the staging directory, the quality is unknown, you have been warned.
[    8.281190] bcm2835_isp: module is from the staging directory, the quality is unknown, you have been warned.
[    8.294637] bcm2835-isp bcm2835-isp: Device node output[0] registered as /dev/video13
[    8.295215] bcm2835-isp bcm2835-isp: Device node capture[0] registered as /dev/video14
[    8.295676] bcm2835-isp bcm2835-isp: Device node capture[1] registered as /dev/video15
[    8.296033] bcm2835-isp bcm2835-isp: Device node stats[2] registered as /dev/video16
[    8.296073] bcm2835-isp bcm2835-isp: Register output node 0 with media controller
[    8.296101] bcm2835-isp bcm2835-isp: Register capture node 1 with media controller
[    8.296124] bcm2835-isp bcm2835-isp: Register capture node 2 with media controller
[    8.296162] bcm2835-isp bcm2835-isp: Register capture node 3 with media controller
[    8.309648] bcm2835_codec: module is from the staging directory, the quality is unknown, you have been warned.
[    8.360892] bcm2835-isp bcm2835-isp: Device node output[0] registered as /dev/video20
[    8.360929] snd_bcm2835: module is from the staging directory, the quality is unknown, you have been warned.
[    8.361506] bcm2835-isp bcm2835-isp: Device node capture[0] registered as /dev/video21
[    8.361938] bcm2835-isp bcm2835-isp: Device node capture[1] registered as /dev/video22
[    8.362283] bcm2835-isp bcm2835-isp: Device node stats[2] registered as /dev/video23
[    8.362319] bcm2835-isp bcm2835-isp: Register output node 0 with media controller
[    8.362348] bcm2835-isp bcm2835-isp: Register capture node 1 with media controller
[    8.362371] bcm2835-isp bcm2835-isp: Register capture node 2 with media controller
[    8.362394] bcm2835-isp bcm2835-isp: Register capture node 3 with media controller
[    8.362633] bcm2835-isp bcm2835-isp: Loaded V4L2 bcm2835-isp
[    8.392103] bcm2835-codec bcm2835-codec: Device registered as /dev/video10
[    8.392417] bcm2835-codec bcm2835-codec: Loaded V4L2 decode
[    8.403980] bcm2835_audio bcm2835_audio: card created with 8 channels
[    8.406254] bcm2835-codec bcm2835-codec: Device registered as /dev/video11
[    8.406314] bcm2835-codec bcm2835-codec: Loaded V4L2 encode
[    8.413144] bcm2835-codec bcm2835-codec: Device registered as /dev/video12
[    8.413206] bcm2835-codec bcm2835-codec: Loaded V4L2 isp
[    8.416159] bcm2835-codec bcm2835-codec: Device registered as /dev/video18
[    8.416221] bcm2835-codec bcm2835-codec: Loaded V4L2 image_fx
[    8.419525] bcm2835-codec bcm2835-codec: Device registered as /dev/video31
[    8.419595] bcm2835-codec bcm2835-codec: Loaded V4L2 encode_image
[    8.698200] i2c i2c-11: Added multiplexed i2c bus 0
[    8.698794] i2c 10-001a: Fixing up cyclic dependency with 3f801000.csi
[    8.699225] i2c i2c-11: Added multiplexed i2c bus 10
[    9.034493] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    9.253803] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    9.282365] cfg80211: loaded regulatory.db is malformed or signature is missing/invalid
[    9.381831] brcmfmac: F1 signature read @0x18000000=0x1541a9a6
[    9.388441] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[    9.389207] usbcore: registered new interface driver brcmfmac
[    9.405324] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43430-sdio.raspberrypi,3-model-b.bin failed with error -2
[    9.545864] smsc95xx 1-1.1:1.0 enxb827eb1463c0: renamed from eth0
[    9.579259] checking generic (3eaf0000 10a800) vs hw (0 ffffffff)
[    9.579298] fb0: switching to vc4 from simple
[    9.580057] Console: switching to colour dummy device 80x30
[    9.589696] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[    9.602249] random: crng init done
[    9.602273] random: 7 urandom warning(s) missed due to ratelimiting
[    9.631834] Registered IR keymap rc-cec
[    9.632095] rc rc0: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0
[    9.632409] input: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0/input0
[    9.675934] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[    9.676132] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[    9.688197] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Jul 19 2021 03:24:18 version 7.45.98 (TOB) (56df937 CY) FWID 01-8e14b897
[   10.535800] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[   10.544302] Registered IR keymap rc-cec
[   10.544600] rc rc0: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0
[   10.544901] input: vc4 as /devices/platform/soc/3f902000.hdmi/rc/rc0/input1
[   10.551046] vc4-drm soc:gpu: bound 3f902000.hdmi (ops vc4_hdmi_ops [vc4])
[   10.551670] vc4-drm soc:gpu: bound 3f004000.txp (ops vc4_txp_ops [vc4])
[   10.552128] vc4-drm soc:gpu: bound 3f206000.pixelvalve (ops vc4_crtc_ops [vc4])
[   10.552592] vc4-drm soc:gpu: bound 3f207000.pixelvalve (ops vc4_crtc_ops [vc4])
[   10.553063] vc4-drm soc:gpu: bound 3f807000.pixelvalve (ops vc4_crtc_ops [vc4])
[   10.553444] vc4-drm soc:gpu: bound 3fc00000.v3d (ops vc4_v3d_ops [vc4])
[   10.621414] [drm] Initialized vc4 0.0.0 20140616 for soc:gpu on minor 0
[   10.622079] vc4-drm soc:gpu: [drm] Cannot find any crtc or sizes
[   10.670157] imx519 10-001a: Device found is imx519
[   10.670755] imx519 10-001a: Consider updating driver imx519 to match on endpoints
[   11.326908] 8021q: 802.1Q VLAN Support v1.8
[   11.588855] Adding 262140k swap on /var/swap.  Priority:-2 extents:1 across:262140k SSFS
[   12.065033] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled
[   13.014231] smsc95xx 1-1.1:1.0 enxb827eb1463c0: hardware isn't capable of remote wakeup
[   13.029768] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled
[   13.162983] smsc95xx 1-1.1:1.0 enxb827eb1463c0: Link is Down
[   13.169502] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save disabled
[   14.291184] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[   16.239050] ICMPv6: process `dhcpcd' is using deprecated sysctl (syscall) net.ipv6.neigh.wlan0.retrans_time - use net.ipv6.neigh.wlan0.retrans_time_ms instead
[   19.076364] uart-pl011 3f201000.serial: no DMA platform data
[   20.952599] vc4-drm soc:gpu: [drm] Cannot find any crtc or sizes
13

Can you provide your SD Card Image download link? We want to restore your settings as much as possible and test them.

14

You might try the GitHub - ayufan-research/camera-streamer: High-performance low-latency camera streamer for Raspberry PI's which seems to claim that it can HW accelerate everything, include JPEG and H264 streams.