How is it possible to get autofocus working along with Picamera2 for IMX519

trezero · September 12, 2022, 12:15am

Where did you get the camera module(s)?
Amazon
Model number of the product(s)?
IMX519
What hardware/platform were you working on?
Raspberry Pi 3B+
Instructions you have followed. (link/manual/etc.)
These are the instructions I followed (most recently)
How to use arducam_64mp & arducam_64mp FAQ](How to use arducam_64mp & arducam_64mp FAQ

This gets Manual Focus to work in Python

[BUG] Raspberry Pi 4: Cannot import name 'Size' or 'ControlType' from libcamera depending on version of libcamera · Issue #218 · raspberrypi/picamera2 · GitHub
This gets everything working in Python EXCEPT Autofocus it seems. With this tutorial complete if I install these
./install_pivariety_pkgs.sh -p libcamera_dev ./install_pivariety_pkgs.sh -p libcamera_apps
Everything else breaks and nothing works

This tutorial gets Autofocus working via command line using libcamera-still -t 3000 -o test.jpg --autofocus
But installing Picamera2 for Python causes this to break and then neither Picamera2 or libcamera work at that point.

What help do I need?
What I really need and the reason I bought this camera, is I need to be able to run this camera in 64-bit Raspberry Pi OS with the ability to use autofocus within a Python application in which I can also use OCR and Tesseract. The goal of my application is to be able to recognize license plates within a range of distances and then generate an audible alert when they are detected. The Autofocus is critical to this application as this is to be mounted on a helmet and to operate while a user is traveling down the street.

I have been trying for weeks to get this camera functioning with Autofocus within any kind of Python application in which I can generate a video, or still frames which I can then send to an OCR application or other object recognition utility to try and recognize. Can someone please provide some instruction of how to get this camera installed and operational with autofocus within Python on a 64-bit install of Raspberry Pi OS?

Some sample output

zero@raspberrypi:~/install/Arducam-Pivariety-V4L2-Driver/focus $ python3 FocuserExample.py 
Traceback (most recent call last):
  File "/home/trezero/install/Arducam-Pivariety-V4L2-Driver/focus/v4l2_utils.py", line 64, in get_device_controls_by_class
    fcntl.ioctl(fd, v4l2.VIDIOC_QUERYCTRL, queryctrl)
OSError: [Errno 25] Inappropriate ioctl for device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/trezero/install/Arducam-Pivariety-V4L2-Driver/focus/FocuserExample.py", line 157, in <module>
    main()
  File "/home/trezero/install/Arducam-Pivariety-V4L2-Driver/focus/FocuserExample.py", line 152, in main
    focuser = Focuser(args.device)
  File "/home/trezero/install/Arducam-Pivariety-V4L2-Driver/focus/Focuser.py", line 40, in __init__
    self.ctrls = v4l2_utils.get_ctrls(self.fd)
  File "/home/trezero/install/Arducam-Pivariety-V4L2-Driver/focus/v4l2_utils.py", line 96, in get_ctrls
    for queryctrl in get_device_controls_by_class(vd, class_):
  File "/home/trezero/install/Arducam-Pivariety-V4L2-Driver/focus/v4l2_utils.py", line 66, in get_device_controls_by_class
    assert e.errno == errno.EINVAL
AssertionError

And another test

trezero@raspberrypi:~/install/Arducam-Pivariety-V4L2-Driver/focus $ libcamera-still -t 1000 -o test.jpg --autofocus
Made X/EGL preview window
[0:15:40.489214996] [1665]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3730-67300b62
[0:15:40.498900250] [1667]  WARN CameraSensorProperties camera_sensor_properties.cpp:174 No static properties available for 'imx519'
[0:15:40.499048428] [1667]  WARN CameraSensorProperties camera_sensor_properties.cpp:176 Please consider updating the camera sensor properties database
[0:15:40.555630621] [1667]  WARN RPI raspberrypi.cpp:1274 Mismatch between Unicam and CamHelper for embedded data usage!
[0:15:40.556914950] [1667] ERROR DelayedControls delayed_controls.cpp:87 Delay request for control id 0x009a090a but control is not exposed by device /dev/v4l-subdev0
[0:15:40.557294535] [1667]  INFO RPI raspberrypi.cpp:1398 Registered camera /base/soc/i2c0mux/i2c@1/imx519@1a to Unicam device /dev/media3 and ISP device /dev/media0
[0:15:40.558873813] [1665]  INFO Camera camera.cpp:1029 configuring streams: (0) 2048x1536-YUV420
[0:15:40.559774755] [1667]  INFO RPI raspberrypi.cpp:763 Sensor: /base/soc/i2c0mux/i2c@1/imx519@1a - Selected sensor format: 2328x1748-SRGGB10_1X10 - Selected unicam format: 2328x1748-pRAA
Segmentation fault

trezero · September 13, 2022, 1:47pm

@Edward any chance you could help here?

trezero · September 14, 2022, 4:32am

@yang Any help? I would really appreciate it

Edward · September 14, 2022, 8:50am

@trezero

Look at your current version of libcamera

dpkg -l | grep libcamera

trezero · September 14, 2022, 11:27pm

0.0.8

Edward · September 15, 2022, 12:32am

@trezero

You should

sudo apt remove libcamera0

Then please try again.

trezero · September 15, 2022, 10:24pm

@Edward

Ok, so I did the following
sudo apt remove libcamera0
wget -O install_pivariety_pkgs.sh https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/install_script/install_pivariety_pkgs.sh
chmod +x install_pivariety_pkgs.sh
./install_pivariety_pkgs.sh -p libcamera_dev
./install_pivariety_pkgs.sh -p libcamera_apps

Updated /boot/config.txt with

# Automatically load overlays for detected cameras
camera_auto_detect=0
dtoverlay=imx519

At this point I can run libcamera-hello and see a live video (without any focus)

Then I do the following:

git clone https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver.git
cd Arducam-Pivariety-V4L2-Driver/focus
python3 FocuserExample.py -d /dev/v4l-subdev0

And I get this result

trezero@raspberrypi:~/Arducam-Pivariety-V4L2-Driver/focus $ python3 FocuserExample.py -d /dev/v4l-subdev0
Traceback (most recent call last):
  File "/home/trezero/Arducam-Pivariety-V4L2-Driver/focus/FocuserExample.py", line 157, in <module>
    main()
  File "/home/trezero/Arducam-Pivariety-V4L2-Driver/focus/FocuserExample.py", line 152, in main
    focuser = Focuser(args.device)
  File "/home/trezero/Arducam-Pivariety-V4L2-Driver/focus/Focuser.py", line 54, in __init__
    raise RuntimeError("Device {} has no focus_absolute control.".format(self.dev))
RuntimeError: Device /dev/v4l-subdev0 has no focus_absolute control.

What am I missing here?

Edward · September 16, 2022, 1:51am

@trezero

What’s your camera SKU number?

and please

./install_pivariety_pkgs.sh -p imx519_low_speed_kernel_driver

If it still doesn’t work, please see below

v4l2-ctl -l -d /dev/v4l-subdev0

trezero · September 16, 2022, 3:15am

Output from the v4l2-ctl command

trezero@raspberrypi:~/install $ v4l2-ctl -l -d /dev/v4l-subdev0

User Controls

                       exposure 0x00980911 (int)    : min=1 max=2444 step=1 default=74 value=2444
                horizontal_flip 0x00980914 (bool)   : default=0 value=0 flags=modify-layout
                  vertical_flip 0x00980915 (bool)   : default=0 value=0 flags=modify-layout

Camera Controls

             camera_orientation 0x009a0922 (menu)   : min=0 max=2 default=2 value=2 flags=read-only
         camera_sensor_rotation 0x009a0923 (int)    : min=0 max=0 step=1 default=0 value=0 flags=read-only

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=728 max=8382252 step=1 default=728 value=728
            horizontal_blanking 0x009e0902 (int)    : min=6906 max=6906 step=1 default=6906 value=6906 flags=read-only
                  analogue_gain 0x009e0903 (int)    : min=0 max=960 step=1 default=0 value=883
                red_pixel_value 0x009e0904 (int)    : min=0 max=4095 step=1 default=4095 value=4095
          green_red_pixel_value 0x009e0905 (int)    : min=0 max=4095 step=1 default=4095 value=4095
               blue_pixel_value 0x009e0906 (int)    : min=0 max=4095 step=1 default=4095 value=4095
         green_blue_pixel_value 0x009e0907 (int)    : min=0 max=4095 step=1 default=4095 value=4095

Image Processing Controls

                     pixel_rate 0x009f0902 (int64)  : min=686000000 max=686000000 step=1 default=686000000 value=686000000 flags=read-only
                   test_pattern 0x009f0903 (menu)   : min=0 max=4 default=0 value=0
                   digital_gain 0x009f0905 (int)    : min=256 max=65535 step=1 default=256 value=256

trezero · September 16, 2022, 3:18am

When I try this script without the AfTrigger it works. With it, I get a Segmentation Fault

#!/usr/bin/python3

from picamera2 import Picamera2

picam2 = Picamera2()

picam2.set_controls({"AfTrigger":0})

# Record a 5 second video.
picam2.start_and_record_video("test.mp4", duration=5)

Any idea how to fix the Segmentation fault?

Edward · September 16, 2022, 3:20am

@trezero

This is caused by the lack of focus control.
Can you take a picture of your camera to show me?

trezero · September 16, 2022, 3:20am

Where do I find the SKU?

Edward · September 16, 2022, 3:20am

You can take a picture of the front of the camera to show me.

trezero · September 16, 2022, 3:26am

Here is my camera

if I dont install anything extra the autofocus always works with just

libcamera-still -t 1000 -o test.jpg --autofocus

But usually as soon as I run the installers to try and get picamera2 working or anything working in Python then I lose autofocus.

trezero · September 16, 2022, 3:28am

My crazy testing process is heavily documented here FYI: Miro | Online Whiteboard for Visual Collaboration

Edward · September 16, 2022, 3:32am

@trezero

You don’t need to install python3-libcamera.
This can lead to conflict.

Please see our tutorial, although this troubleshooting is not perfect.

trezero · September 16, 2022, 3:33am

What should I have in /boot/config.txt?

Edward · September 16, 2022, 3:34am

@trezero

Your configuration environment is correct, no changes are needed.
Your dependency has some conflicts.

This is because focus control is unique to us.
If you have python3-libcamera installed, this will cause you to use the official libcamera, which has no focus control.

You need to delete the python3-libcamera .

trezero · September 16, 2022, 4:15am

trezero@raspberrypi:~ $ sudo apt remove python3-libcamera
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package 'python3-libcamera' is not installed, so not removed
The following packages were automatically installed and are no longer required:
  freeglut3 libfuse2 libglu1-mesa libturbojpeg0 python3-opengl python3-pidng
  python3-piexif python3-simplejpeg python3-v4l2
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

trezero · September 16, 2022, 4:16am

Looks like I did not have it installed this round