IMX519 takes black&white photos with horizontal lines

javinair · April 25, 2023, 2:36pm

Hello guys!

I’m having the following problem and I hope you have any clue about what’s happening. The device I have is an Arducam 16Mpx (IMX519) from Kickstarter campaign, using a Raspberry PI3B.

First of all, these are my outputs:

uname -a

Linux krvkweather 5.15.61-v7+ #1579 SMP Fri Aug 26 11:10:59 BST 2022 armv7l GNU/Linux

cat /etc/os-release

PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

cat /proc/meminfo

MemTotal:         944268 kB
MemFree:          685824 kB
MemAvailable:     806184 kB
Buffers:           35536 kB
Cached:           123500 kB
SwapCached:            0 kB
Active:            95192 kB
Inactive:         112092 kB
Active(anon):        756 kB
Inactive(anon):    48540 kB
Active(file):      94436 kB
Inactive(file):    63552 kB
Unevictable:          16 kB
Mlocked:              16 kB
SwapTotal:        102396 kB
SwapFree:         102396 kB
Dirty:                52 kB
Writeback:             0 kB
AnonPages:         48308 kB
Mapped:            41804 kB
Shmem:              1024 kB
KReclaimable:      14856 kB
Slab:              30452 kB
SReclaimable:      14856 kB
SUnreclaim:        15596 kB
KernelStack:        1168 kB
PageTables:         1440 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      574528 kB
Committed_AS:     280188 kB
VmallocTotal:    1114112 kB
VmallocUsed:        5812 kB
VmallocChunk:          0 kB
Percpu:              400 kB
CmaTotal:         262144 kB
CmaFree:          258264 kB

cat /boot/config.txt

# For more options and information see
# http://rpf.io/configtxt
# Some settings may impact device functionality. See link above for details

# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border
#overscan_left=16
#overscan_right=16
#overscan_top=16
#overscan_bottom=16

# uncomment to force a console size. By default it will be display's size minus
# overscan.
#framebuffer_width=1280
#framebuffer_height=720

# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1

# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4

# uncomment for composite PAL
#sdtv_mode=2

#uncomment to overclock the arm. 700 MHz is the default.
#arm_freq=800

# Uncomment some or all of these to enable the optional hardware interfaces
dtparam=i2c_arm=on
#dtparam=i2s=on
dtparam=spi=on

# Uncomment this to enable infrared communication.
#dtoverlay=gpio-ir,gpio_pin=17
#dtoverlay=gpio-ir-tx,gpio_pin=18

# Additional overlays and parameters are documented /boot/overlays/README

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

# Automatically load overlays for detected cameras
camera_auto_detect=1

# Automatically load overlays for detected DSI displays
display_auto_detect=1

# Enable DRM VC4 V3D driver
dtoverlay=vc4-kms-v3d
#dtoverlay=arducam-pivariety
max_framebuffers=2

# Disable compensation for displays with overscan
disable_overscan=1

[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1

[all]

[pi4]
# Run as fast as firmware / board allows
arm_boost=1

[all]
dtoverlay=imx519
dtparam=i2c1=on
dtoverlay=pi3-miniuart-bt
dtoverlay=miniuart-bt
core_freq=250

./install_pivariety_pkgs.sh -l

=================================================
Hardware Revision: a02082
Kernel Version: 5.15.61-v7+
OS Codename: bullseye
ARCH: armv7l
=================================================

libcamera-hello --list-cameras

Available cameras
-----------------
0 : imx519 [4656x3496] (/base/soc/i2c0mux/i2c@1/imx519@1a)
    Modes: 'SRGGB10_CSI2P' : 1280x720 [120.02 fps - (1048, 1042)/2560x1440 crop]
                             1920x1080 [60.00 fps - (408, 674)/3840x2160 crop]
                             2328x1748 [30.00 fps - (0, 0)/4656x3496 crop]
                             3840x2160 [21.01 fps - (408, 672)/3840x2160 crop]
                             4656x3496 [10.00 fps - (0, 0)/4656x3496 crop]

What’s the problem?
I’m running a Telegram bot that takes photos on demand with the following command:
/usr/bin/libcamera-still -n --autofocus-on-capture -o test.jpg

Sometimes photos are as expected, nice; but sometimes it takes photos like the second one:

I’m trying also to perform a timelapse (2 photos per minute during 1 hour) and this is the result: timelapse - YouTube

I updated yesterday the pivariety_pkgs and reinstalled imx519_slow_driver, libcamera and libcamera_dev.
I’m afraid to move to a newer libs/kernel/distro version.

If you need further information, do not hesitate to request it.

Thanks in advance.

Edward · April 26, 2023, 1:05am

@javinair

Can you provide the full script?
Or are you executing the same commands over and over with a robot?

javinair · April 26, 2023, 5:27am

Hello @Edward.
I interact with my script through a Telegram bot. For example, if I want to take a picture, I send the /takephoto command and it does the following (takes the photo and sends it back to me):

def takePhoto(update, context):
    dt = datetime.now()
    ts = int(datetime.timestamp(dt))
    filename = str(ts)
    test_photo.take_photo(filename)
    context.bot.send_photo(
        update.message.chat_id, photo=open(f"{filename}.jpg", "rb")
    )

def loadFocus():
    focuser = Focuser("/dev/v4l-subdev1")
    focuser.set(Focuser.OPT_FOCUS, data['focus'])

def load_camera_config():
    global data
    f = open("current_config.json")
    data = json.load(f)
    loadFocus()

def take_photo(filename):
    load_camera_config()
    parameters = [
        "/usr/bin/libcamera-still",
        "--width",
        str(data["width"]),
        "--height",
        str(data["height"]),
        "autofocus-on-capture",
        "-o",
        f"{filename}.jpg".format(filename=filename)
    ]
    launch_command(parameters)


def launch_command(parameters):
    subprocess.run(
        parameters,
        stdout=subprocess.DEVNULL, 
        stderr=subprocess.STDOUT)

I started to use the Focuser utility in order to focus manually (I have a headless configuration and I didn’t know how to focus properly in that version).

Repeating the same sequence, sometimes photos are as expected; sometimes are b&w with those lines.

The problem is not related with the bot itself because I can see the picture before it is sent to Telegram and if it has artifacts, these are present there.

Thanks.

Edward · April 26, 2023, 7:18am

@javinair

I’ll try to save a picture for a day and see if it goes wrong

javinair · April 26, 2023, 7:30am

Thanks my friend!
As I said, it does it randomly. Maybe I repeat the same request 20 times and some of them are with problems.
It’s really weird

javinair · April 26, 2023, 7:50am

@Edward
Check this new timelapse I’ve just taken. A photo every 30 seconds during 2 hours.

Thanks.

Edward · April 26, 2023, 8:13am

@javinair

Fine, I am testing it.

Tomorrow i will give you some results.

Edward · April 26, 2023, 8:25am

@javinair

I want to ask again, what resolution did you use?

javinair · April 26, 2023, 8:44am

@Edward
For these last tests: 3280x2464. In fact, I see this resolution is not present on the available camera resolutions on my first post, but I’m pretty sure I’ve been using this resolution for any previous reason.

libcamera-hello --list-cameras

Available cameras
-----------------
0 : imx519 [4656x3496] (/base/soc/i2c0mux/i2c@1/imx519@1a)
    Modes: 'SRGGB10_CSI2P' : 1280x720 [120.02 fps - (1048, 1042)/2560x1440 crop]
                             1920x1080 [60.00 fps - (408, 674)/3840x2160 crop]
                             2328x1748 [30.00 fps - (0, 0)/4656x3496 crop]
                             3840x2160 [21.01 fps - (408, 672)/3840x2160 crop]
                             4656x3496 [10.00 fps - (0, 0)/4656x3496 crop]

Meanwhile, I’m going to do also some tests at FHD resolution (1920x1080).

The timelapse videos uploaded to Youtube are 640x480, and I join the sources like this:
ffmpeg -framerate 30 -pattern_type glob -i "timelapse*.jpg" -s:v 640x480 -c:v libx264 -crf 17 -pix_fmt yuv420p timelapse.mp4

As I said before, some of these .jpg are with error, so the problem is not related with the timelapse creation.

Thanks.

Edward · April 26, 2023, 8:59am

@javinair

This resolution is actually cropped from the highest resolution. So no problem.
I have written a script and am saving it all the time. I haven’t found the problem you mentioned so far. Maybe I should run it for a longer time.

#!/bin/bash

while true
do
    libcamera-still -o $(date +"%Y%m%d_%H%M%S").jpg --autofocus-on-capture -n
    sleep 1
done

javinair · April 26, 2023, 10:06am

@Edward
Ok, meanwhile I’m doing some other testing too (single photos and 2h timelapse) at FHD resolution.
I’ll tell you later results.

Thanks.

javinair · April 26, 2023, 3:33pm

Hello again @Edward

I’ve been doing some tests, allí of them using 1920x1080, photos and timelapse vídeo.

As you can see, it seems to work flawless, except the focus part.
With the libcamera versión that I’m using, I’m not able to use the --autofocus parameter. Even using the --autofocus-on-capture seems to don’t be done all the time.
Is there any way to improve the focus?

Thank you so much.

Some frames of the video seems to have some kind of color glitch. I suppose it is because of the automatic white balance (AWB).

Edward · April 27, 2023, 2:34am

@javinair

I tested it for one night and found no bad image problems.

–autofocus-on-capture will focus once before saving the image each time, which is the best focusing solution. If you just want to record distance, you can try fixed focus.

Are you referring to those smoke-like things?

javinair · April 27, 2023, 5:51am

Thanks @Edward

All the tests I’ve done at 1920x1080 went successful

These things I suppose are related with direct Sun reflection on the lens. I’m talking about the “color temperature” of some frames. I think it’s because of the AWB and the lens changes its configuration.

javinair · April 27, 2023, 12:17pm

I’m back again @Edward …

This morning I’ve taken a timelapse using the following command (a photo every 30s during 4h):
/usr/bin/libcamera-still -t 14400000 --timelapse 30000 --framestart 1 --width 1920 --height 1080 --autofocus-on-capture --awb cloudy -o timelapse%04d.jpg

And creating the video with this command:
ffmpeg -y -framerate 30 -pattern_type glob -i timelapse*.jpg -s:v 1920x1080 -c:v libx264 -crf timelapse.mp4

And this is the result:

I have checked several frames before the video and some of them have also the following problems:

Focus seems to be playing different on each frame. (Is there a way to keep focus set to infinite?)
Some frames have artifacts. They are totally grey, or some part of them.

Edward · April 28, 2023, 3:29am

@javinair

Yes, you can fix the focal length.
libcamera has –lens-position parameter, which can be set.

I also conducted an outdoor test. I tested it for a full day, but I still didn’t find any abnormal images. I was a little confused.

I want to run some more tests.

But I have a 5-day vacation below, and I may have to reply to you next week.

javinair · April 28, 2023, 5:41am

Ok @Edward, no worries.
I’m going to perform some tests too with the --lens-position parameter.

Enjoy your holidays!

Edward · May 5, 2023, 6:12am

@javinair

I’m back. I organized a meeting about your problem, we think that the broken picture may be due to instability, can you take a picture of the full connection status of your camera? We also want to know the length of the flex cable.

javinair · May 10, 2023, 8:15am

Hello again @Edward, sorry for the late reply.

I’ve been using the regular flex cable that came with the camera. I unplugged/plugged it again, cleaning contacts (isopropyl alcohol) and blowing connectors (compressed air can).

In the end, I decided to buy a new camera. I expected there is some problem with the camera itself… Checking that you tried it with the same configuration as me and it worked perfectly…

Thank you so much for your attention.