How do I get an IMX477 to work with jetson xavier nx?

I’ve been googling a while without success. Surely there’s a how-to somewhere…

Also, which way does the ribbon cable go? Doesn’t seem to be any markings or pin 0 or … ???

THANX!

Hi @wwzeitler ,

This is a tutorial on how to install the driver: https://www.arducam.com/docs/camera-for-jetson-nano/native-jetson-cameras-imx219-imx477/imx477/

The attached picture shows how to connect.

OK, I did the above steps, and no /dev/video*

In general, how would I do a basic test of the camera?

BTW, for anyone else struggling with this, apparently the ribbon orientation is the contacts are towards the motherboard. (You’ll note that on the male end of the ribbon one side has contacts and the other insulation).

@wwzeitler

It requires some basic programming or script skills to install the driver and troubleshooting.

Would you please send the result of each step screenshot in order to help you diagnose the issue.

Or please contact our [email protected] email and date a time for remote debugging.

Thanks.

Lee

Salut,

I followed this guide with the DEB packages option: https://developer.ridgerun.com/wiki/index.php?title=Raspberry_Pi_HQ_camera_IMX477_Linux_driver_for_Jetson#Raspberry_Pi_HQ_camera_IMX477_Linux_driver_for_NVIDIA.C2.AEJetson_Xavier.E2.84.A2_NX

Unfortunately I am not successful in getting anything from my imx477. Can’t express my frustration on this peace of hardware …

 

This is not necessarily a hardware problem, can you run the dmesg command and send the result and hardware connection diagram to us?

At one point I was seeing /dev/video0, now I’m not. I need to retrace my steps to see where it went wrong. I’m carefully chronicling my steps here: https://zeitbytes.com/

I have flashed a fresh SD card with the latest xavier image and only applied the apt-get upgrade. After that, I see the IMX219 errors in dmesg, which is expectable. I then installed the arducam deb package (~8MB) and after the reboot, the dmesg showed imx477 entries and I was able to display the camera on the screen. When connecting the HDMI adapter with the ~15 cm long ribbon cable of the HQ Pi camera, the camera image was lost and only a red screen was visible.

So I can rule out hardware failure, which is good. I now need to add all the steps to move from the SD card to the SSD and to install all the python packages to get my jetson back to normal. I will check after each step that the camera is still working to identify where the camera goes into conflict. This will take some time …

I also replaced the lens of the Arducam by a zoom lens from Waveshare to be able to do close pictures and have a good manual focus.

I hope I will find a way to get the images fromthe camera to the neural network in pytorch.

Ciao, Mathias

minor update: I now am able to see the device under /dev/video0

It took a reboot to get that device and I do not know why it did not show up in the first place.

Ciao, Mathias

Reinstalled everything, /dev/video0 is showing up. Now I’m trying to run a simple opencv app that uses the camera:

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

while(True):
    # Capture frame-by-frame
    ret, frame = cap.read()

    # Our operations on the frame come here
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Display the resulting frame
    cv2.imshow('frame',gray)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

I get:

[ WARN:0] global /mnt/ssd/opencv/opencv-baf07c8/modules/videoio/src/cap_gstreamer.cpp (1761) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Cannot identify device '/dev/video-1'.
[ WARN:0] global /mnt/ssd/opencv/opencv-baf07c8/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /mnt/ssd/opencv/opencv-baf07c8/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[ WARN:0] global /mnt/ssd/opencv/opencv-baf07c8/modules/videoio/src/cap_v4l.cpp (893) open VIDEOIO(V4L2:/dev/video0): can't open camera by index
Traceback (most recent call last):
  File "video.py", line 11, in 
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.4.0) /mnt/ssd/opencv/opencv-baf07c8/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

On my installation, your code is running (not working). All is as expected but the image is empty.

This morning, when restarting the xavier (nothing touched since last run), the image from the camera went all red except for a small image band at the top of the image. I replaced the shipped HDMI cable with a spare and the image is back (using the gst stream pipeline, not cv2)

Very unreliable setup and a camera far from being usable.

With the code above and no operation on the frame, I get a flat green image. Using the same code with a Logitech webcam c920 works perfectly. The python code shows no error or warning.

@GastonLagaffe

Since you have seen the /dev/video0, it is very good progress.

In order to isolate the problem, we sill would like you to follow our document guide

to run the different command-line scripts, they are well proofed and should work if the hardware setup correctly.

After that, we can help you how to make it work with your own code.

Thanks.

Lee

Salut Lee,

the sample command as in 4.1 in the guide work, however afterwards the camera is no longer accessible and only a reboot frees it up.

To use the camera wit python and openCV, the simple code above from wwzeitler can be used to test. It is from the OpenCV documentation.

The image that is returned from the read of the capture device is flat 154 for green and 0 for red/blue for all pixels. As the identical code gets good images from a web cam, the problem can not be with openCV, it must be with the driver that feeds into /dev/video0

@GastonLagaffe

If commands from the guide 4.1 section work, it means the camera and driver works. It is good sign and we can move on forward.

I will forward the python and OpenCV test code issue to our engineer.

Cause our engineers are now on our National day holidays, the reply will be delayed.

Sorry for the inconvenience. If you are urgent, please send an email to [email protected]

PS: if you can share your hardware setup photos, it will be helpful for us to diagnose.

Just got a quick answer from our engineer, he said the driver output RAW10 format data which is not natively supported by the OpenCV, need extra formatting processing to display the image.

So you’d better use the ISP pipeline to read the image like the code below:

https://github.com/JetsonHacksNano/CSI-Camera

I have tried the JetsonHacks code. No matter what I do, the test cap.isOpened() is always flase and the code returns “camer not found”.

calling openCV for the normal /dev/vide0 with cap = cv2.VideoCapture(0) works but returns the flat-gren image. I also compiled the simple_camera.cpp and it works, but the python3 call fails, no matter what I try. I have now spent 4 hours today only on this and it may be the moment to trash that camera and look for something that works. At least I have no trust that this works in a stable way and survives the next updates

We still haven’t got your hardware setup photos, can you elaborate on this?

There is no reason that the code returns “camera not found”, since you can find the camera from /dev/video0 and can run the command line scripts from our guide.

Like I said the OpenCV is not natively supporting the RAW10 format from the driver. You have to do some conversion if you would like to use OpenCV. Our engineer will work on this when they come back from the holidays until Oct. 9th.

If you need further assistant, please contact [email protected] to find the right time for remote debugging.

Thanks.