How do I get an IMX477 to work with jetson xavier nx?

xavier setup

This is the view on the setup. The xavier has no connections other than the HDMI ans USB for the screen and the dongle for the keyboard and mouse.

I had no actions over the weekend but will try to load the drivers propsed by ridgerun:

https://developer.ridgerun.com/wiki/index.php?title=Raspberry_Pi_HQ_camera_IMX477_Linux_driver_for_Jetson#Downloading_the_debian_packages

To make is clear what software I am deploying on the Jetson Xavier NX, I have created a script that does all the installation calls. The file is on github (https://github.com/GastonLagaffe2013/JetsonXavierNX)

 

I just did a minimum installation, meaning flashing the SDcard, updating the packages and installing nothing but openCV (I even wiped the SSD):

$ sudo apt-get update $ sudo apt-get upgrade $ sudo apt-get -y install apt-utils nano $ sudo dpkg -i arducam-nvidia-l4t-kernel_4.9.140-32.4.3-20200721164156_arm64_imx477.deb $ sudo reboot $ sudo apt-get -y install python3-pip $ sudo apt-get -y autoremove $ python3 -m pip install --upgrade pip $ python3 -m pip install --upgrade setuptools $ sudo apt-get -y install python3-opencv $ git clone https://github.com/JetsonHacksNano/CSI-Camera $ cd CSI-Camera/ $ python3 simple_camera.py nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, width=(int)1280, height=(int)720, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink Unable to open camera
No improvement ...

Hi @GastonLagaffe ,

Can you send me the dmesg log?

Salut Wong,

you can find the dmesg here: https://github.com/GastonLagaffe2013/JetsonXavierNX/blob/main/dmesg_20201007.log

Ciao, Mathias

Hi @GastonLagaffe ,

From the dmesg log, you should be able to see /dev/video0.
Can the camera work properly with gstreamer command line?
https://www.arducam.com/docs/camera-for-jetson-nano/native-jetson-cameras-imx219-imx477/imx477/#4-example-pipelines

Asmentioned above, I can run the gstreamer command line but accessing the camera through python fails.

In the meantime I have setup a Jetson Nano with the same commands as in post #28653 above on a fresh flashed SDcard. Same results.

I have the Jetson Xavier and Nano side-by-side so I can test both platforms

Ciao, Mathias

I suggest to date time for a remote debugging, is it possible?

Hi @GastonLagaffe ,

We are still in the holiday, and there is no equipment around me to conduct the test. I will conduct the test immediately after the holiday. Please be patient.

ok - will play with cardiology models in the meantime :wink: - when is the holiday over?

Ciao, Mathias

Hi @GastonLagaffe ,

After my test, I found that the camera script will not work as long as opencv is manually installed. You need to uninstall the manually installed opencv by command: sudo apt remove python3-opencv

By the way, jetson’s sd Image comes with opencv4.1.1, you don’t need to install it manually.

Salut, I have removed the python3-openCV package and the cv2 version went from 3.2.0 to 4.1.1 and the code is working now. Thanks a lot for your patient support!

For my application I need maximum FPS rather than resolution, which is currently 60. Any plans to enable the higher FPS as described in the data sheet ?

Ciao, Mathias

Hi @GastonLagaffe ,

The csi interface of jetson xavier nx devkit only supports 2lane, so the frame rate cannot be increased to the maximum.

uname -a

Linux jetson-desktop 4.9.140-tegra #1 SMP PREEMPT Tue Jul 21 16:30:49 CST 2020 aarch64 aarch64 aarch64 GNU/Linux

Before install, I had a working raspberry pi v2 camera and USB camera (using GStreamer, cv2, and TensorFlow).

Then I turned off the jetson xavier nx and installed the IMX477 to the cam1 camera ribbon connector.

Here are the install steps I took:

sudo dpkg -i arducam-nvidia-l4t-kernel_4.9.140-32.4.3-20200721164156_arm64_imx477.deb

sudo dpkg -i nvidia-l4t-kernel-dtbs_4.9.140-tegra-32.4.3-20200625213407_arm64.deb

sudo dpkg -i arducam-nvidia-l4t-kernel_4.9.140-32.4.3-20200721164156_arm64_imx477.deb

sudo shutdown -r now

The raspberry pi v2 camera and USB camera no longer work. I can see the USB camera in dmseg, but there is no /dev/video1. When selecting the rasberry pi camera it now comes back “No camera found.”

Here is what I see with v4l2-ctl :

v4l2-ctl --list-devices & v4l2-ctl --list-formats-ext -d /dev/video0
[1] 26258
vi-output, imx477 9-001a (platform:15c10000.vi:0):
/dev/video0

ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘RG10’
Name : 10-bit Bayer RGRG/GBGB
Size: Discrete 4032x3040
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)

 

Here is the test Python script

import cv2
print(cv2.version)

#camSet=‘nvarguscamerasrc sensor-id=-1 ! video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1, format=NV12 ! nvvidconv flip-method=2 ! video/x-raw, width=800, height=600, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink’

#cam=cv2.VideoCapture(camSet)

cam=cv2.VideoCapture(’/dev/video0’)

while True:
_, frame = cam.read()
cv2.imshow(‘myCam’,frame)
if cv2.waitKey(1)==ord(‘q’):
break

cam.release()
cv2.destroyAllWindows()

 

Here is the output when I run the python script

4.1.1
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module source reported: Could not read from resource.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (886) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
VIDIOC_STREAMON: Remote I/O error
Traceback (most recent call last):
File “camera.py”, line 15, in <module>
cv2.imshow(‘myCam’,frame)
cv2.error: OpenCV(4.1.1) /home/nvidia/host/build_opencv/nv_opencv/modules/highgui/src/window.cpp:352: error: (-215:Assertion failed) size.width>0 && size.height>0 in function ‘imshow’

Here is what I see in dmseg output:

[ 2320.613598] Call trace:
[ 2320.613610] [<ffffff8008b34b24>] vb2_start_streaming+0xd4/0x158
[ 2320.613619] [<ffffff8008b36894>] vb2_core_streamon+0x16c/0x1a0
[ 2320.613626] [<ffffff8008b36b68>] __vb2_init_fileio+0x2a0/0x338
[ 2320.613632] [<ffffff8008b37b68>] __vb2_perform_fileio+0x3b8/0x5c0
[ 2320.613640] [<ffffff8008b37dbc>] vb2_read+0x4c/0x60
[ 2320.613648] [<ffffff8008b38c2c>] vb2_fop_read+0xbc/0x100
[ 2320.613656] [<ffffff8008b11cfc>] v4l2_read+0x6c/0xd8
[ 2320.613666] [<ffffff800825b100>] __vfs_read+0x48/0x110
[ 2320.613672] [<ffffff800825c09c>] vfs_read+0x94/0x150
[ 2320.613679] [<ffffff800825d78c>] SyS_read+0x54/0xb0
[ 2320.613688] [<ffffff8008083900>] el0_svc_naked+0x34/0x38
[ 2320.685887] misc tegra_camera_ctrl: tegra_camera_update_isobw: Warning, Requested ISO BW 26250000 has been capped to VI’s max BW 3925327
[ 2320.687559] tegra-i2c 3180000.i2c: no acknowledge from address 0x1a
[ 2320.690596] regmap_util_write_table_8:regmap_util_write_table:-121
[ 2320.690730] imx477 9-001a: Error writing mode
[ 2323.307633] tegra194-vi5 15c10000.vi: no reply from camera processor
[ 2323.354238] tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms
[ 2323.400280] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel
[ 2323.412891] tegra194-vi5 15c10000.vi: err_rec: successfully reset the capture channel
[ 2323.430918] ------------[ cut here ]------------
[ 2323.484488] WARNING: CPU: 0 PID: 26786 at /home/wang/Arducam/Jetson/l4t-32.4.3-NX/Linux_for_Tegra/source/public/kernel_src/kernel/kernel-4.9/drivers/media/v4l2-core/videobuf2-core.c:1370 vb2_start_streaming+0xd4/0x158
[ 2323.542073] Modules linked in:

[ 2323.542414] CPU: 0 PID: 26786 Comm: python3 Tainted: G W 4.9.140-tegra #1
[ 2323.542495] Hardware name: NVIDIA Jetson Xavier NX Developer Kit (DT)
[ 2323.542592] task: ffffffc1c5ea5400 task.stack: ffffffc1e2920000
[ 2323.542705] PC is at vb2_start_streaming+0xd4/0x158
[ 2323.542839] LR is at vb2_start_streaming+0x6c/0x158
[ 2323.542944] pc : [<ffffff8008b34b24>] lr : [<ffffff8008b34abc>] pstate: 20400045
[ 2323.543017] sp : ffffffc1e2923b30
[ 2323.543098] x29: ffffffc1e2923b30 x28: 0000000000000000
[ 2323.543320] x27: ffffffc1f2c2f800 x26: 0000000000000001
[ 2323.543526] x25: ffffff8009fdeeb0 x24: 0000000000000012
[ 2323.543733] x23: 0000000000000000 x22: ffffffc1e2923d10
[ 2323.543965] x21: ffffffc1f52a7768 x20: ffffffc1f52a7780
[ 2323.544171] x19: 00000000ffffff87 x18: 0000000000000000
[ 2323.544382] x17: 000000000000000e x16: 0000000000000007
[ 2323.544586] x15: 0000000000000000 x14: 0000000000018566
[ 2323.544792] x13: 00000000000001c7 x12: 071c71c71c71c71c
[ 2323.545052] x11: 000000000000000b x10: 0000000000000a10
[ 2323.545257] x9 : ffffffc1e29233a0 x8 : ffffffc1c5ea5e70
[ 2323.545464] x7 : fefefeff646c606d x6 : 000000002734de28
[ 2323.545673] x5 : 0000000000000000 x4 : 0000000000000001
[ 2323.545879] x3 : 0000000000000000 x2 : 0000000000010000
[ 2323.546119] x1 : 0000000000000000 x0 : ffffffc1b66d61b8

[ 2323.546416] —[ end trace 064f43179332ca36 ]—

 

And since the jetson xavier nx does not boot anyone, I’ll re-image the sd card and start over.

Thanks in advance.

 

 

 

Hi @rymd80 ,

I don’t know what happened to you, but I tested it on xavier nx as you said, and it works normally.

Can you reproduce this phenomenon every time?