Timestamps in CONTINUOUS_MODE IMX219/OV2311

edul · May 27, 2021, 5:20am

1.Which seller did you purchase the product(s) from?
Other - www.botland.com.pl
2.The Model number of the product(s) you have purchased?
https://botland.store/raspberry-pi-cameras/16885-arducam-ov2311-2mpx-global-shutter-camera-module-for-raspberry-pi-noir-monochrome.html
https://botland.store/raspberry-pi-cameras/6128-raspberry-pi-noir-camera-hd-v2-8mpx-original-night-camera-for-raspberry-pi-640522710898.html
3.Which Platform are you using the product(s) on?
Linux
4.Which instruction are you following?
https://github.com/ArduCAM/ArduCAM_USB_Camera_Shield
5.Has your product ever worked properly?
N/A
6.What problems are you experiencing?
Hi,
I’m trying to get valid timestamps for frames to later sync with other sensors in my system. Unfortunately ArduCamOutData struct field u64Time provides some random numbers.
Is it possible to get reasonable timestamp in CONTINUOUS_MODE? (Assigning timestamp after reading data from FIFO ArduCam_readImage is sub optimal imho)
My initial idea was to save timestamp when ArduCam_captureImage is called and later assign it to frame in image reading thread.
7.What attempts at troubleshooting have you already made?

8.How would you like us to help you?
Please share your opinion and suggest solution for reasonable timestamping. Providing updated C/C++ documentation would also help.

wong · May 28, 2021, 2:23am

Hi @edul

Sorry, the current SDK does not provide a timestamp. You can get the system time when readingImage, but this method has an uncertain delay.
Next week, I will take a closer look at the SDK to find the best way to add timestamps.

wong · May 30, 2021, 7:54pm

HIi @edul,

try this:

https://drive.google.com/file/d/1f18TwXnXLu0Q8eDEnE4t9Jvkl_TSrPD7/view

note: replace Arducam_SDK folder and execute make install_sdk

edul · May 30, 2021, 10:04pm

Hi @wong,

Thanks for sharing this with me, I will test it and let you know.
Can you describe how(when) timestamps are assigned in this version?

edul · May 31, 2021, 5:45am

Hi @wong,

With new version frame u64Time still provides random number. Could you provide some valid example you used for testing?

wong · May 31, 2021, 5:40pm

Maybe it was not replaced correctly?
This is my test result:

wong · May 31, 2021, 5:46pm

By the way, the unit of time is microseconds.

edul · June 1, 2021, 12:43am

I tried on few different sensors and it’s still doesn’t make sense.
Here is branch I used in tests https://github.com/ArduCAM/ArduCAM_USB_Camera_Shield/compare/master…e-dul-xb:install_sdk_dev_timestamps

To be 100% sure I removed old libs and clean then install everything again.

#####################
dev@dev:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ make clean
rm -f ArduCam_Demo
dev@dev:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ sudo rm -rf /usr/lib/libArduCamLib.so*
dev@dev:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ make install-sdk
sudo cp -d ./Arducam_SDK/lib*.so* /usr/lib
sudo cp ./Arducam_SDK/ArduCamLib.h /usr/include
dev@dev:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ ls -lha /usr/lib/libArduCamLib.so*
lrwxrwxrwx 1 root root 22 Jun 1 07:28 /usr/lib/libArduCamLib.so -> libArduCamLib.so.2.0.0
lrwxrwxrwx 1 root root 22 Jun 1 07:28 /usr/lib/libArduCamLib.so.2 -> libArduCamLib.so.2.0.0
-rwxr-xr-x 1 root root 353K Jun 1 07:28 /usr/lib/libArduCamLib.so.2.0.0
###########

Then compiled and run:
###########

dev@dev:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ make
g++ ArduCam_Demo.cpp Arducam_SDK/ArduCamLib.h -o ArduCam_Demo -lArduCamLib -lusb-1.0 -lpthread -larducam_config_parser pkg-config --cflags --libs opencv -L. -I. -std=gnu++11 -g
dev@dev:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ ./ArduCam_Demo …/…/…/Config/USB3.0_UC-425_Rev.C+UC-628_Rev.B/OV2311/OV2311_MIPI_2Lane_RAW8_8b_1600x1300_60fps.cfg
device num:4
index: 0 Serial:AU3S-2114-0002
index: 2 Serial:
index: 2 Serial:
index: 4 Serial:
Serial: AU3S-2114-0002
setFramerate not a function

capture thread create successfully.
read thread create successfully.
Capture began, rtn_val = 0
Gtk-Message: 07:28:57.716: Failed to load module “canberra-gtk-module”
Gtk-Message: 07:28:57.716: Failed to load module “canberra-gtk-module”
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
-------------------11 fps
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
########

Any suggestions?
btw. going back to question from previous message: Can you describe how(when) timestamps are assigned in this version?

wong · June 1, 2021, 5:55pm

Hi @edul

I think I found my mistake, a part that I missed, this version should be correct:
https://drive.google.com/file/d/1KfoFNYBrC0tkvI4fFtDG7d3Hli3uQUhm/view

Regarding how timestamp is assigned:
This version of timestamps will be assigned the current time when a frame is received.

edul · June 2, 2021, 2:31am

Hi @wong,

New version assigns reasonable timestamps, thank you.

This version of timestamps will be assigned the current time when a frame is received.

In streaming demo it would look something like this?

This solution would aggregate USB communication related delays and jitter. Please correct me if I’m wrong.
On the other hand I would suggest to assign it earlier when capture starts, I think it would be more reasonable.

Please let me know what you think about it.

wong · June 2, 2021, 5:56pm

Hi @edul

This solution would aggregate USB communication related delays and jitter. Please correct me if I’m wrong.

Your sequence diagram is basically correct, except for the captureImage part, the captureImage part is not blocking, in fact it is handling libusb event processing. https://libusb.sourceforge.io/api-1.0/group__libusb__poll.html

On the other hand I would suggest to assign it earlier when capture starts, I think it would be more reasonable.

You are right, it can be done, I will conduct some simple tests.

wong · June 2, 2021, 7:03pm

You can try this version, I added a timestamp when I first received the data of a certain frame:
https://drive.google.com/file/d/1gJspFr9pG5vhrqPbXVwVVgflHsFOaBpd/view?usp=sharing

edul · June 7, 2021, 12:28am

Hi @wong

Thanks for update, I will try to test it and let you know how it works for me