Timestamps in CONTINUOUS_MODE IMX219/OV2311

1.Which seller did you purchase the product(s) from?
Other - www.botland.com.pl
2.The Model number of the product(s) you have purchased?
https://botland.store/raspberry-pi-cameras/16885-arducam-ov2311-2mpx-global-shutter-camera-module-for-raspberry-pi-noir-monochrome.html
https://botland.store/raspberry-pi-cameras/6128-raspberry-pi-noir-camera-hd-v2-8mpx-original-night-camera-for-raspberry-pi-640522710898.html
3.Which Platform are you using the product(s) on?
Linux
4.Which instruction are you following?
https://github.com/ArduCAM/ArduCAM_USB_Camera_Shield
5.Has your product ever worked properly?
N/A
6.What problems are you experiencing?
Hi,
I’m trying to get valid timestamps for frames to later sync with other sensors in my system. Unfortunately ArduCamOutData struct field u64Time provides some random numbers.
Is it possible to get reasonable timestamp in CONTINUOUS_MODE? (Assigning timestamp after reading data from FIFO ArduCam_readImage is sub optimal imho)
My initial idea was to save timestamp when ArduCam_captureImage is called and later assign it to frame in image reading thread.
7.What attempts at troubleshooting have you already made?

8.How would you like us to help you?
Please share your opinion and suggest solution for reasonable timestamping. Providing updated C/C++ documentation would also help.

Hi @edul

Sorry, the current SDK does not provide a timestamp. You can get the system time when readingImage, but this method has an uncertain delay.
Next week, I will take a closer look at the SDK to find the best way to add timestamps.

HIi @edul,

try this:

https://drive.google.com/file/d/1f18TwXnXLu0Q8eDEnE4t9Jvkl_TSrPD7/view

note: replace Arducam_SDK folder and execute make install_sdk

Hi @wong,

Thanks for sharing this with me, I will test it and let you know.
Can you describe how(when) timestamps are assigned in this version?

Hi @wong,

With new version frame u64Time still provides random number. Could you provide some valid example you used for testing?

Maybe it was not replaced correctly?
This is my test result:

By the way, the unit of time is microseconds.

I tried on few different sensors and it’s still doesn’t make sense.
Here is branch I used in tests https://github.com/ArduCAM/ArduCAM_USB_Camera_Shield/compare/master…e-dul-xb:install_sdk_dev_timestamps

To be 100% sure I removed old libs and clean then install everything again.

#####################
[email protected]:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ make clean
rm -f ArduCam_Demo
[email protected]:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ sudo rm -rf /usr/lib/libArduCamLib.so*
[email protected]:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ make install-sdk
sudo cp -d ./Arducam_SDK/lib*.so* /usr/lib
sudo cp ./Arducam_SDK/ArduCamLib.h /usr/include
[email protected]:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ ls -lha /usr/lib/libArduCamLib.so*
lrwxrwxrwx 1 root root 22 Jun 1 07:28 /usr/lib/libArduCamLib.so -> libArduCamLib.so.2.0.0
lrwxrwxrwx 1 root root 22 Jun 1 07:28 /usr/lib/libArduCamLib.so.2 -> libArduCamLib.so.2.0.0
-rwxr-xr-x 1 root root 353K Jun 1 07:28 /usr/lib/libArduCamLib.so.2.0.0
###########

Then compiled and run:
###########

[email protected]:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ make
g++ ArduCam_Demo.cpp Arducam_SDK/ArduCamLib.h -o ArduCam_Demo -lArduCamLib -lusb-1.0 -lpthread -larducam_config_parser pkg-config --cflags --libs opencv -L. -I. -std=gnu++11 -g
[email protected]:/code/arducam_ws/ArduCAM_USB_Camera_Shield/Linux_x86/Cpp/Streaming_demo$ ./ArduCam_Demo …/…/…/Config/USB3.0_UC-425_Rev.C+UC-628_Rev.B/OV2311/OV2311_MIPI_2Lane_RAW8_8b_1600x1300_60fps.cfg
device num:4
index: 0 Serial:AU3S-2114-0002
index: 2 Serial:
index: 2 Serial:
index: 4 Serial:
Serial: AU3S-2114-0002
setFramerate not a function

capture thread create successfully.
read thread create successfully.
Capture began, rtn_val = 0
Gtk-Message: 07:28:57.716: Failed to load module “canberra-gtk-module”
Gtk-Message: 07:28:57.716: Failed to load module “canberra-gtk-module”
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
-------------------11 fps
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
time: 15
time: 0
time: 94227671415760
time: 79
time: 0
time: 94227671415904
########

Any suggestions?
btw. going back to question from previous message: Can you describe how(when) timestamps are assigned in this version?

Hi @edul

I think I found my mistake, a part that I missed, this version should be correct:
https://drive.google.com/file/d/1KfoFNYBrC0tkvI4fFtDG7d3Hli3uQUhm/view

Regarding how timestamp is assigned:
This version of timestamps will be assigned the current time when a frame is received.

Hi @wong,

New version assigns reasonable timestamps, thank you.

This version of timestamps will be assigned the current time when a frame is received.
In streaming demo it would look something like this?

This solution would aggregate USB communication related delays and jitter. Please correct me if I’m wrong.
On the other hand I would suggest to assign it earlier when capture starts, I think it would be more reasonable.

Please let me know what you think about it.

Hi @edul

This solution would aggregate USB communication related delays and jitter. Please correct me if I’m wrong.
Your sequence diagram is basically correct, except for the captureImage part, the captureImage part is not blocking, in fact it is handling libusb event processing. https://libusb.sourceforge.io/api-1.0/group__libusb__poll.html
On the other hand I would suggest to assign it earlier when capture starts, I think it would be more reasonable.

You are right, it can be done, I will conduct some simple tests.

You can try this version, I added a timestamp when I first received the data of a certain frame:
https://drive.google.com/file/d/1gJspFr9pG5vhrqPbXVwVVgflHsFOaBpd/view?usp=sharing

Hi @wong

Thanks for update, I will try to test it and let you know how it works for me :slight_smile: