IMX462 & libcamera stillCampture Role, slow cycle time and low performance

tianyi.lu · April 14, 2023, 9:41pm

Where did you get the camera module(s)?
UCTRONICS
Model number of the product(s)?
IMX462
What hardware/platform were you working on?
Raspberry Pi 4B
OS Info:
PRETTY_NAME=“Debian GNU/Linux 11 (bullseye)”
NAME=“Debian GNU/Linux”
VERSION_ID=“11”
VERSION=“11 (bullseye)”
VERSION_CODENAME=bullseye
ID=debian

Version: 6.1.21-v8+

Instructions you have followed. (link/manual/etc.)
Quick Start - Arducam Wiki
Problems you were having?
I am playing with this camera to take a static image each time when the user give a command to my system. The user might send a bunch of command to take many images and each command might be along with a different exposure time, so, I have to stop the camera and start over again each time when I try to take the image. I also need to make sure the cycle time during the whole process as fast as possible. Therefore, I built a customized program based on libcamera and grabbed the image using StreamRole::StillCapture and pixelformat = libcamera::formats::NV12 ( I just want to get grayscale image ). My workflow likes this below,

constexpr static unsigned int expTimes = 16;

std::valarray<int64_t> expTimes_us(expTimes);

glossCamera->openCamera();
for (unsigned ii = 0; ii < expTimes; ++ii)
{
    auto startTime = std::chrono::high_resolution_clock::now();
    // stop camera anyway
    glossCamera->stopCamera();
    // clear current configuration
    glossCamera->tearDown();
    glossCamera->configureCamera();
    glossCamera->startCamera();
    // Then get the image, waiting until the new frame is ready or the timeout is reached
    ImageBuffer imgBuffer;
    glossCamera->getLatestFrame(imgBuffer);
    auto endTime = std::chrono::high_resolution_clock::now();

    expTimes_us[ii] = static_cast<int64_t>(std::chrono::duration_cast<std::chrono::milliseconds>(endTime - startTime).count());
}

// stop and close the camera
glossCamera->stopCamera();
glossCamera->closeCamera();

It works fine, but just the performance is not good as I expected, the Cycle Time is over 400ms. But I just set the exposure time as 1000us, resolution as 960x560 and disable the Auto_Exposure, Auto_Focus as well as denoise.

The dmesg log from your hardware?

pi@oceandevpi64:~/workspace/arducam/libglosscameras $ dmesg | grep arducam
[ 0.076986] platform fe801000.csi: Fixed dependency cycle(s) with /soc/i2c0mux/i2c@1/arducam_pivariety@c
[ 8.266402] arducam-pivariety 10-000c: firmware version: 0x10002
[ 8.716214] arducam-pivariety 10-000c: Consider updating driver arducam-pivariety to match on endpoints

Troubleshooting attempts you’ve made?
I tried to set the StreamRole as libcamera::StreamRole::Raw, and then I got the Raw10 format pixel. But the cycle time is still at low performance ( over 400ms).
What help do you need?
My questions are,
a. For this specific pivariety camera, what is the performance supposed to be in each cycle like this?
b. I might do some inappropriate configuration. So, do you have any suggestion that I can try to set to increase the performance?

Edward · April 17, 2023, 2:38am

@tianyi.lu

You’re turning the camera on and off repeatedly every time, which can be a factor in timing. You can just set the exposure each time, then skip two frames and wait for the exposure to take effect, then save the image.

tianyi.lu · April 17, 2023, 8:41pm

Thanks for your reply. I tried that and made the request and buffer reused m_curRequest->reuse(libcamera::Request::ReuseBuffers); but I still need to skip 7 frames to wait for the exposure time to take effect not 2 frames. There might be some other configuration that I did not configure correctly.

I also did an experiment to see how fast the camera will adjust the auto exposure. Basically, I disable the auto exposure and set a low exposure time (100us) at camera initial time, after acquiring a few frames, I enable the auto exposure and log the completed request meta data to monitor the real exposure time that the camera used. I noticed that the camera will skip 7 frames and then tried to compute the exposure time according to the ambient light. During the auto adjustment, it took 2 frames to apply a new exposure time just like what you said. So, I am a little confusing that why it takes more frames to adjust the exposure time manfully than the camera automatically adjusts that by itself?

I posted the log messages after enabling the auto exposure here. TIA.

<548463955776> 2023/04/17 20:56:21 268696 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 268743 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 268757 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 268778 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 268792 INFO Request Reply: ExposureTime=88
<548463955776> 2023/04/17 20:56:21 268805 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 268824 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 268850 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 268877 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 268892 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 268911 INFO Request Reply: Lux=1048.481079
<548463955776> 2023/04/17 20:56:21 268925 INFO Request Reply: AeLocked=true
<548463955776> 2023/04/17 20:56:21 268942 INFO Request Reply: ColourGains=[ 1.135209, 2.570635 ]
<548463955776> 2023/04/17 20:56:21 268958 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 268973 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 268989 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 269010 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 269024 INFO Request Reply: SensorTimestamp=2451917079000
<548463955776> 2023/04/17 20:56:21 269063 WARNING Frame: 1
<548463955776> 2023/04/17 20:56:21 280611 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 280746 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 280816 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 280899 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 280981 INFO Request Reply: ExposureTime=88
<548463955776> 2023/04/17 20:56:21 281054 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 281129 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 281207 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 281288 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 281357 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 281430 INFO Request Reply: Lux=1055.846191
<548463955776> 2023/04/17 20:56:21 281501 INFO Request Reply: AeLocked=true
<548463955776> 2023/04/17 20:56:21 281574 INFO Request Reply: ColourGains=[ 1.135585, 2.575887 ]
<548463955776> 2023/04/17 20:56:21 281646 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 281716 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 281787 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 281862 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 281937 INFO Request Reply: SensorTimestamp=2451933745000
<548463955776> 2023/04/17 20:56:21 282029 WARNING Frame: 2
<548463955776> 2023/04/17 20:56:21 292723 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 292850 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 292924 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 293088 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 293195 INFO Request Reply: ExposureTime=88
<548463955776> 2023/04/17 20:56:21 293306 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 293417 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 293536 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 293661 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 293774 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 293893 INFO Request Reply: Lux=1055.846191
<548463955776> 2023/04/17 20:56:21 294004 INFO Request Reply: AeLocked=true
<548463955776> 2023/04/17 20:56:21 294122 INFO Request Reply: ColourGains=[ 1.135585, 2.575887 ]
<548463955776> 2023/04/17 20:56:21 294234 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 294343 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 294453 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 294580 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 294697 INFO Request Reply: SensorTimestamp=2451950413000
<548463955776> 2023/04/17 20:56:21 294861 WARNING Frame: 3
<548463955776> 2023/04/17 20:56:21 307854 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 307984 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 308054 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 308200 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 308305 INFO Request Reply: ExposureTime=88
<548463955776> 2023/04/17 20:56:21 308461 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 308575 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 308689 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 308815 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 308929 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 309049 INFO Request Reply: Lux=1061.033081
<548463955776> 2023/04/17 20:56:21 309158 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 309276 INFO Request Reply: ColourGains=[ 1.135942, 2.580877 ]
<548463955776> 2023/04/17 20:56:21 309395 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 309506 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 309611 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 309748 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 309863 INFO Request Reply: SensorTimestamp=2451967079000
<548463955776> 2023/04/17 20:56:21 310057 WARNING Frame: 4
<548463955776> 2023/04/17 20:56:21 321701 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 321825 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 321897 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 322042 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 322497 INFO Request Reply: ExposureTime=88
<548463955776> 2023/04/17 20:56:21 322625 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 323051 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 323196 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 323325 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 323441 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 323575 INFO Request Reply: Lux=1063.716064
<548463955776> 2023/04/17 20:56:21 324013 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 324162 INFO Request Reply: ColourGains=[ 1.136282, 2.585617 ]
<548463955776> 2023/04/17 20:56:21 324283 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 324393 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 324866 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 325005 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 325124 INFO Request Reply: SensorTimestamp=2452000414000
<548463955776> 2023/04/17 20:56:21 325310 WARNING Frame: 5
<548463955776> 2023/04/17 20:56:21 334111 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 334237 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 334308 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 334390 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 334475 INFO Request Reply: ExposureTime=88
<548463955776> 2023/04/17 20:56:21 334551 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 334627 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 334723 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 334811 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 334890 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 334979 INFO Request Reply: Lux=1063.716064
<548463955776> 2023/04/17 20:56:21 335055 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 335135 INFO Request Reply: ColourGains=[ 1.136282, 2.585617 ]
<548463955776> 2023/04/17 20:56:21 335212 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 335286 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 335360 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 335443 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 335497 INFO Request Reply: SensorTimestamp=2452017081000
<548463955776> 2023/04/17 20:56:21 335618 WARNING Frame: 6
<548463955776> 2023/04/17 20:56:21 351265 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 351387 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 351458 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 351530 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 351615 INFO Request Reply: ExposureTime=88
<548463955776> 2023/04/17 20:56:21 351686 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 351762 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 351849 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 351935 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 352012 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 352085 INFO Request Reply: Lux=1067.782349
<548463955776> 2023/04/17 20:56:21 352158 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 352231 INFO Request Reply: ColourGains=[ 1.136604, 2.590120 ]
<548463955776> 2023/04/17 20:56:21 352305 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 352379 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 352481 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 352564 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 352638 INFO Request Reply: SensorTimestamp=2452033748000
<548463955776> 2023/04/17 20:56:21 352737 WARNING Frame: 7
<548463955776> 2023/04/17 20:56:21 363195 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 363321 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 363390 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 363466 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 363546 INFO Request Reply: ExposureTime=296
<548463955776> 2023/04/17 20:56:21 363623 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 363678 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 363804 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 363890 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 363965 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 364043 INFO Request Reply: Lux=1067.782349
<548463955776> 2023/04/17 20:56:21 364116 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 364191 INFO Request Reply: ColourGains=[ 1.136604, 2.590120 ]
<548463955776> 2023/04/17 20:56:21 364266 INFO Request Reply: DigitalGain=1.125000
<548463955776> 2023/04/17 20:56:21 364339 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 364413 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 364521 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 364597 INFO Request Reply: SensorTimestamp=2452050419000
<548463955776> 2023/04/17 20:56:21 364694 WARNING Frame: 8
<548463955776> 2023/04/17 20:56:21 383945 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 384077 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 384148 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 384221 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 384296 INFO Request Reply: ExposureTime=296
<548463955776> 2023/04/17 20:56:21 384378 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 384486 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 384568 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 384652 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 384729 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 384803 INFO Request Reply: Lux=1096.019531
<548463955776> 2023/04/17 20:56:21 384877 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 384955 INFO Request Reply: ColourGains=[ 1.136910, 2.594398 ]
<548463955776> 2023/04/17 20:56:21 385030 INFO Request Reply: DigitalGain=1.014657
<548463955776> 2023/04/17 20:56:21 385106 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 385189 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 385274 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 385355 INFO Request Reply: SensorTimestamp=2452067080000
<548463955776> 2023/04/17 20:56:21 385454 WARNING Frame: 9
<548463955776> 2023/04/17 20:56:21 396667 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 396797 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 396867 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 396940 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 397026 INFO Request Reply: ExposureTime=459
<548463955776> 2023/04/17 20:56:21 397101 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 397170 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 397255 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 397343 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 397418 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 397499 INFO Request Reply: Lux=1096.019531
<548463955776> 2023/04/17 20:56:21 397578 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 397654 INFO Request Reply: ColourGains=[ 1.136910, 2.594398 ]
<548463955776> 2023/04/17 20:56:21 397730 INFO Request Reply: DigitalGain=1.014657
<548463955776> 2023/04/17 20:56:21 397806 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 397880 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 397959 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 398036 INFO Request Reply: SensorTimestamp=2452083747000
<548463955776> 2023/04/17 20:56:21 398132 WARNING Frame: 10
<548463955776> 2023/04/17 20:56:21 417156 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 417280 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 417351 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 417423 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 417498 INFO Request Reply: ExposureTime=577
<548463955776> 2023/04/17 20:56:21 417574 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 417652 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 417741 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 417827 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 417902 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 417980 INFO Request Reply: Lux=1077.984375
<548463955776> 2023/04/17 20:56:21 418057 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 418132 INFO Request Reply: ColourGains=[ 1.137201, 2.598463 ]
<548463955776> 2023/04/17 20:56:21 418207 INFO Request Reply: DigitalGain=1.013883
<548463955776> 2023/04/17 20:56:21 418281 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 418354 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 418434 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 418510 INFO Request Reply: SensorTimestamp=2452100420000
<548463955776> 2023/04/17 20:56:21 418610 WARNING Frame: 11
<548463955776> 2023/04/17 20:56:21 430075 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 430204 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 430275 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 430359 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 430432 INFO Request Reply: ExposureTime=681
<548463955776> 2023/04/17 20:56:21 430512 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 430589 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 430675 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 430764 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 430824 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 430939 INFO Request Reply: Lux=1077.984375
<548463955776> 2023/04/17 20:56:21 431012 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 431086 INFO Request Reply: ColourGains=[ 1.137201, 2.598463 ]
<548463955776> 2023/04/17 20:56:21 431162 INFO Request Reply: DigitalGain=1.013883
<548463955776> 2023/04/17 20:56:21 431235 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 431313 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 431392 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 431467 INFO Request Reply: SensorTimestamp=2452117080000
<548463955776> 2023/04/17 20:56:21 431568 WARNING Frame: 12
<548463955776> 2023/04/17 20:56:21 450843 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 450975 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 451056 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 451127 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 451209 INFO Request Reply: ExposureTime=681
<548463955776> 2023/04/17 20:56:21 451287 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 451363 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 451444 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 451531 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 451606 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 451686 INFO Request Reply: Lux=1075.723633
<548463955776> 2023/04/17 20:56:21 451760 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 451860 INFO Request Reply: ColourGains=[ 1.137478, 2.602324 ]
<548463955776> 2023/04/17 20:56:21 451930 INFO Request Reply: DigitalGain=1.005955
<548463955776> 2023/04/17 20:56:21 452003 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 452077 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 452156 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 452235 INFO Request Reply: SensorTimestamp=2452133749000
<548463955776> 2023/04/17 20:56:21 452340 WARNING Frame: 13
<548463955776> 2023/04/17 20:56:21 463379 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 463547 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 463624 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 463704 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 463781 INFO Request Reply: ExposureTime=740
<548463955776> 2023/04/17 20:56:21 463852 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 463926 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 464013 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 464100 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 464199 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 464273 INFO Request Reply: Lux=1075.723633
<548463955776> 2023/04/17 20:56:21 464346 INFO Request Reply: AeLocked=false
<548463955776> 2023/04/17 20:56:21 464450 INFO Request Reply: ColourGains=[ 1.137478, 2.602324 ]
<548463955776> 2023/04/17 20:56:21 464532 INFO Request Reply: DigitalGain=1.005955
<548463955776> 2023/04/17 20:56:21 464606 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 464682 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 464763 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 464837 INFO Request Reply: SensorTimestamp=2452150415000
<548463955776> 2023/04/17 20:56:21 464938 WARNING Frame: 14
<548463955776> 2023/04/17 20:56:21 484100 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 484223 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 484293 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 484373 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 484471 INFO Request Reply: ExposureTime=740
<548463955776> 2023/04/17 20:56:21 484553 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 484628 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 484708 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 484792 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 484867 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 484946 INFO Request Reply: Lux=1074.867188
<548463955776> 2023/04/17 20:56:21 485021 INFO Request Reply: AeLocked=true
<548463955776> 2023/04/17 20:56:21 485101 INFO Request Reply: ColourGains=[ 1.137740, 2.605991 ]
<548463955776> 2023/04/17 20:56:21 485176 INFO Request Reply: DigitalGain=1.001666
<548463955776> 2023/04/17 20:56:21 485250 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 485325 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 485405 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 485478 INFO Request Reply: SensorTimestamp=2452167084000
<548463955776> 2023/04/17 20:56:21 485575 WARNING Frame: 15
<548463955776> 2023/04/17 20:56:21 496744 DEBUG Request completed…
<548463955776> 2023/04/17 20:56:21 496871 DEBUG Callback function Current Thread ID: 548463955776
<548463955776> 2023/04/17 20:56:21 496940 INFO Request completed successfully!
<548463955776> 2023/04/17 20:56:21 497014 INFO Request Reply: NoiseReductionMode=1
<548463955776> 2023/04/17 20:56:21 497093 INFO Request Reply: ExposureTime=785
<548463955776> 2023/04/17 20:56:21 497174 INFO Request Reply: AeEnable=true
<548463955776> 2023/04/17 20:56:21 497250 INFO Request Reply: FrameDurationLimits=[ 16666, 200000 ]
<548463955776> 2023/04/17 20:56:21 497337 INFO Request Reply: AnalogueGain=1.000000
<548463955776> 2023/04/17 20:56:21 497424 INFO Request Reply: ColourCorrectionMatrix=[ 1.831948, -0.820011, -0.011936, -0.430259, 1.753510, -0.323250, -0.071906, -0.713727, 1.785643 ]
<548463955776> 2023/04/17 20:56:21 497486 INFO Request Reply: FrameDuration=16666
<548463955776> 2023/04/17 20:56:21 497599 INFO Request Reply: Lux=1074.867188
<548463955776> 2023/04/17 20:56:21 497672 INFO Request Reply: AeLocked=true
<548463955776> 2023/04/17 20:56:21 497771 INFO Request Reply: ColourGains=[ 1.137740, 2.605991 ]
<548463955776> 2023/04/17 20:56:21 497845 INFO Request Reply: DigitalGain=1.001666
<548463955776> 2023/04/17 20:56:21 497919 INFO Request Reply: ColourTemperature=4500
<548463955776> 2023/04/17 20:56:21 497993 INFO Request Reply: SensorBlackLevels=[ 4096, 4096, 4096, 4096 ]
<548463955776> 2023/04/17 20:56:21 498074 INFO Request Reply: ScalerCrop=(0, 0)/1920x1080
<548463955776> 2023/04/17 20:56:21 498149 INFO Request Reply: SensorTimestamp=2452183747000
<548463955776> 2023/04/17 20:56:21 498245 WARNING Frame: 16

Edward · April 18, 2023, 1:23am

@tianyi.lu

I’m not sure about this issue at the moment, and I need to study the relevant part of the source code. I’m sorry that I don’t have much time to study this. If you are interested, you can take a look at the source code.

tianyi.lu · April 18, 2023, 10:01pm

I appreciate it.

I have another question. I am sorry for not posting this in the same thread but this is the new thing that relating to our product. I assume that IMX462 with board UC-755 and adapter board UC-667 is able to support external trigger, right? Would you be able to tell me how to hook up the wires to enable external trigger and is there any particular settings that needs to be done for this? I looked up on your sites and just found the article for accessing the Global Shutter Camera here, Access Global Shutter Camera using external trigger snapshot mode - Arducam Wiki

I guess I need to set up the trigger mode if I expect to use external trigger through v4l2-ctl command. I also encountered a very whacky issue when I tried to show the control list over v4l2-ctl -d /dev/video0 -l command , but there was nothing showed up. I also tried to use the command v4l2-ctl -d /dev/v4l-subdev0 -l shortly afterwards. It showed up some of the control list this time but still no trigger_mode.

Here is the bash command that I used to try.

pi@oceandevpi64:~ $ dmesg | grep arducam
[ 0.077878] platform fe801000.csi: Fixed dependency cycle(s) with /soc/i2c0mux/i2c@1/arducam_pivariety@c
[ 7.779296] arducam-pivariety 10-000c: firmware version: 0x10002
[ 8.104311] arducam-pivariety 10-000c: Consider updating driver arducam-pivariety to match on endpoints
pi@oceandevpi64:~ $ v4l2-ctl -l
pi@oceandevpi64:~ $ ls /dev/video*
/dev/video0 /dev/video11 /dev/video13 /dev/video15 /dev/video18 /dev/video20 /dev/video22 /dev/video31
/dev/video10 /dev/video12 /dev/video14 /dev/video16 /dev/video19 /dev/video21 /dev/video23
pi@oceandevpi64:~ $ v4l2-ctl -d /dev/video0 -l
pi@oceandevpi64:~ $ v4l2-ctl -d /dev/v4l
v4l/ v4l-subdev0
pi@oceandevpi64:~ $ v4l2-ctl -d /dev/v4l
v4l/ v4l-subdev0
pi@oceandevpi64:~ $ v4l2-ctl -d /dev/v4l-subdev0 -l

User Controls
                   exposure 0x00980911 (int)    : min=1 max=1048573 step=1 default=1022 value=1022
            horizontal_flip 0x00980914 (bool)   : default=0 value=0
              vertical_flip 0x00980915 (bool)   : default=0 value=0
Camera Controls
         camera_orientation 0x009a0922 (menu)   : min=0 max=2 default=2 value=2 flags=read-only
     camera_sensor_rotation 0x009a0923 (int)    : min=0 max=0 step=1 default=0 value=0 flags=read-only
Image Source Controls
          vertical_blanking 0x009e0901 (int)    : min=45 max=12420 step=1 default=45 value=45
        horizontal_blanking 0x009e0902 (int)    : min=280 max=280 step=1 default=280 value=280 flags=read-only
              analogue_gain 0x009e0903 (int)    : min=100 max=20000 step=1 default=100 value=100
Image Processing Controls
                 pixel_rate 0x009f0902 (int64)  : min=148500000 max=148500000 step=1 default=148500000 value=148500000 flags=read-only
pi@oceandevpi64:~ $ v4l2-ctl -d /dev/v4l-subdev0 -C trigger_mode
unknown control ‘trigger_mode’

Edward · April 19, 2023, 1:29am

@tianyi.lu

I checked the firmware, for this camera, there is no external trigger function.
Where did you see that we advertise that this camera supports external triggering?

tianyi.lu · April 20, 2023, 2:49am

Thanks for the confirmation. I just noticed that there is a trigger pin on the camera board. So, I was assuming it might support external trigger.

For v4l2-ctl command, would you please help me to take look at that? Except for v4l2-ctl -d /dev/v4l-subdev0 -l this command, the other commands showed nothing even though I specified the -d as /dev/video0. What is the difference of /dev/video0 and /dev/v4l-subdev0? TIA

Edward · April 21, 2023, 2:22am

@tianyi.lu

look this
bcm2835-unicam 和媒体控制器 API - 现已上线 - 树莓派论坛 (raspberrypi.com)

tianyi.lu · April 26, 2023, 10:24pm

Hi @Edward ,

I got a Global Shutter Camera (OV2311) today, the board Rev. is UC-788. I was trying to follow the instruction to configure external trigger. But I still have some questions below,

Should I convert the voltage from 3.3v to 1.8v? (Currently, I do not perform any conversion)
If there is no signal sent to the XVS pin, the pin will be floating? which means that it also might be triggered even though there is no any wire hooked up to that pin?
Seems like the camera could still acquire 2 frames after the request timeout. So, should I need to drop 2 frames if timeout happens and camera is restarted?

ERROR: Device timeout detected, attempting a restart!!!
#579 (0.00 fps) exp 995.00 ag 6.00 dg 1.00
#580 (119.06 fps) exp 995.00 ag 6.00 dg 1.00
[3:19:06.468528248] [8892] WARN V4L2 v4l2_videodevice.cpp:2007 /dev/video0[13:cap]: Dequeue timer of 1000000.00us has expired!
[3:19:06.468700506] [8892] ERROR RPI raspberrypi.cpp:2013 Unicam has timed out!
[3:19:06.468764839] [8892] ERROR RPI raspberrypi.cpp:2014 Please check that your camera sensor connector is attached securely.
[3:19:06.468821727] [8892] ERROR RPI raspberrypi.cpp:2015 Alternatively, try another cable and/or sensor.
ERROR: Device timeout detected, attempting a restart!!!
#581 (0.00 fps) exp 995.00 ag 6.00 dg 1.00
#582 (119.05 fps) exp 995.00 ag 6.00 dg 1.00

Edward · April 27, 2023, 1:44am

@tianyi.lu

The OV2311 camera module typically operates on a 1.8V power supply, so it’s recommended to convert the voltage from 3.3V to 1.8V to avoid any potential damage to the camera module. You can use a voltage regulator or a voltage level shifter to convert the voltage from 3.3V to 1.8V.
Yes, if there is no signal sent to the XVS pin, it may float and potentially trigger the camera unexpectedly. To avoid this, you can connect a pull-down resistor to the XVS pin to keep it at a low level when there is no external trigger signal.
The OV2311 camera module has a feature called “pre-fetch,” which allows it to capture and store up to two frames in advance. If a request timeout occurs, the camera will still continue to capture frames until the pre-fetch frames are exhausted. In this case, you may need to drop the frames captured during the pre-fetch period if they are not useful to your application. However, if you need to capture all frames and don’t want to miss any frames, you can increase the request timeout value to ensure that all frames are captured.

tianyi.lu · April 27, 2023, 7:25pm

Hi @Edward,

I appreciate it. It is really helpful, I increased the timeout number and it works fine.

But here I got some other issues,

According to the external trigger instruction, I am assuming that the camera is supposed to be triggered by one pulse as long as the pulse width is not less than 2us. However, I found that actually, the camera needs 3 pulses to be triggered. So, basically, I created trigger signal 3 times with 2 seconds sleep time between each trigger, and the pulse width for each trigger is about 1ms.

/home/pi/workspace/arducam/libglosscameras/trigger.py:9: RuntimeWarning: This channel is already in use, continuing anyway. Use GPIO.setwarnings(False) to disable warnings.
GPIO.setup(ledPin, GPIO.OUT) # LED pin set as output
Pull down the trigger signal!
Start Trigger! Press CTRL+C to exit
trigger: 0
trigger: 1
trigger: 2

Another question is about the trigger mode configuration. Right now, I find that the trigger_mode could be configured by the command v4l2-ctl -d /dev/v4l-subdev0 -c trigger_mode=1, but I noticed that every time since the pi get reboot, the trigger_mode will be back to the default value 0. So, would you be able to provide me any suggestion how can I configure that by using some interface of libcamera?
For long exposure time, the software trigger could get a reasonable one but the external trigger get the one seems like in a very short exposure time. I cannot understand that. Seems like there is a settings called Low Light Compensation according to this. But I know this link is for UVC Camera.

Edward · April 28, 2023, 3:41am

@tianyi.lu

Qustion1 i need test.pls give me some time.

Qustion2

libcamera has no parameter control. The camera will reset every time the Raspberry Pi is restarted, so the values will be restored.

Qustion3
In the external trigger mode, the exposure line will become very short, we recommend users to use the supplementary light in the external trigger mode.

For question 1, I want to test it again.

But I have a 5-day vacation below, and I may have to reply to you next week.

tianyi.lu · April 28, 2023, 9:39pm

@Edward

No worries and enjoy your vacation.

One more thing if you have time after the vacation, could you explain more detail about the the exposure line will become very short? Because it does not make sense to me since we can not control the exposure time in external trigger mode. So, I assume that there must be some other ways to control the exposure time in external trigger mode.

Edward · May 6, 2023, 7:24am

@tianyi.lu

I just had a meeting with my team to discuss your question, and I will conduct a thorough test and provide a report. Please allow me a few days to complete this process.

Edward · May 12, 2023, 8:10am

@tianyi.lu

After conducting a detailed test, I’ve confirmed that the brightness is very low in external trigger mode. I will need to research this issue further, as it appears to be related to camera characteristics.

tianyi.lu · May 17, 2023, 10:25pm

@Edward

Thanks for your investigation.
Another thing will be the multi-time trigger pulses which I asked a while ago. So, did you also conduct the test as well?

Actually, in the external trigger mode, I tested the camera with different exposure time, seems like the the number of pulses that camera needs is somehow connected to the exposure time set to the camera. I notice that it might need more pulses to trigger the camera as the exposure time goes up. I might got some electric issue as well but if you have any chance, it will be much more helpful you could conduct some test on your side. Thanks in advance.

Edward · May 18, 2023, 12:49am

@tianyi.lu

Yes, this is the case, and I wrote documentation about it:
https://docs.arducam.com/Raspberry-Pi-Camera/Global-Shutter-Camera/external-trigger/#external-trigger-example

I don’t know why this phenomenon occurs now, and you can use software to solve this problem at present.