I need to livestream video from the IMX298 camera module on RaspberryPi 4. I have written a simple program based on the video2stdout.c example from MIPI_Camera SDK. I added ability to get input from stdin and modify camera parameters (exposure, gain, focus…) and it is partially working but I have few problems.
Firstly, for video streaming I need constant framerate output from the encoder. I send the H264 encoded video through pipe into ffmpeg process where audio is added and then streamed with RTMP. I observed that the encoder output framerate depends on exposure value and because the framerate varies it doesn’t work nicely with the rest of the system. Is it possible to set the encoder to output a nice constant framerate H264 encoded stream? I tried outputting RAW video and then encode it with h264_omx encoder in ffmpeg but it was really slow.
The second problem is with camera parameters setting after longer period of streaming. After streaming for 8 hours I tried to change exposure and separate color gains but it didn’t work. Then I tried exitting the program and I just got a message something like “failed to write register” and “failed to close camera”. The video also seemed to be stuck. In my application I need the video output to be stable for longer periods of time (24h at least).
Is my goal achievable with this camera module and the SDK? The process of getting this to work was not really user friendly so far.
During experiments with camera settings I was unable to get a good colors from the camera. The image was always oversaturated and the contrast was too large. Darker parts of the image are just black and brighter are too bright. In outdoor environment it does not matter that much but indoors the camera is not usable. Even my old smartphone camera has better colors in video. Sadly if I open the output video in color grading software the color levels cannot be improved because the darker or brighter parts of the image are missing all the details. How can this be solved? We wanted to use this camera in a larger project as small and portable video streaming cameras but we will have to reconsider if the image quality issue couldn’t be solved.
I have been tinkering with the camera module and SDK for a while and I partially solved my problems. By tweaking camera parameters and using gamma filter on the output data I managed to get much better colors. The problem with variable framerate could be solved by getting raw video data (not H264 encoded).
Is it possible to get a raw video data from the camera module with 1920x1080 resolution at 30 fps with full 10bit colors? What is the format of the output data from raw_callback?
I got the yuv_callback almost working. The only problem is that the color planes are offset to side (screenshot https://imgur.com/a/fP84Lo2). What could cause this issue? The yuv420p output format is usable for us but it loses some of the image information so the best would be to get the raw 10bit video data to be able to do some processing on the video.
I have been tinkering with the camera module and SDK for a while and I partially solved my problems. By tweaking camera parameters and using gamma filter on the output data I managed to get much better colors. The problem with variable framerate could be solved by getting raw video data (not H264 encoded).
Is it possible to get a raw video data from the camera module with 1920x1080 resolution at 30 fps with full 10bit colors? What is the format of the output data from raw_callback?
I got the yuv_callback almost working. The only problem is that the color planes are offset to side (screenshot https://imgur.com/a/fP84Lo2). What could cause this issue? The yuv420p output format is usable for us but it loses some of the image information so the best would be to get the raw 10bit video data to be able to do some processing on the video.
the raw_callback return raw10 data. I think it is what you want to get.
I have updated our imx298 configuration and it can up to above 60 fps. Please download our new MIPI_Camera library and run make install to install newlibarducam_mipicamera.so
Yes, I understood that from the example code but what does raw10 mean? There are many raw pixel formats. For example, pixel could be structured as RGGB or GBRG or RGB etc.
In ffmpeg there are these pixel formats for raw bayer input:
bayer_bggr8
bayer_rggb8
bayer_gbrg8
bayer_grbg8
bayer_bggr16le
bayer_bggr16be
bayer_rggb16le
bayer_rggb16be
bayer_gbrg16le
bayer_gbrg16be
bayer_grbg16le
bayer_grbg16be
I tried all of them in combination with the arducam_unpack_raw10_to_raw8 or arducam_unpack_raw10_to_raw16 functions but without success.
I would like to know the exact specifications of the output data. For example: for resolution 1920x1088 the output data could be an array of 1920*1088=2088960 pixels and each pixel consists of 4 bytes representing colors in order red, green, green, blue.
This is the information I need.