Hi, I’m using an Arducam 64MP camera on a Raspberry Pi CM4 (the module with 2x CSI-2 inputs). I’m doing ROI/digital zoom and as far as I can tell it’s handled by the RPi ISP, so the zoom is basically limited by the camera’s native resolution. I have modified libcamera-dev and libcamera-apps installed. The modes I’m working with are 4624x3472 @ 10fps and 2312x1736 @ 30fps, and also 4624x3472 / 9152x6944 for higher quality stills.
The problem: I want to use the full camera resolution for the zoom-in in the viewfinder/video stream as well, but I don’t think that’s supported right now. Currently for large zoom values I have to pick between either good native resolution with a horrible framerate (exposure time also seems high, so lots of motion blur), or a reasonable framerate with a pretty bad zoomed-in image.
What I’d ideally want: when a large zoom value is requested, switch to a higher camera resolution (that part works today) but then only query a specified ROI, so the sensor/ISP isn’t scanning and transferring the whole image when most of it gets thrown away by the digital zoom. That should let me get higher update rates. I’m not sure this is possible, probably not at the moment.
Also, I couldn’t find in the spec what temperature range the camera can be stored and used at – will it work below zero deg C?
Some earlier questions I had: can two 64MP cameras be connected to the two CSI-2 inputs of the CM4; is a 2GB RAM CM4 enough for viewfinder/video from both cameras plus capturing 64MP stills one by one; where do I get the modified libcamera source;
Right now I’m using the controls::ScalerCrop control, which I can set without restarting the camera, and I can read back the actual ROI on every CompletedRequest. I’m aiming to build a custom C++ application that references the modified libcamera implementation. No logs yet, happy to grab some if useful.
Thanks for the detailed write-up. A few things to get out of the way first, then we can dig into the zoom question.
Yes, two 64MP cameras on the two CSI-2 inputs of the CM4 works – we’ve managed to run that setup ourselves.
For two cameras plus 64MP stills, go for a bigger RAM module (4GB or 8GB). 2GB is going to be tight.
The source code is no different from the official libcamera; it’s open source, so you can just refer to the official repo.
Manual focus is supported – you set it by manually entering the command.
What we call “digital zoom” here is done by the RPi ISP; the modified libcamera vs the official one only differs in the ISP adjustments.
On your main point: your instinct is right. The ROI/digital zoom is executed by the Raspberry Pi ISP, not on the sensor, so the effective detail you get when zoomed in is bounded by the camera’s native resolution. That’s exactly why the 2312x1736 @ 30fps zoom looks so much softer than the 4624x3472 or 9152x6944 modes.
Can you confirm one thing for me – when you request a crop with ScalerCrop while in the lower-resolution 30fps mode, are you seeing the full sensor readout happen and then the ISP just crops, or is it cropping the already-downscaled stream? Also, which temperature range are you looking at (storage vs operating), and roughly how cold are we talking?
Here’s some evidence to show what I mean. First is the viewfinder at high zoom when the camera is in 4624x3472 mode @ 10fps:
And here’s the same zoomed-in image when the camera is in 2312x1736 mode with the more usable 30fps:
You can see the quality difference clearly. To answer your question – the zoom looks like it’s cropping the already-reduced stream, not doing a full readout and then cropping, which is why it degrades so much.
On the temperature side, I’m mostly interested in whether it’ll operate below 0 deg C and what the storage range is. Do you have those numbers?
Also, good news on my end: I confirmed I can set the controls::ScalerCrop control without restarting the camera, and I get the actual ROI back on every CompletedRequest. That’s exactly what I needed. That python example you have for controlling focusing is really helpful too – I want to build a custom C++ app and reference the modified libcamera implementation.
Those two screenshots make it very clear – thanks, that’s a good illustration of the limitation.
So to summarize where we are technically:
Root cause: digital zoom is performed by the Raspberry Pi ISP, not the sensor. Because the zoom is a crop of what the ISP is processing, its effective quality is limited by the camera’s native resolution. A lower streaming resolution (like 2312x1736 @ 30fps) therefore gives noticeably lower zoomed-in quality than 4624x3472 or 9152x6944. That matches exactly what you’re seeing in the two images.
On your “request only the ROI to avoid transferring the whole frame” idea – that’s the part that isn’t currently possible. The crop is done on the ISP side and the ISP still has to receive the full frame from the sensor, so you can’t skip the scan/transfer of the unused area yet. The way to keep image quality up when you need a big zoom is to stay on the higher-resolution mode (4624x3472 or 9152x6944) and accept the lower framerate.
For your custom C++ application:
To change the crop/ROI between frames, set controls::ScalerCrop – you’ve confirmed you can do this without restarting the camera.
Read back the actual ROI from each CompletedRequest (you’re already doing this).
Use the official (open source) libcamera source as your reference – our modified version only differs in the ISP adjustments related to digital zoom, so the API you’re coding against is the same.
Regarding below 0 deg C: which value do you need – operating or storage? Tell me the exact numbers you need and I’ll check the sensor module datasheet for you.
I’ve also sent you the focusing python example in a private message in case it got lost in the thread – check your messages.
That makes sense now – the whole frame still has to come off the sensor and the ISP does the crop, so there’s no way to only pull the ROI. That explains the tradeoff I was stuck on.
So my plan is: for large zoom values, stay in the high-res mode for quality, and use ScalerCrop when I need to move the window around between frames. I’ll build the C++ app against the official libcamera and just account for the ISP adjustments you mentioned.
One last thing on the temperature – I’ll take whatever operating range you can find, storage would be a bonus.
Marking the zoom part solved for me. Thanks for the help and the focusing example.
Sounds good, that’s the right approach for the zoom. Quick wrap-up of this thread:
Digital zoom / ROI runs on the RPi ISP, not the sensor, so zoomed-in image quality is bounded by the camera’s native resolution.
Because of that, streaming at a lower resolution (2312x1736 @ 30fps) gives much lower zoomed-in quality than the higher modes (4624x3472 or 9152x6944). Requesting only an ROI to avoid transferring the full frame is not currently possible.
To change crop/ROI between frames, set controls::ScalerCrop (no camera restart needed) and read the actual ROI from each CompletedRequest.
For manual focus, set it via the command, and for code, use the official open-source libcamera as reference – our modified version only adds the ISP adjustments related to digital zoom.
I’ll follow up on the operating/storage temperature range once I’ve pulled the datasheet. Glad the zoom side is sorted – post back if the C++ side gives you any trouble.
