I think i am having the same problem as you
the camera will not fully focus (though it is capable of doing so in libcamera-hello)
ill try to recreate what you did