Discussion Thread: Raspberry Pi 5 AI Co-Processor

Announced today:
https://www.raspberrypi.com/news/raspberry-pi-ai-kit-available-now-at-70/

Further information:

What are everyone’s thoughts on this product?
Will it help with the processing of 64 MP images from ArduCam cameras?
Does the ArduCam cameras support use of the AI Co-processor?

What are everyone’s thoughts on this product?
I like it. I found out about it yesterday. It gives 13 tera operations per second (TOPS) vs 2 TOPS with Coral TPU u get for the same price. Also it has much lower power requirement. Coral TPU runs 2 TOPS per 1 watt of power but HAILO runs 8 TOPS per 1 watt of power.

Will it help with the processing of 64 MP images from ArduCam cameras?
yes but with AI stuff you probably want to use lower resolution for increased speed

Does the ArduCam cameras support use of the AI Co-processor?
yes

Futhermore you could use ToF (time of flight camera) Arducam camera that constantly sends out and measures how long it takes for light to travel for each pixel - it basically maps 3d environment around you that you can feed to AI for increased cognition.

1 Like

The main problem I have using a Raspberry Pi on my current project is it takes about 24 seconds to stitch the two 64 MP images together. Then about 20+ minutes to run AI / classification task on Raspberry Pi.

So if this gets that time down to something sensible (<10s for AI), then I won’t need to send GBs of images off the Raspberry Pi.

why do u need such high resolution?

I can’t disclose the exact use-case as this is a business I’m working with using these.

However, I am looking at objects more than 600-750 metres away. I tried with lower resolution (16 MP) sensors but it was not adequate. At that distance, the objects when using < 16 MP only appear to be 10-15 pixels high. Whereas, with the 64 MP sensor, I can see the objects are 4 times bigger - which leads to much better detections.

That’s the idea anyway. We haven’t tested the AI with real-world conditions and large/super-resolution images.

with combat drones you can just then add more capable hardware. with RPI+HAILO (70$) u get 13 TOPS if thats not enough then in other extreme there is nvidia A100 (10 000$) u get 624 TOPS. Theres also lots of intermediate solutions with lower prices:

  • Jetson AGX Orin:
  • Up to 275 TOPS (INT8 precision)
  • Jetson AGX Xavier:
  • Up to 32 TOPS (INT8 precision)
  • Jetson Xavier NX:
  • Up to 21 TOPS (INT8 precision)

I have a Jetson Orin Nano to test out as well.

Not sure how many TOPS it does.

  • Jetson Orin Nano 8GB:
  • Up to 40 TOPS (INT8 precision)
  • Jetson Orin Nano 4GB:
  • Up to 20 TOPS (INT8 precision)

chatgpt says

u could also apply scalercrop parameter to process a 16MP rectangle of the 64MP sensor. this way the field of view is 4x lower but the object appears same amount of pixels as in the 64MP image. let me know how did things work out and what kind of speed did you achieve cause im also curious about this