esp32-cam object detection: Three Architectures That Ship

esp32-cam object detection is one of those projects that looks simple in a demo and gets complicated the moment you ask which chip is actually doing the detecting. The honest answer is that there are three working architectures, and they are not interchangeable: a lightweight detector running on the ESP32-CAM board itself, a YOLO-class model running on newer ESP32 silicon with AI acceleration, and an off-device setup where the board only captures and streams images while a computer does the detection. Source

This post walks through all three, with the real numbers and constraints from Espressif's esp-detection project, Edge Impulse's FOMO documentation, the EloquentEsp32cam library, and a working off-device build documented by Maker Guides. The goal is to help you pick the right architecture before you order parts, not after.

First, be clear about which "ESP32-CAM" you mean

The classic ESP32-CAM, the AI Thinker style board with a camera connector that most tutorials use, is built around the original ESP32 chip. It has no NPU and no vector instructions for neural network math, and it shows: anything beyond a small, specialized model will not run at usable speed on it. Newer boards in the family, like the ESP32-P4 and ESP32-S3, are a different story, and much of the current Espressif tooling targets them. Source

This distinction trips people up because the P4 gets discussed in the same breath as camera projects. As we covered in our ESP32-P4 speech recognition post, the P4 has real ML muscle but no NPU either; it gets its speed from a faster core and vector extensions. The practical consequence: when a project claims "real-time object detection on ESP32," check which ESP32 before you believe the frame rate.

Path 1: FOMO running on the classic board

Edge Impulse's FOMO (Faster Objects, More Objects) is the most credible way to run detection on the classic ESP32-CAM itself. FOMO trades bounding boxes for centroids: instead of drawing a rectangle around each object, it tells you where each object's center point is and how many objects it found. Edge Impulse says this design uses up to 30 times less processing power and memory than MobileNet SSD or YOLOv5, and demonstrates 30 fps on an Arduino Nicla Vision, a Cortex-M7 class microcontroller, using 245K of RAM. Those are Edge Impulse's own figures, measured on their chosen demo setup, but the underlying point is structural: centroid detection is fundamentally cheaper than full bounding-box regression. Source

On the firmware side, the EloquentEsp32cam Arduino library wraps the whole pipeline. Its Edge Impulse integration lets you capture a frame and run a FOMO model in a few calls: fomo.run() executes inference, fomo.foundAnyObject() checks the result, and fomo.first gives you the top detection with its label and coordinates. The README includes a no-PSRAM example, which matters because PSRAM-less ESP32-CAM variants are common in the wild. The library also handles the unglamorous parts, camera pinouts, brownout protection, resolution settings, that usually eat a weekend. Source

The limits are real. You get centroids, not boxes, so size and orientation are gone. You train on your own images through Edge Impulse, so a custom model means a data collection session. And the model stays small and specialized: this is "detect my product on a conveyor" territory, not "detect any of 80 everyday objects."

Path 2: YOLO-class detection on newer ESP32 silicon

If you want real bounding boxes on-device, Espressif's esp-detection project is the reference implementation. It is built on Ultralytics YOLOv11, produces ultra-lightweight single-class models, and deploys through ESP-DL. The headline model, ESPDet-Pico, has 0.36 million parameters, and Espressif reports it matching the 80-class YOLOv11n (2.6 million parameters) on cat detection accuracy, a single-class versus single-class comparison. Espressif's published numbers: over 18 fps on ESP32-P4 and over 7 fps on ESP32-S3 at 224 by 224 input, with the 224 cat model scoring 69.9 mAP (0.5:0.95) on the cat subset of COCO val2017. Source

Read those numbers carefully, because they are the most-quoted and most-misread in this niche. The 18 fps figure is on ESP32-P4, not on the classic ESP32-CAM. The accuracy claim is for a single-class model against a single-class slice of COCO, not general-purpose detection. And the project ships examples for cats, dogs, and pedestrians: useful templates for training your own single-class model, not a ready-made detector for arbitrary scenes. Source

The project is actively developed, with its 1.0.0 release in April 2025 and a non-square-resolution training update in June 2025, and it carries an AGPL v3 license, which is worth checking against your product plans before you build on it. Source

Path 3: off-device detection, the board as a camera only

The third architecture gives up on on-device inference entirely and often wins anyway. The ESP32-CAM does what it does well, capture JPEG frames and serve them over WiFi, and a computer on the network runs a full detector on the stream. Maker Guides documents a complete build of this pattern: the board runs a small web server that hands out snapshots over HTTP, and a Python application on a PC pulls the images, runs YOLO, and annotates results across 80 object classes. Source

The trade-offs flip around. You get full-size models and full bounding boxes with no embedded constraints, at the cost of needing the network hop and an always-on machine. Latency depends on your WiFi, not on a model quantization budget. And the board side becomes almost trivial, which is why this pattern shows up so often in prototypes that later migrate on-device once the model and the target silicon are chosen. Source

One security note from our own coverage habits: an ESP32-CAM streaming JPEG on your LAN is another agent-adjacent device on the network, and the same discipline we described for ESP32 MCP servers applies. Keep it on a segmented network, and treat its web interface as an attack surface, not a convenience.

How to choose, in our assessment

This section is our read, separate from the sources. The decision comes down to three questions. First, do you need bounding boxes, or is "something is here" enough? Centroid answers favor FOMO on the classic board. Second, are you committed to the classic ESP32-CAM hardware, or can you pick silicon? If you can choose, a P4 board with esp-detection gets you real boxes on-device, and our ESP32 wake word detection post covers the same "match the model to the silicon" pattern in audio. Third, is there always a computer nearby? If yes, off-device detection is the fastest path to something working and the easiest to iterate on.

Our default advice: prototype off-device, because you learn what your model actually needs to detect without fighting firmware. Move to FOMO on the classic board when the use case is a small, fixed set of objects and cost per unit matters. Choose P4-class hardware with esp-detection when you need boxes on-device at interactive frame rates and can design the board in. What we would not do is start by buying classic ESP32-CAM boards on the assumption that YOLO will somehow run on them, because the published numbers that suggest it are measured on different silicon.