ESP32 Wake Word Detection: Offline Listening That Works

ESP32 wake word detection lets a battery-friendly microcontroller sit in a room, listen continuously, and wake the rest of your product only when someone says the magic phrase, all without sending a single byte of audio to the cloud. That one capability solves three product problems at once: privacy, because raw audio never leaves the device; latency, because detection happens on the chip instead of after a network round trip; and cost, because there is no speech API bill. Espressif ships the whole stack through ESP-Skainet and the ESP-SR library under its own MIT-style license, free to use on Espressif hardware, with pre-trained wake word models included, and this post walks through how it works, which hardware to pick, how custom wake words happen, and what to watch before you ship. Source

How WakeNet detects a wake word

Espressif's wake word engine is called WakeNet, and its design tells you exactly what it is optimized for. According to the ESP-SR documentation, WakeNet is a neural network engine built for low-power embedded MCUs, and it supports up to five simultaneous wake words. Audio enters as 16 kHz mono 16-bit PCM, the engine extracts MFCC spectral features over 30ms frames, and a small convolutional network scores each frame. For a continuous stream, WakeNet averages recognition results over several frames into a smoothed prediction and only fires when that value crosses a set threshold, which is the main defense against false triggers from TV audio or conversation. Source

Model choice is per chip. The current WakeNet9 and WakeNet9l models run on ESP32, ESP32-S3, and ESP32-P4 using a dilated convolution structure, with 9l tuned for wake words spoken at very fast speeds. WakeNet9s targets the smaller ESP32-C3, C5, and C6 parts using depthwise separable convolution to fit their tighter budgets. WakeNet1 through 8 are out of use, so any tutorial referencing them is stale. Source

In code you rarely call WakeNet directly. It is built into the Audio Front-End (AFE), which is enabled by default, and detection results come back through the AFE fetch interface. The AFE itself bundles acoustic echo cancellation, voice activity detection, blind source separation, and noise suppression, and Espressif's two-mic AFE has been qualified as a software audio front-end for Amazon Alexa Built-in devices. You can disable WakeNet at runtime with disable_wakenet and re-enable it later, which matters for power management. Source

Hardware: pick the S3 and a real mic path

ESP-Skainet's own README is blunt about hardware: the ESP32-S3 is recommended for speech recognition because it supports AI instructions and high-speed octal SPI PSRAM, and the newest models deploy on the S3 first. Supported boards include the ESP32-Korvo, ESP32-S3-Korvo-1 and Korvo-2, and the ESP-BOX line, all of which integrate proper audio input modules. The input stream can come from a microphone or from wav and pcm files in flash or on an SD card, which is handy for repeatable testing. Source

The practical lesson: do not prototype wake word detection on a bare dev board with a random analog mic. Wake word accuracy lives and dies on the audio path, and the Korvo boards exist precisely because mic array layout, echo cancellation, and gain staging are most of the problem. Start on a supported board, get the stock examples running, and only then port to custom hardware.

Stock wake words and custom ones

Out of the box, Espressif provides free wake words such as "Hi, Lexin" and "Hi, ESP". For a branded product, you want your own phrase, and Espressif supports custom wake words through its Speech Wake Words Customization Process rather than public self-serve training. Plan for that lead time in your schedule; it is a service engagement, not a weekend script. Source

Start with the wake_word_detection example in the esp-skainet repository rather than writing your own pipeline. The example wires the AFE, WakeNet, and the ring buffers together correctly, and your first milestone should be boring: flash it to a Korvo board, say the stock wake word, and watch the detection log. Only after that baseline works should you swap in your own wake word or adjust thresholds, because a working reference makes every later regression debuggable. Source

Whatever phrase you choose, a few properties make wake words work well in practice. Three to five syllables with varied phonemes trigger more reliably than short or vowel-heavy phrases. Distinctiveness from common speech matters more than cleverness, because every phonetic overlap with everyday words is a false-trigger source. And since WakeNet supports up to five wake words, you can ship a primary phrase plus alternates, but each extra word spends part of your false-accept budget, so test them as a set.

From wake word to action

A wake word by itself does nothing; its job is to gate the expensive thing. ESP-Skainet's natural pairing is MultiNet, the offline speech command recognition model. MultiNet lets you add your own commands without retraining the model and currently supports up to 200 Chinese or English commands, such as turning on a light or an air conditioner. The classic architecture is: WakeNet listens always, a detected wake word activates MultiNet for a short window, the command maps to a local action, and the device goes back to low-cost listening. Source

For products that need real conversation rather than fixed commands, use the wake word as the privacy-preserving trigger for a cloud-assisted agent instead: the device listens offline, and only after the wake word does audio stream to your backend. Our ESP32 AI voice agent post covers that cloud-connected architecture; the two patterns combine cleanly, with WakeNet as the local gatekeeper in front of the network path.

Production notes: power, noise, and protection

Always-on listening is a power decision before it is a feature. A WakeNet pipeline that never sleeps will dominate the battery budget of a portable device, so duty-cycle aggressively: keep the mic and AFE running at the lowest workable rate, use the runtime disable and enable calls around periods when listening is pointless, and lean on deep sleep for everything downstream of the wake event. Our guide to ESP32 deep sleep power consumption has the numbers for the sleep side of that budget.

Test in the acoustic conditions you ship into, not the bench. Kitchens, TVs, and kids are false-trigger generators, and the smoothing threshold in WakeNet is tunable for exactly this reason. Record real wav files in target environments and replay them through the SD card input path for repeatable regression testing when you change firmware. Source

It also helps to define acceptance numbers before tuning. Decide the false-accept rate you can tolerate, for example one accidental trigger per day of typical room audio, and the detection rate you need at one meter and three meters with normal speech. Then tune the WakeNet smoothing threshold against recordings until both hold. Teams that skip this step ship a device that either ignores users or wakes up at the television, and both failures look identical in a one-line bug report. Source

Finally, treat your custom wake word model as intellectual property. It ships in flash alongside your firmware, so the same protections apply as for any firmware asset: secure boot and flash encryption keep a competitor from lifting your trained model off the chip. See our post on ESP32 flash encryption and secure boot for the setup.