wakenet custom cnn model: How Custom Wake Words Ship on ESP32

A wakenet custom cnn model is what you need the moment your product's wake word is not "Alexa" or "Hi Lexin". WakeNet is Espressif's neural-network wake word engine for ESP32-class chips, and while Espressif ships pre-trained models for a handful of phrases, any brand building a voice-first device eventually wants its own name on the trigger phrase. This post walks through what the model actually is, how Espressif's customization process works, what the corpus requirements cost you in effort, and the open-source alternative for teams that want to train their own. Source

What WakeNet actually is

WakeNet is a wake word engine built on neural networks, designed for low-power embedded MCUs, and it ships inside Espressif's ESP-SR speech recognition framework. A single WakeNet model currently supports up to five wake words. The pipeline starts with MFCC feature extraction: the input audio is 16 kHz, mono, signed 16-bit, and each analysis frame uses a 30 ms window and step size. Triggering is smoothed: the engine averages recognition results over several frames and only fires when that moving average crosses a set threshold, which keeps single-frame noise from waking the device. Source

The architecture has gone through nine generations. WakeNet1 through WakeNet8 are out of use. The current lineup splits by chip family: WakeNet9 and WakeNet9l target ESP32, ESP32-S3, and ESP32-P4 and are built on a dilated convolution structure, with 9l tuned to catch wake words spoken at very fast speeds. WakeNet9s targets the RISC-V parts (ESP32-C3, C5, and C6) and uses a depthwise separable convolution structure, which trades some capacity for a much smaller compute and memory footprint on chips without the S3's vector instructions. Source

At runtime, WakeNet sits inside the audio front-end (AFE), which handles echo cancellation, beamforming, and noise suppression before features reach the CNN. The AFE returns detection results through its fetch interface, and WakeNet can be switched off at build time (afe_config->wakeNet_init = false) or toggled at runtime with disable_wakenet and enable_wakenet calls, which is how battery devices gate listening windows. Source

How a custom WakeNet model gets made

Espressif runs wake word customization as a paid service, and its documentation lays out three routes. Route one is TTS-only: the model is trained entirely on text-to-speech samples, with no human recordings required. Route two combines Espressif's crowdsourced recordings, about 500 speakers, with TTS augmentation. Route three uses a corpus the customer provides, which must contain at least 20,000 qualified samples. For routes two and three, once the corpus is ready, Espressif says training and optimization usually takes two to three weeks. Source

The pricing model is worth noting because it is unusual for voice tech: Espressif charges a one-time customization fee and no license fee based on device count or period of use. The fee varies by method, and pricing is negotiated with their sales team. Espressif has also opened some of its existing wake words, such as "Hi Leixi" and "Nihao Xiaoxin", for commercial use at no cost, and says more free commercial-use wake words are planned. Source

There are hard constraints on the model itself. Each WakeNet model supports up to five wake words. A wake word usually consists of three to six syllables; the documentation's examples are phrases like "Hi Leixin" and "nihaotianmao". You can run more than one WakeNet model together, but every added model consumes more memory and compute, which matters on a part with a few hundred kilobytes of SRAM. Source

The corpus requirements, if you collect your own

If you go the customer-corpus route, the bar is specific. Audio must be 16 kHz, 16-bit signed integer, mono WAV. The speaker pool must exceed 500 people, balanced across gender and age, and include at least 100 children. Recording happens in a quiet room under 40 dB, ideally a professional audio room, on a high-fidelity microphone. Source

Each speaker records the wake word 30 times: 15 utterances at one meter from the microphone and 15 at three meters, each set split into five fast, five normal, and five slow readings. Espressif recommends encoding speaker metadata in the filename, for example female_age_fast_id.wav. Doing the arithmetic, 500 speakers times 30 utterances is 15,000 recordings, so the 20,000-sample minimum effectively means more speakers or augmentation on top. This is the step most product teams underestimate: corpus collection is a logistics project with acoustics requirements, not a weekend of recordings in the office. Source

Flashing and running the model

Once Espressif delivers a model, integration is a build-time exercise. You select the WakeNet model in idf.py menuconfig under the ESP Speech Recognition component, alongside the noise suppression model (nsnet2 for maximum suppression, nsnet3 for a balance between noise reduction and speech distortion), the VAD model, and the MultiNet command-recognition model. The build system packs the selected models into a binary that lands in a dedicated flash partition, and the docs instruct you to reserve space for it in partitions.csv, with their example allocating 6,000K. Source

The application code then interacts with the model through the AFE fetch loop rather than calling WakeNet directly, so the wake word arrives as an event in the same stream that carries the post-trigger audio for speech recognition or streaming to a cloud agent. Source

The open alternative: microWakeWord

For teams that want to train their own model without Espressif's service, the main open route is microWakeWord, the framework ESPHome adopts for on-device wake word detection. The training process lives on the microWakeWord GitHub repository, and trained models are referenced as JSON manifests: from ESPHome's official model repository, a GitHub shorthand URL, a full URL, or a local file. Source

The runtime knobs map closely to WakeNet's smoothing idea. Each model carries a probability_cutoff, the threshold below which a detection is rejected; raising it reduces false accepts but increases false rejections. A sliding_window_size averages probabilities over a window, where a smaller window lowers latency but admits more false accepts. An optional VAD model gates detection so non-speech sounds are less likely to trigger, and models can be enabled and disabled individually at runtime, with state persisted to flash. Source

The trade-off is honest: microWakeWord models are smaller and self-serve, and they power plenty of Home Assistant voice satellites, but you own data collection, training, and threshold tuning yourself, and the community models do not go through the multi-speaker, multi-distance corpus rigor Espressif specifies. We did not find published false-accept benchmarks that compare the two directly, so treat the choice as a build-versus-buy decision and validate on your own hardware in your own acoustic environment. Source

Hardware is half the model

Espressif is blunt that wake-up performance depends heavily on hardware design and cavity structure. Its guidance: start from Espressif's reference schematic and PCB, have acoustic specialists design the cavity (Espressif points at mainstream smart speakers as references and does not provide industrial design guidance), and get microphone placement relative to speakers right. Source

The verification plan Espressif lists is a useful checklist for any voice hardware team. Recording tests: play a 90 dB sample at 0.1 meters and adjust gain so recording does not saturate; run a 0 to 20 kHz sweep recorded at 16 kHz to catch frequency aliasing; record 100 samples and feed them to a cloud speech recognition API to confirm a target recognition rate. Playback tests check the power amplifier and speaker for distortion. Audio algorithm tests cover acoustic echo cancellation, beamforming, and noise suppression. Customers can also send one or two hardware units to Espressif for wake-word performance tuning on the specific device. Source

Our read

This section is our assessment, not Espressif's documentation. The decision that matters for a product team is not which CNN architecture the chip runs; it is which customization route matches your stage. TTS-only training is the fast path for a prototype or a niche phrase where you can tolerate some false accepts. The crowdsourced and customer-corpus routes are what you buy when the wake word is your brand and the device ships in volume, because the 500-speaker, two-distance, three-speed corpus is what makes detection robust across accents, ages, and rooms.

Two practical points line up with what we have written elsewhere. First, the wake word is only the front door of a voice pipeline; the audio after the trigger goes to your agent, and the design of that full path is what we covered in our ESP32 AI voice agent piece. Second, if you are evaluating chips, the WakeNet9s versus WakeNet9 split is a real constraint: the C-series parts run the smaller depthwise model, so phrase complexity and simultaneous wake words are cheaper on an S3 or P4. Our broader ESP32 wake word detection post covers the baseline engine behavior, and the ESP32-P4 speech recognition piece is the reference for the newer silicon.

Our advice in one line: prototype on microWakeWord or a stock WakeNet phrase, budget the Espressif customization fee and the two-to-three-week training window into your hardware timeline, and treat corpus and acoustics as first-class workstreams, not afterthoughts.