Can Home Assistant voice run on the same box as the Zigbee coordinator without missing button presses?

Yes on a mini PC that still has idle CPU when someone speaks, and no on a Raspberry Pi that is running local speech-to-text and the Zigbee stick at the same time. Button presses go missing when the coordinator’s serial loop stalls or when USB noise from the voice hardware hits the stick. The voice satellite in the room is not the problem. The heavy pipeline on the same board as the radio is.

A button is a tiny Zigbee frame. It has to be read when it arrives. Whisper, Piper, and a wake-word model are the opposite kind of work: a burst of CPU, sometimes a burst of USB audio, for a second or two. If that burst is on the same Pi as ZHA or Zigbee2MQTT, the press during the sentence is the one that vanishes. The light automation never starts. The conversation still gets an answer, so you blame the button.

What is actually sharing the box

The coordinator is a USB dongle or a serial device. Its process has to poll that port steadily. Home Assistant’s Assist pipeline, if you run speech-to-text locally, is a model on the CPU or the iGPU. Those can share a machine the way a web server and a database can share a machine: fine until the one spikes. On an Intel N100 with the stick on a short USB 2 extension, a small model, and one voice satellite, presses keep arriving while the pipeline runs. I would still watch one evening of button logs before I trust it. On a Pi 4 with Wyoming Whisper and a Sonoff stick in a neighboring USB 3 port, I would not trust it. The misses show up as Zigbee events that exist in the air and never in the log, clustered in the seconds after a voice command.

The satellite is allowed to be in the room. An ESP32-S3 or a dedicated mic satellite does the wake word and ships audio to the server. That audio path is Wi-Fi. It does not need to be the Zigbee stick. People stack them because there is one home server and they want one plug. One plug is fine when the server is bored. Voice is the feature that stops it being bored, in spikes, which is the worst shape for a radio poll loop.

The two different misses

CPU stall is the miss during speech. The button works all day and fails while Assist is transcribing. The fix is a lighter model, a faster box, or moving speech-to-text off the coordinator’s host. You do not fix it by re-pairing the button. Re-pairing helps a bad parent. It does not help a host that was busy.

USB noise is the miss that does not care if anyone is speaking. A USB 3 device, a sound card, a Coral, or the stick plugged into the rear ports next to them, drops frames whenever the bus is active. Voice makes the bus active, so it looks like a voice bug. Move the coordinator to an extension cable away from the case and away from USB 3. If the misses stop even while you talk, it was noise. If they only stop when you disable local speech-to-text, it was CPU. Those are different afternoons, and the extension cable is the one to do first because it is cheap and it helps both.

Zigbee2MQTT in a container and the voice container on one Docker host share the CPU scheduler. Give the coordinator container a little more guaranteed CPU if the host is a mini PC and you insist on sharing. Do not do that on a Pi and call it a solution. There is not enough CPU to guarantee. A Pi that must run the stick should not also run Whisper. Use a cloud speech-to-text only if you accept audio leaving the house, or run the model on another machine and point Wyoming at it. The stick stays bored. Bored is what a button needs.

Who should split them

Split them if the box is a Pi, if button automations are how the rooms work, and if you want local speech. Put the coordinator where it already is, stable, and run the voice pipeline on the mini PC, or the other way around if the mini PC is the one with the stick and the Pi was a mistake. A voice satellite does not require the coordinator to move. It requires the heavy model to move.

Keep them together if the host is a recent Intel mini PC, the stick is on an extension, and a test evening shows button events in the log during a voice command. Press the button while a sentence is being recognized, ten times. Ten arrivals means the share is fine. A miss means you do not get to keep the diagram. Move the model. The buttons were there first, and they are the ones that have to work when the assistant is the thing using the CPU.

How to test without rebuilding the house

Turn on debug logging for the coordinator for ten minutes, not for a day. Press a button that should toggle a light, while nobody is talking. Then press it again while a voice command is in the middle of being recognized. You want the Zigbee event in the log both times, and the light both times. If the quiet press works and the talking press does not, the pipeline is the stall. If both fail, stop looking at Whisper and move the USB stick. A test that only happens in silence will tell you the install is fine and then fail at dinner, which is when people use both features.

Wake word on the satellite can stay. It is a small always-on detector on a different chip. The server should not also be running a second wake word on a microphone plugged into the coordinator’s host unless you have CPU to spare and you have done the USB test. Two wake words are how you get double replies and a stick that never quite idles. One satellite, one pipeline, the stick on a cable. That is the shape that keeps presses.

If you already miss presses and you have not added voice, voice is not the cause. A far button, a bulb used as a router, or a stick inside a closed metal case will miss with the assistant uninstalled. Add voice only after a quiet day of presses is clean. Otherwise you will debug the wrong subsystem for a week. The assistant is allowed to be the new load. It is not allowed to be the excuse for a radio that was already dropping frames.

Keep the coordinator’s USB path stable across reboots. A by-id serial device, not a tty that swaps when the speaker dongle enumerates first. Voice hardware that reorders the serial ports will point Zigbee2MQTT at the sound card after an update, and every button will miss until you notice the log is nonsense. That looks like a voice conflict. It is a device path. Pin the path. Then run the ten presses again. The box can host both only when those presses land while the model is busy, and not before. If a press takes two seconds to show up, treat that as a miss. A late hallway light is the same failure as a lost one, just ruder, because you already flipped the dumb switch. The log line has to be immediate enough that the automation feels like the button. Anything slower means the coordinator waited behind the model, and the share is not actually working.

Satellites that talk to a Wyoming endpoint on another host keep this simple. The Home Assistant box can still display the pipeline and play the reply. The GPU or CPU that runs the model can be the other machine, the one that does not own the stick. Button presses then have nothing to wait for except the radio, which is the only wait they ever needed. That split is the one I would ship in a house where the buttons matter more than the novelty of a single server doing every job. One box is a convenience. A missed press is the cost of finding out the convenience was too small. Leave a note in the dashboard that says where the model runs and where the stick is plugged in. The next person who adds a microphone dongle will otherwise plug it into the free port next to the coordinator and recreate the miss you just fixed. The free port is not free if it shares a controller with the radio. Use the extension. Keep the model off the Pi. Press the button while the house is talking, and only then call the box shared. If that press is late, they are not sharing. They are taking turns, and the button is the one that waits. Move the model before you move the stick again. The radio was fine until the pipeline got loud.