Voice UX that feels alive: endpointing, barge-in, and latency
A voice interface feels alive or dead based on a few mechanics: how it decides you finished speaking (endpointing), whether you can interrupt it while it talks (barge-in), how quickly the first spoken words arrive, and whether it ever fakes a state. These matter more than voice quality, and each one is a design decision you can evaluate in any product.
Why some voice UIs feel dead
The failures are consistent across products. The assistant cuts you off mid-sentence because its silence threshold was too eager. It talks over you and cannot be stopped, so you sit through a paragraph you no longer want. It goes silent for seconds and then delivers a wall of speech. It animates a pulsing listening state while nothing is actually listening. None of these are voice-quality problems. All of them are turn-taking problems, and turn-taking is what makes conversation feel like conversation.
Endpointing: deciding when you finished
Endpointing is the system's guess about when your utterance ended. Hands-free systems mostly guess from silence: no speech for some threshold means you are done. The threshold is a trap in both directions. Too short and the system cuts off anyone who pauses to think, which trains people to speak in unnatural rushed bursts. Too long and every exchange gains dead air, which makes the product feel slow even when recognition is instant.
- Fixed silence timeouts are simple and wrong at the edges: fine for short commands, hostile to anyone thinking out loud.
- Adaptive endpointing uses the shape of what was said so far to decide whether a pause is a comma or a period; better, and still a guess.
- User-controlled endpointing removes the guess entirely: the user holds a button while speaking and releases it when done.
Violet's companion takes the third path. The mic button is hold-to-talk: pressing it starts a recognizer, releasing it ends the turn. Nobody gets cut off mid-thought, because the system never guesses where the thought ended. The tradeoff is stated plainly: this is a hands-on interaction, chosen for a device that is already in your hands.
Barge-in: the interruption contract
Barge-in means the user can interrupt the assistant while it is speaking, and the assistant yields immediately. It is the interruption contract of human conversation, and its absence is why talking assistants can feel like answering machines. The rule worth copying: any new user input, in any modality, silences the assistant now.
In Violet's companion this is implemented in both directions. Pressing the mic button cancels any spoken reply before listening starts. Typing does the same: the moment you start typing into the composer while Violet is speaking, the speech stops. Signing out also silences her. There is no state where she keeps talking over you.
One implementation detail worth knowing if you build this on the web: when speech synthesis is cancelled mid-utterance, Chrome fires the utterance's error event instead of its end event. If your code only resets its speaking flag on end, an interrupted reply leaves the UI convinced it is still speaking. Violet's client wires the same handler to both events.
Latency: time to first word
Users judge a voice reply by how quickly it starts, not how quickly it finishes. A reply whose first sentence begins in under a second feels attentive even if the full answer takes ten. The standard craft win is per-sentence synthesis: as reply text streams in, split it at sentence boundaries and hand the first sentence to the synthesizer while the rest is still arriving. The speech plays continuously while generation catches up behind it.
Honest note on where Violet stands: her web companion currently speaks a reply after it has finished streaming as text, in one utterance. The reply is on screen and readable while it streams, so the text path carries the early feedback, and the spoken path is a deliberate, simple layer on top. Per-sentence synthesis is the known next step of this craft, not something the shipped client already does. That is exactly the kind of limit an honest product states instead of implying.
Push-to-talk versus continuous listening
| Property | Push-to-talk | Continuous with wake word |
|---|---|---|
| Who ends the turn | The user, by releasing the button | The system, by guessing from silence |
| Microphone exposure | Live only while held | Standby detector always running |
| Cut-off risk | None; pauses are free | Real; thresholds cut off slow thinkers |
| False activations | None | Possible; TV and lookalike phrases trigger it |
| Hands and eyes | Requires a hand | Free |
| Honest privacy story | Simple to state and verify | Requires explaining standby, buffers, retention |
In Violet's hold-to-talk flow, the interim transcript paints live into the composer as you speak, so you watch her hear you in real time. On release, the final transcript is sent through the same send path as typed text: voice is an input method, not a separate system with separate rules. If nothing was recognized, the composer is restored to exactly what you had typed before the hold. The keyboard can drive the same interaction for accessibility: holding Enter or Space on the mic button talks, releasing sends.
Spoken confirmations and consequence
When a voice command changes something, the confirmation should match the weight of the change. For undoable actions, act first and confirm briefly afterward: a short spoken line naming what happened, with an easy undo. Making a user verbally approve every reversible change turns conversation into form-filling. For consequential actions, the order reverses: nothing happens without explicit approval first, and the approval mechanism should live outside the model rather than inside its judgment. This split is a design law in Violet's architecture: her confirmation gate for consequential actions is enforced outside the model, and the product's standing rule is that nothing happens without your say.
Honest failure states
Voice runs on capabilities the platform may not grant, and a voice UI earns trust by how it behaves when they are missing. Violet's companion treats each failure as something to say plainly rather than mask:
- No speech recognition in the browser: the mic button stays visible but says so on press. It never shows a fake listening state.
- Microphone blocked by the browser: a plain message says the site is blocked and how to allow it.
- No microphone found: the message says exactly that.
- No speech synthesis: the voice-replies toggle stays hidden and replies fall back to text, silently and honestly.
- Spoken replies are off by default on every device, and the preference is stored locally once you change it.
Common mistakes that kill voice UIs
- No barge-in: the assistant cannot be interrupted, so users learn to dread long answers.
- Eager endpointing: cutting people off mid-pause, which makes them rush and resent the product.
- Speaking only complete replies when text could stream or the first sentence could play early, so every answer starts with dead air.
- Fake states: pulsing listening animations with no live recognizer behind them.
- Verbal confirmation prompts for trivially undoable actions, which makes conversation feel like a phone tree.
- Voice as a separate silo: commands that behave differently from the same request typed, which breaks the user's model of the product.
Questions
What is barge-in in a voice interface?
Barge-in is the ability to interrupt the assistant while it is speaking and have it yield immediately. Good implementations treat any new user input as an interruption: new speech, a mic press, or typing. Without barge-in, users must wait out every reply, which is the fastest way to make a voice UI feel dead.
What is endpointing and why do assistants cut people off?
Endpointing is the system's decision about when you finished speaking. Hands-free systems usually infer it from silence, and a threshold short enough to feel responsive is often short enough to cut off someone pausing to think. Push-to-talk avoids the guess entirely: the user ends the turn by releasing the button.
Does Violet speak her replies?
Only if you turn voice replies on; the toggle is off by default and stored per device. She speaks using the browser's own speech synthesis, stops the moment you type or press the mic, and where the browser has no synthesis the toggle stays hidden and replies remain text. Violet is pre-launch.