How We Got Real-Time Voice Agent Latency Under 800ms
Overcoming the acoustic pause: WebRTC, speculative inference, and streaming TTS buffers.
Turn latency is the single factor that decides whether a voice agent feels natural or frustrating. Here is the architecture we use to shave 400ms off standard telephony pipelines.
In voice conversational agents, callers abandon on silence, not errors. If a human asks a question and hears nothing for 1.4 seconds, they assume the call dropped or repeat themselves, interrupting the model.
We achieved sub-800ms turn-taking by discarding standard HTTP REST cascades and switching to full-duplex WebRTC with audio frame chunking.
By streaming transcription tokens straight into the LLM context window while the user is still speaking the final syllables, the model starts token prediction before acoustic silence detection even fires.
Combined with edge-rendered audio synthesis buffers, our voice agents interrupt gracefully and feel responsive.