saved
How we built a realtime system for responsive voice AI in six months
Hraness cites a source capture. The source author remains the source.
gist
OpenAI engineers Justin Uberti and Zahan Malkani explain GPT-Live, a full-duplex voice system that drops the turn detector and streams audio in and out of a stateful voice model while frontier models and tools run asynchronously. Six months of work rebuilt inference, context compaction handoffs, a Go media frontend, WebRTC transport with WARP and Instant Connect, and silent production shadow tests so ChatGPT Voice stays sub-second responsive and can later power a GPT-Live API.
ideas
- Full-duplex removes the turn detector from the audio path. The voice model listens and speaks at once; deeper reasoning and tool use consult frontier models without stalling speech.
- Keep a dedicated live media path. Audio stays on a fast client-to-model loop written in Go over WebRTC; application logic and tools sit behind an async RPC boundary so slow backends cannot stall frames.
- Statefulness needs seamless handoffs. Long sessions grow context past limits; compaction and instance swaps warm a replacement in parallel so the conversation never gaps.
- Derive turns after the fact. UI, analytics, and safety still need discrete messages, so the application server segments overlapping speech into provisional then finalized turns without imposing turn-taking on the live path.
- Startup and capacity are first-class latency. WARP plus Instant Connect start media with one UDP packet; silent production tests reframed capacity around concurrent sessions that keep every frame on schedule.
quotes
“GPT‑Live puts the voice model in control of the conversation”
“Audio moves between the client and the voice model on a dedicated fast path.”
“the client can now start a session with a single UDP packet.”
“Bringing GPT‑Live to ChatGPT scale required a whole new system built around one fundamental principle: the voice must flow.”