OpenAI reveals GPT-Live's continuous inference and asynchronous delegation design

OpenAI explained how it rebuilt voice infrastructure for GPT-Live on August 3. The full-duplex voice model can listen and speak simultaneously, removing a separate turn detector from the critical audio path. Deeper reasoning and tools run through asynchronous delegation, allowing the spoken exchange to continue while another model works.

Audio gets a dedicated fast path

Incoming and outgoing media stream continuously between the client and voice model. Application logic, tools, persistence, and delegated reasoning sit behind an asynchronous RPC boundary, so their latency does not directly stall audio frames. OpenAI rewrote the media frontend and inference logic in Go from a Python asyncio implementation and reports that the new system’s high-percentile frame delivery matched the former median. WebRTC handles transport under packet loss, drift, and connection changes.

Stateful sessions add a different failure problem. To move a long conversation between model instances, the system warms a replacement, prefills it with context, runs both temporarily, and cuts over after the new instance is ready. The same mechanism supports context compaction: the active instance keeps talking while compacted context and a new key-value cache are prepared elsewhere.

Realtime budgets cross model boundaries

Asynchronous delegation is not free. A search or reasoning result must still return soon enough to remain relevant to the conversation. OpenAI pre-creates and prefills the delegated model session, keeps affinity across requests, and tunes reasoning effort, output limits, and tool round trips.

Voice-agent builders should separately measure media gaps, time to first useful delegated result, handoff failures, context drift, interruption behavior, and recovery after worker loss. Keeping the live path small improves isolation, but users experience the system as one conversation; consistency across its parallel paths remains a product requirement.