OpenAI launches three realtime voice models in the API

OpenAI launched three realtime audio models in its API on May 7. GPT-Realtime-2 handles conversational reasoning and tool use, GPT-Realtime-Translate performs live speech translation, and GPT-Realtime-Whisper streams speech-to-text while the speaker talks. All three became available through the Realtime API.

The models separate three production jobs

GPT-Realtime-2 targets voice agents that must keep context, recover from interruptions, call several tools, and explain audible progress. The release adds configurable reasoning effort and expands context from 32K to 128K. Realtime-Translate focuses on speech-to-speech translation across the language set documented by OpenAI. Realtime-Whisper provides continuous transcription rather than waiting for a completed recording.

That separation lets developers choose a focused path instead of routing every voice task through one general model. It also creates composition choices: a product may use direct speech-to-speech for conversation, transcription for audit or search, and translation only when participants cross language boundaries.

Evaluate the whole spoken interaction

Text accuracy is only one dimension of a voice product. Teams should measure time to first audio, interruption recovery, tool-call latency, repeated or dropped words, proper-name handling, and behavior when a speaker corrects themselves. Translation needs bilingual review for meaning, register, and named entities. Transcription needs tests with noise, overlapping speech, accents, and domain vocabulary.

Tool transparency also deserves a contract. Audible preambles should state what is happening without claiming success before a tool returns. If the model or network fails, the interface needs a recoverable state rather than silence. Record user consent and retention policy before storing audio or transcripts, and keep model, voice, reasoning effort, and session configuration with evaluation results.