TopolearnVoice apps need different building blocks from text apps, and choosing the right one decides latency and cost.
OpenAI covers speech in three ways:
- Speech to text (transcription): turn an audio file or live audio into text.
- Text to speech: turn text into spoken audio in a chosen voice.
- Realtime API: a live, two-way voice conversation where the model listens and speaks directly (speech to speech), with low delay.
A chained voice app runs transcription, then a normal Responses call, then text to speech. It is easy to build, debug and log, and you can use any text model, but each hop adds delay. The Realtime route feels more natural and handles interruptions, at the cost of a different, event-based programming model.
The Realtime API works over a long-lived connection:
- WebRTC (the browser technology for live calls) for apps on the user's device.
- WebSocket, a two-way connection that stays open, for server-to-server apps.
- SIP for phone calls.
You configure a session by sending a session.update event (model, voice, audio format, turn detection, instructions), then stream audio in and receive audio and text events back. Sessions can also call functions.
This area changes especially fast: the Realtime beta interface was shut down in May 2026, and older speech models are scheduled for retirement in 2027. Use the model names the audio guide lists today.
Example
import json
# A Realtime session is configured by sending events over an open
# WebSocket or WebRTC data channel. This builds the configuration event;
# see the Realtime guide for opening the connection itself.
event = {
"type": "session.update",
"session": {
"type": "realtime",
"model": "gpt-realtime-2.1", # check current model names
"output_modalities": ["audio"],
"audio": {
"input": {"format": {"type": "audio/pcm", "rate": 24000},
"turn_detection": {"type": "semantic_vad"}},
"output": {"format": {"type": "audio/pcm", "rate": 24000},
"voice": "marin"},
},
"instructions": "Speak clearly and briefly.",
},
}
print(json.dumps(event)[:80], "...") # ws.send(json.dumps(event))Remember
- Chained (STT -> text model -> TTS) is simple; Realtime is lower-latency speech to speech
- Realtime uses WebRTC, WebSocket or SIP and is event-based
- Audio model names change fast; check the audio guide