What is Voice agent?

OpenAI's voice agents guide treats the main design question as how speech connects to reasoning and tools. It describes three architectures, and the choice affects how much control a developer has and how much there is to build.

In the speech-to-speech design, one model interprets audio, decides what to do and replies in speech inside a single session, using the Realtime API. That session handles turns, tools, interruptions and handoffs. The page describes it as speech, reasoning and tool use in one session, where one model interprets audio, decides what to do and responds in speech.

In the chained design, the application runs speech-to-text, then an agent workflow, then text-to-speech. This lets a team store transcripts, run policy checks, call internal systems and swap each component independently, since the application manages the speech-to-text, agent workflow and text-to-speech stages itself.

A further option, GPT-Live, is described as a full-duplex model that can listen and speak at the same time. It handles the spoken exchange and passes reasoning and tool use to a separate backend, which can be an existing text workflow.

The guide advises picking the audio architecture first, then designing the rest as for text. For GPT-Live, the guide suggests measuring task and tool outcomes, conversational timing, speech and language quality, and session reliability independently.