Traditional voice agents are often built as a chain of separate components:

microphone
   ↓
speech-to-text
   ↓
LLM
   ↓
text-to-speech
   ↓
speaker

That architecture works, but it forces developers to coordinate buffers, turn detection, cancellation, latency, context synchronization, and the especially awkward case in which the user starts talking while the system is still answering.

With GPT-Live-1, OpenAI brings a full-duplex voice model to the API: it can process incoming audio while producing outgoing audio. The important change is not merely more natural speech. It is a different architecture for conversational applications.

OpenAI released GPT-Live-1 in the API on September 10, 2026 and conceptually separates the system into two layers:

user
  ⇅ audio
GPT-Live-1
  │
  ├─ conversation, rhythm, interruptions and turn-taking
  │
  └─ delegation
       ↓
reasoning backend / agent / tools

The voice layer no longer has to be the entire “brain” of the agent.

Full duplex: listening while speaking

In a cascaded pipeline, a reply is normally produced after several sequential steps. Every handoff adds latency and every component needs some notion of when a turn starts and ends.

GPT-Live-1 processes the interaction continuously. That helps it handle:

  • interruptions;
  • long pauses;
  • mid-sentence corrections;
  • short acknowledgements such as “yeah” or “mm-hmm”;
  • background noise;
  • intent changes while the system is speaking.

OpenAI reports a 30-percentage-point improvement over GPT-Realtime-2.1 on Full Duplex Bench. Its published measurements also show response latency of roughly 0.798 seconds versus 1.41 seconds for the previous model.

The practical difference is that some orchestration logic that used to live in application code now lives inside the interaction model itself.

GPT-Live-1 can delegate complex work to another model or agent while it keeps the conversation moving.

A typical architecture can look like this:

                   ┌──────────────────┐
microphone ───────▶│                  │──────▶ speaker
                   │    GPT-Live-1    │
                   │                  │
                   └────────┬─────────┘
                            │ delegation
                            ▼
                   ┌──────────────────┐
                   │ backend agent    │
                   │ Luna / Terra /   │
                   │ Astra / external │
                   └────────┬─────────┘
                            │
                     tools / APIs / DB

That lets you use a cheap, fast backend for routine tasks and escalate to a stronger model only when a request needs deeper reasoning.

For example:

"Where is my order?"
        ↓
GPT-Live-1 keeps the conversation flowing
        ↓
backend queries the order API
        ↓
verified result
        ↓
GPT-Live-1 communicates it by voice

If the user adds “and change it to store pickup” while the backend is still working, the conversation does not have to freeze.

Two ways to delegate

The API exposes two main patterns.

1. Responses delegation

This is the more managed option. GPT-Live-1 prepares the backend request, supplies conversation context, and receives the result.

A simplified configuration can look like this:

const session = {
  model: "gpt-live-1",
  delegation: {
    type: "responses",
    responses: {
      model: "gpt-5.6-terra",
      instructions: "Solve the task and return only verified information."
    }
  }
};

The voice model and reasoning model are selected independently.

For high-volume workloads, Luna may make sense; for more difficult work, Terra or Astra can be a better fit depending on quality, cost, and latency requirements.

2. Client delegation

Here the application controls the backend completely.

const session = {
  model: "gpt-live-1",
  delegation: {
    type: "client"
  }
};

When GPT-Live-1 needs help, it emits session.delegation.created. The application keeps the delegation identifier, runs its agent, and sends a result back later.

Conceptually:

async function onDelegation(event) {
  const id = event.delegation.id;

  const context = buildContextFromTranscripts();
  const result = await runMyAgent(context);

  live.send({
    type: "session.commentary.append",
    delegation_id: id,
    content: result
  });
}

This is especially attractive if you already have your own agent, multi-model workflow, RAG system, internal microservices, or even a different model provider.

A critical detail: the delegation event does not contain the full task

With client delegation, do not assume session.delegation.created includes the exact text the user said.

Your application must maintain its own context from transcripts and internal state.

GPT-Live exposes events such as:

session.input_transcript.delta
session.output_transcript.delta
session.delegation.created

Transcript events contain fragments and timestamps. That makes it possible to reconstruct context such as:

user: "book it for Thursday"
user: "no, wait, make that Friday"

The backend needs the final intent rather than blindly executing the first fragment.

WebRTC for browser applications

For a web application, OpenAI recommends WebRTC.

The separation is useful:

browser
 ├─ media track → audio
 └─ data channel → JSON events

The browser captures the microphone and plays audio directly through the media path. Control events travel through a data channel.

The API secret should not live in the browser. The correct pattern is:

browser
   ↓ requests session
trusted backend
   ↓ uses OPENAI_API_KEY
OpenAI

Your server creates or negotiates the session and the client receives only what it needs to establish the connection.

That reduces the risk of exposing credentials in JavaScript shipped to end users.

WebSockets for server-side integrations

When audio arrives from another service—for example telephony, a gateway, a bot, or a backend application—WebSockets are often a more natural fit.

PBX / Twilio / backend
        ↓
     WebSocket
        ↓
    GPT-Live-1

In this mode the primary socket can carry both audio and control events.

OpenAI also supports sideband connections so a backend can observe and control an existing session without routing the audio through that same connection.

Telephony

GPT-Live-1 includes support for phone-oriented deployments, and OpenAI documents integration paths with providers and frameworks such as Twilio, Telnyx, LiveKit, and Daily/Pipecat.

That maps naturally to cases such as:

  • reservations;
  • customer support;
  • order tracking;
  • virtual reception;
  • help desks;
  • assistants that escalate to humans.

The advantage of full duplex is especially visible on the phone, where a rigid “speak → wait → listen” rhythm quickly feels artificial.

Tool calling: sensitive actions still belong in your backend

GPT-Live-1 can trigger delegated work, but sensitive operations still belong to the application.

For example, a backend might expose a function such as:

change_reservation(date, party_size)

When the model asks for that function, the application should:

1. validate arguments
2. check permissions
3. request confirmation when needed
4. execute the real API
5. return a structured result
6. continue the response

With Responses delegation, function results are sent back as function_call_output, after which the application explicitly continues delegated work with response.create.

That matters because “the model requested a tool” and “the action was authorized” are not the same thing.

Interrupting speech does not automatically cancel backend work

This is one of the easiest details to miss.

Imagine:

user: "cancel my flight"
        ↓
backend starts working
        ↓
user: "wait, don't!"

Interrupting the spoken output does not necessarily cancel the workflow already running in the backend.

OpenAI explicitly leaves durable task state and execution policy to the application.

A safer workflow therefore uses explicit states:

requested
  ↓
awaiting_confirmation
  ↓
approved
  ↓
executing
  ↓
completed

Actual backend cancellation should be a deliberate workflow transition rather than an accidental side effect of someone talking over the model.

thinking versus commentary

With client delegation, the backend has two useful ways to add information back to the live model.

session.thinking.append adds context for the model’s reasoning without implying that the content should be spoken immediately.

session.commentary.append adds a result intended to feed the spoken conversation.

That lets you separate:

internal state / auxiliary data
          ↓
       thinking

result that should be communicated
          ↓
      commentary

This is useful for preventing the agent from verbalizing logs, internal IDs, or operational details the user does not need to hear.

Short prompt in front, complex rules in the backend

OpenAI recommends keeping the live model’s prompt focused on interaction behavior:

- tone
- pace
- when to delegate
- when to ask for clarification
- how to handle silence

Long business rules belong in the backend:

- refund policies
- authorization
- spending limits
- database access
- tool workflows
- validation

That is a healthy architectural split because it avoids turning the conversational prompt into a giant operations manual.

Observability: keep transcripts, delegation, and outcomes

A serious implementation should correlate at least:

session_id
transcript timestamps
delegation_id
backend request
tools executed
result
final spoken response

The key point is that “backend completed” does not automatically mean “the user heard the result.” Delegated work and spoken output have separate lifecycles.

For auditing and debugging, observe both.

Cost

GPT-Live-1 costs $0.05 per minute for the front-end voice layer, billed per second.

That is approximately:

10 minutes   → $0.50
30 minutes   → $1.50
60 minutes   → $3.00
1,000 hours  → $3,000

Backend model and tool usage are billed separately.

That split also creates an optimization opportunity: not every turn needs Astra. A router can use Luna for simple operations and escalate only difficult work.

A reasonable production pattern

A complete architecture might end up looking like this:

                        ┌───────────────┐
                        │    browser    │
                        │ mic + speaker │
                        └───────┬───────┘
                                │ WebRTC
                                ▼
                        ┌───────────────┐
                        │  GPT-Live-1   │
                        │ full-duplex   │
                        └───────┬───────┘
                                │ delegation
               ┌────────────────┴────────────────┐
               ▼                                 ▼
         fast backend                     complex backend
        GPT-5.6 Luna                       Terra / Astra
               │                                 │
               └──────────────┬──────────────────┘
                              ▼
                       policy / auth layer
                              ▼
                     APIs · DB · MCP · tools
                              │
                              ▼
                      durable task state

The policy layer should remain external to the model. The model can propose an action; the application decides whether that action is permitted.

What disappears—and what does not

The claim that GPT-Live-1 “kills” the cascaded pipeline is useful but incomplete.

It does dramatically reduce the need to manually coordinate:

STT → LLM → TTS

But it does not eliminate:

  • authorization;
  • durable state;
  • error handling;
  • tool execution;
  • observability;
  • persistence;
  • security policy;
  • routing between models.

It simplifies the conversational layer and makes the rest of the agent architecture more explicit.

The most important idea

GPT-Live-1 turns voice into a continuous interface in front of an agentic system.

The pattern changes from:

voice → text → chatbot → text → voice

to something closer to:

human ⇄ intelligent voice interface ⇄ agent ⇄ tools

That change is deeper than a TTS improvement. It lets the model that talks be different from the model that reasons, lets work continue while the conversation keeps moving, and leaves real control over permissions and actions in the application.

For developers, that separation between real-time interaction and agentic work may be the most important part of GPT-Live-1.

Sources