GPT Live

OpenAI Live (gpt-live-1) is a full-duplex voice model. It listens while it speaks, so it handles interruptions, backchannels (“mm-hmm”) and overlapping speech on its own. It goes in the pipeline’s llm slot with no separate STT, TTS, VAD or turn detector. The voice model does not call tools itself. When a request needs reasoning or a tool, it delegates to a backend model, keeps talking while the backend works, and then says the result in its own words.

OpenAI Live is available in the Python SDK.

Setup

Set your OpenAI API key in the worker environment. Generate a key from the OpenAI dashboard:

export OPENAI_API_KEY=<key>

Usage

Pass OpenAILive to the pipeline’s llm slot and give the agent its tools as usual. The backend model calls them; the voice model speaks the result.

Python

from zeroruntime import Agent, Pipeline, function_tool
from zeroruntime.plugins import OpenAIBackendConfig, OpenAILive


class SupportAgent(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions=(
                "You are a friendly support agent. Keep replies short.\n"
                "Delegation policy:\n"
                "Backend tools:\n"
                "- Look up an order\n"
                "Delegate to the backend when:\n"
                "- The caller asks about an order.\n"
                "Do not delegate to the backend when:\n"
                "- The caller is just chatting or confirming what you said."
            ),
            agent_id="openai-live-support",
            pipeline=Pipeline(
                llm=OpenAILive(
                    model="gpt-live-1",
                    voice="marin",
                    config=OpenAIBackendConfig(
                        model="gpt-5.6-terra",
                        instructions="Look up the order before answering. Reply in one or two short sentences.",
                        tool_choice="auto",            # "auto" | "required" | "none"
                        parallel_tool_calls=False,    
                        web_search=False,         
                    ),
                ),
            ),
        )

    @function_tool
    async def get_order_status(self, order_id: str) -> dict:
        """Look up an order by its ID.

        Args:
            order_id: The order ID.
        """
        return {"order_id": order_id, "status": "out for delivery"}

Leave vad and turn_detector out of the pipeline. GPT-Live decides when to listen and speak; a local VAD would cut its audio whenever the caller makes a listening sound.

Delegation

The type of config picks who answers when the voice model delegates.

`config`	Who answers	Use it when
`OpenAIBackendConfig` (default)	An OpenAI Responses model, which calls your agent’s `@function_tool`	You want OpenAI end to end
`OpenAIDelegateLLMConfig`	A text LLM you choose, which runs your agent’s `@function_tool`	You want another provider answering, or you need vision

Omit config to use the backend with its defaults.

Answering with your own LLM

Python

from zeroruntime.inference import GoogleLLM
from zeroruntime.plugins import OpenAIDelegateLLMConfig, OpenAILive

model = OpenAILive(
    model="gpt-live-1",
    voice="marin",
    config=OpenAIDelegateLLMConfig(
        llm=GoogleLLM(model="gemini-3-flash-preview"),
        instructions="Use the tools to answer. Reply in one or two short sentences.",
    ),
)

The delegate LLM is configured like any other provider. Through zeroruntime.inference it runs on the gateway with no vendor key; from zeroruntime.plugins it needs its own key.

Writing the delegation policy

GPT-Live decides whether to delegate from its instructions; the backend then picks which tool to call from your tool schemas. Give the agent’s instructions a delegation policy in three labeled parts:

Delegation policy:
Backend tools:
- <what the backend can do, in plain words>
Delegate to the backend when:
- <requests that need a tool or reasoning>
Do not delegate to the backend when:
- <small talk, confirmations, questions the prompt already answers>

Keep tone and the delegation policy in the agent’s instructions, and business rules and tool guidance in the config’s instructions.

Steering a live call

Three session methods add context to the running model from anywhere in your agent: on_enter, a tool body, or a background task.

Method	The model	Use it for
`session.append_instructions(text)`	follows it from now on	A new rule, exact wording, “start wrapping up”
`session.append_thinking(text)`	knows it and uses it if it comes up	A caller’s profile, a booking just made
`session.append_commentary(text)`	says it aloud at the next natural pause, in its own words	News the caller should hear now

Python

@function_tool
async def book_table(self, name: str, time: str) -> dict:
    """Book a table.

    Args:
        name: The name for the booking.
        time: The booking time.
    """
    await self.session.append_thinking(f"Booking 101 is confirmed for {name} at {time}.")
    return {"booked": True, "booking_id": "101"}

Each append can contain up to 500 tokens.
Appends accumulate in the model’s context; they do not replace the agent’s instructions or previous appends.
Do not put secrets in thinking updates, since they may still be spoken aloud.

Only duplex models take appends. With any other model, the call logs a warning and sends nothing.

Vision

Neither the voice model nor the OpenAI backend takes images, so vision needs OpenAIDelegateLLMConfig with an LLM that reads images. With Room(vision=True), each time GPT-Live delegates, the caller’s latest camera or screen frame goes to the delegate LLM with the request. With OpenAIBackendConfig, frames are dropped with a warning.

Configuration

Configure GPT-Live’s voice model and its delegated backend using the constructor options below.

Constructor

Parameter	Type	Default	Description
`model`	`str`	`"gpt-live-1"`	GPT-Live model ID.
`voice`	`str`	`"marin"`	Output voice.
`api_key`	`str`	`None`	OpenAI API key. Falls back to the `OPENAI_API_KEY` environment variable when unset.
`config`	`OpenAIBackendConfig` or `OpenAIDelegateLLMConfig`	`None`	Picks the delegation mode. `None` uses `OpenAIBackendConfig` with its defaults.
`base_url`	`str`	`None`	OpenAI API base URL. Defaults to `https://api.openai.com/v1`.

`OpenAIBackendConfig`

Field	Type	Default	Description
`model`	`str`	`"gpt-5.6-terra"`	Responses model for delegated work. `gpt-5.6-luna` is cheaper.
`instructions`	`str`	`None`	Business rules and tool guidance, separate from the voice prompt.
`tool_choice`	`str`	`"auto"`	How the backend selects tools: `auto`, `required`, or `none`.
`parallel_tool_calls`	`bool`	`None`	Whether the backend may request several tool calls at once.
`reasoning_effort`	`str`	`None`	Backend reasoning effort, such as `low` or `medium`. Values depend on the model.
`service_tier`	`str`	`None`	`auto`, `default`, `flex`, or `priority`.
`max_output_tokens`	`int`	`None`	Cap on tokens per backend response, at least `16`.
`web_search`	`bool`	`False`	Give the backend OpenAI’s hosted web search tool.

`OpenAIDelegateLLMConfig`

Field	Type	Default	Description
`llm`	`LLM`	required	Any text LLM. It answers each delegation using your agent’s function tools.
`instructions`	`str`	`None`	Instructions that frame each of the LLM’s calls, separate from the voice prompt.

Import paths

SDK	Import	Constructor
Python	`from zeroruntime.plugins import OpenAILive, OpenAIBackendConfig, OpenAIDelegateLLMConfig`	`OpenAILive(model=..., voice=..., config=...)`

Examples

Runnable agents are in the duplex examples:

Example	What it shows
`openai_live_delegate_llm.py`	Answering delegations with an LLM you choose
`openai_live_restaurant_booking.py`	Booking a table, with a slot that frees up mid-call

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Duplex models

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Delegation

Answering with your own LLM

Writing the delegation policy

Steering a live call

Vision

Configuration

Constructor

`OpenAIBackendConfig`

`OpenAIDelegateLLMConfig`

Import paths

Examples

​Setup

​Usage

​Delegation

​Answering with your own LLM

​Writing the delegation policy

​Steering a live call

​Vision

​Configuration

​Constructor

​OpenAIBackendConfig

​OpenAIDelegateLLMConfig

​Import paths

​Examples

Setup

Usage

Delegation

Answering with your own LLM

Writing the delegation policy

Steering a live call

Vision

Configuration

Constructor

`OpenAIBackendConfig`

`OpenAIDelegateLLMConfig`

Import paths

Examples