> ## Documentation Index > Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt > Use this file to discover all available pages before exploring further. # GPT Live > Use OpenAI GPT-Live, a full-duplex voice model, in a Zero Runtime pipeline. Setup and usage in Python and JavaScript. OpenAI Live (`gpt-live-1`) is a **full-duplex** voice model. It listens while it speaks, so it handles interruptions, backchannels ("mm-hmm") and overlapping speech on its own. It goes in the pipeline's `llm` slot with no separate STT, TTS, VAD or turn detector. The voice model does not call tools itself. When a request needs reasoning or a tool, it **delegates** to a backend model, keeps talking while the backend works, and then says the result in its own words. ## Setup Set your OpenAI API key in the worker environment. Generate a key from the [OpenAI dashboard](https://platform.openai.com/api-keys): ```bash theme={null} export OPENAI_API_KEY= ``` ## Usage Pass `OpenAILive` to the pipeline's `llm` slot and give the agent its tools as usual. The backend model calls them; the voice model speaks the result. ```python Python theme={null} from zeroruntime import Agent, Pipeline, function_tool from zeroruntime.plugins import OpenAIBackendConfig, OpenAILive class SupportAgent(Agent): def __init__(self) -> None: super().__init__( instructions=( "You are a friendly support agent. Keep replies short.\n" "Delegation policy:\n" "Backend tools:\n" "- Look up an order\n" "Delegate to the backend when:\n" "- The caller asks about an order.\n" "Do not delegate to the backend when:\n" "- The caller is just chatting or confirming what you said." ), pipeline=Pipeline( llm=OpenAILive( model="gpt-live-1", voice="marin", config=OpenAIBackendConfig( model="gpt-5.6-terra", instructions="Look up the order before answering. Reply in one or two short sentences.", reasoning_effort="medium", tool_choice="auto", # "auto" | "required" | "none" parallel_tool_calls=False, web_search=False, ), ), ), ) @function_tool async def get_order_status(self, order_id: str) -> dict: """Look up an order by its ID. Args: order_id: The order ID. """ return {"order_id": order_id, "status": "out for delivery"} ``` ```typescript Node JS theme={null} import { Agent, Pipeline, function_tool } from '@zeroruntime/js-sdk'; import { OpenAIBackendConfig, OpenAILive } from '@zeroruntime/js-sdk/plugins'; class SupportAgent extends Agent { constructor() { super({ instructions: 'You are a friendly support agent. Keep replies short.\n' + 'Delegation policy:\n' + 'Backend tools:\n' + '- Look up an order\n' + 'Delegate to the backend when:\n' + '- The caller asks about an order.\n' + 'Do not delegate to the backend when:\n' + '- The caller is just chatting or confirming what you said.', pipeline: Pipeline({ llm: OpenAILive({ model: 'gpt-live-1', voice: 'marin', config: OpenAIBackendConfig({ model: 'gpt-5.6-terra', instructions: 'Look up the order before answering. Reply in one or two short sentences.', reasoning_effort: 'medium', tool_choice: 'auto', // 'auto' | 'required' | 'none' parallel_tool_calls: false, web_search: false, }), }), }), }); } get_order_status = function_tool({ name: 'get_order_status', description: 'Look up an order by its ID.', parameters: { order_id: { type: 'string', description: 'The order ID.' }, }, execute: async ({ order_id }) => ({ order_id, status: 'out for delivery' }), }); } ``` Leave `vad` and `turn_detector` out of the pipeline. GPT-Live decides when to listen and speak; a local VAD would cut its audio whenever the caller makes a listening sound. ## Delegation The type of `config` picks who answers when the voice model delegates. | `config` | Who answers | Use it when | | - | - | - | | `OpenAIBackendConfig` (default) | An OpenAI Responses model, which calls your agent's function tools | You want OpenAI end to end | | `OpenAIDelegateLLMConfig` | A text LLM you choose, which runs your agent's function tools | You want another provider answering, or you need vision | Omit `config` to use the backend with its defaults. ### Answering with your own LLM ```python Python theme={null} from zeroruntime.inference import GoogleLLM from zeroruntime.plugins import OpenAIDelegateLLMConfig, OpenAILive model = OpenAILive( model="gpt-live-1", voice="marin", config=OpenAIDelegateLLMConfig( llm=GoogleLLM(model="gemini-3-flash-preview"), instructions="Use the tools to answer. Reply in one or two short sentences.", ), ) ``` ```typescript Node JS theme={null} import { GoogleLLM } from '@zeroruntime/js-sdk/inference'; import { OpenAIDelegateLLMConfig, OpenAILive } from '@zeroruntime/js-sdk/plugins'; const model = OpenAILive({ model: 'gpt-live-1', voice: 'marin', config: OpenAIDelegateLLMConfig({ llm: GoogleLLM({ model: 'gemini-3-flash-preview' }), instructions: 'Use the tools to answer. Reply in one or two short sentences.', }), }); ``` The delegate LLM is configured like any other provider. Through `zeroruntime.inference` (`@zeroruntime/js-sdk/inference`) it runs on the gateway with no vendor key; from `zeroruntime.plugins` (`@zeroruntime/js-sdk/plugins`) it needs its own key. ### Writing the delegation policy GPT-Live decides **whether** to delegate from its instructions; the backend then picks **which** tool to call from your tool schemas. Give the agent's instructions a delegation policy in three labeled parts: ```text theme={null} Delegation policy: Backend tools: - Delegate to the backend when: - Do not delegate to the backend when: - ``` Keep tone and the delegation policy in the agent's `instructions`, and business rules and tool guidance in the config's `instructions`. ## Steering a live call Three session methods add context to the running model from anywhere in your agent: `on_enter`, a tool body, or a background task. | Method | The model | Use it for | | - | - | - | | `session.append_instructions(text)` | follows it from now on | A new rule, exact wording, "start wrapping up" | | `session.append_thinking(text)` | knows it and uses it if it comes up | A caller's profile, a booking just made | | `session.append_commentary(text)` | says it aloud at the next natural pause, in its own words | News the caller should hear now | ```python Python theme={null} @function_tool async def book_table(self, name: str, time: str) -> dict: """Book a table. Args: name: The name for the booking. time: The booking time. """ await self.session.append_thinking(f"Booking 101 is confirmed for {name} at {time}.") return {"booked": True, "booking_id": "101"} ``` ```typescript Node JS theme={null} book_table = function_tool({ name: 'book_table', description: 'Book a table.', parameters: { name: { type: 'string', description: 'The name for the booking.' }, time: { type: 'string', description: 'The booking time.' }, }, execute: async function (this: Agent, { name, time }) { await this.session!.append_thinking(`Booking 101 is confirmed for ${name} at ${time}.`); return { booked: true, booking_id: '101' }; }, }); ``` * Each append can contain up to 500 tokens. * Appends accumulate in the model's context; they do not replace the agent's instructions or previous appends. * Do not put secrets in thinking updates, since they may still be spoken aloud. Only duplex models take appends. With any other model, the call logs a warning and sends nothing. ## Vision Neither the voice model nor the OpenAI backend takes images, so vision needs `OpenAIDelegateLLMConfig` with an LLM that reads images. With `Room(vision=True)` (`Room({ vision: true })` in Node JS), each time GPT-Live delegates, the caller's latest camera or screen frame goes to the delegate LLM with the request. With `OpenAIBackendConfig`, frames are dropped with a warning. ## Configuration Configure GPT-Live's voice model and its delegated backend using the constructor options below. *The Python and Node JS SDKs share these field names.* ### Constructor | Parameter | Type | Default | Description | | - | - | - | - | | `model` | `str` | `"gpt-live-1"` | GPT-Live model ID. | | `voice` | `str` | `"marin"` | Output voice. | | `api_key` | `str` | `None` | OpenAI API key. Falls back to the `OPENAI_API_KEY` environment variable when unset. | | `config` | `OpenAIBackendConfig` or `OpenAIDelegateLLMConfig` | `None` | Picks the delegation mode. `None` uses `OpenAIBackendConfig` with its defaults. | | `base_url` | `str` | `None` | OpenAI API base URL. Defaults to `https://api.openai.com/v1`. | ### `OpenAIBackendConfig` | Field | Type | Default | Description | | - | - | - | - | | `model` | `str` | `"gpt-5.6-terra"` | Responses model for delegated work. `gpt-5.6-luna` is cheaper. | | `instructions` | `str` | `None` | Business rules and tool guidance, separate from the voice prompt. | | `tool_choice` | `str` | `"auto"` | How the backend selects tools: `auto`, `required`, or `none`. | | `parallel_tool_calls` | `bool` | `None` | Whether the backend may request several tool calls at once. | | `reasoning_effort` | `str` | `None` | Backend reasoning effort, such as `low` or `medium`. Values depend on the model. | | `service_tier` | `str` | `None` | `auto`, `default`, `flex`, or `priority`. | | `max_output_tokens` | `int` | `None` | Cap on tokens per backend response, at least `16`. | | `web_search` | `bool` | `False` | Give the backend OpenAI's hosted web search tool. | ### `OpenAIDelegateLLMConfig` | Field | Type | Default | Description | | - | - | - | - | | `llm` | `LLM` | required | Any text LLM. It answers each delegation using your agent's function tools. | | `instructions` | `str` | `None` | Instructions that frame each of the LLM's calls, separate from the voice prompt. | ## Import paths | SDK | Import | Constructor | | - | - | - | | Python | `from zeroruntime.plugins import OpenAILive, OpenAIBackendConfig, OpenAIDelegateLLMConfig` | `OpenAILive(model=..., voice=..., config=...)` | | Node JS | `import { OpenAILive, OpenAIBackendConfig, OpenAIDelegateLLMConfig } from '@zeroruntime/js-sdk/plugins'` | `OpenAILive({ model, voice, config })` | ## Examples Runnable agents are in the [Python duplex examples](https://github.com/ZeroRuntimeAI/zeroruntime-python-examples/tree/main/duplex): | Example | What it shows | | - | - | | `openai_live_delegate_llm.py` | Answering delegations with an LLM you choose | | `openai_live_restaurant_booking.py` | Booking a table, with a slot that frees up mid-call | Run one with `uv run duplex/openai_live_delegate_llm.py`, or `python duplex/openai_live_delegate_llm.py` with the virtualenv active. Runnable agents are in the [Node JS duplex examples](https://github.com/ZeroRuntimeAI/zeroruntime-js-examples/tree/main/duplex): | Example | What it shows | | - | - | | `openai_live_delegate_llm.ts` | Answering delegations with an LLM you choose | | `openai_live_restaurant_booking.ts` | Booking a table, with a slot that frees up mid-call | Run one with `npx tsx duplex/openai_live_delegate_llm.ts`. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.