OpenAI

OpenAI is an LLM plugin. It takes the transcribed conversation and generates the reply using OpenAI’s chat models, with tool calling and an optional low-latency WebSocket streaming mode.

Setup

Set your OpenAI API key in the worker environment. Generate a key from the OpenAI dashboard:

export OPENAI_API_KEY=<key>

Usage

Import the plugin and pass it to the pipeline’s llm slot.

from zrt.plugins import OpenAILLM

llm = OpenAILLM(
    model="gpt-5.4-nano",
    temperature=0.7,
)

# Pipeline(llm=llm, ...)

The default model is gpt-5.4-nano in the Python SDK and gpt-4o in the JavaScript and Go SDKs. Pass model explicitly to avoid surprises across SDKs.

Low-latency streaming

Set streaming=True (Python) to use OpenAI’s WebSocket Responses API instead of HTTP chat completions. The connection is reused across turns and continues with previous_response_id for lower per-turn latency.

Python

from zrt.plugins import OpenAILLM

llm = OpenAILLM(model="gpt-5.4-nano", streaming=True)

Azure OpenAI

To run OpenAI chat models through Azure OpenAI, use the separate AzureOpenAILLM plugin. It reads its configuration from Azure environment variables when arguments are omitted:

export AZURE_OPENAI_API_KEY=<key>
export AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com

Python

from zrt.plugins import AzureOpenAILLM

llm = AzureOpenAILLM(
    azure_endpoint="https://<resource>.openai.azure.com",
    deployment="<deployment-name>",
    api_version="2024-10-21",
)

# Pipeline(llm=llm, ...)

deployment is the custom name you chose when deploying a model in Azure (not the underlying model name). Authenticate with api_key, or fall back to the AZURE_OPENAI_API_KEY environment variable.

Configuration Options

Constructor parameters for the Python SDK (OpenAILLM). The JavaScript and Go SDKs expose model, temperature, and max_output_tokens/maxOutputTokens plus common sampling options in their idiomatic form (camelCase / LLMOptions struct).

Core

Parameter	Type	Default	Description
`api_key`	`str`	`None`	OpenAI API key. Falls back to the `OPENAI_API_KEY` environment variable when unset.
`model`	`str`	`"gpt-5.4-nano"`	Chat model used to generate replies.
`temperature`	`float`	`0.7`	Sampling randomness; lower is more deterministic.
`tool_choice`	`str`	`None`	How the model decides when to call tools (`"none"`, `"auto"`, `"required"`, or a specific tool spec).
`max_output_tokens`	`int`	`1024`	Caps the length of each reply. The legacy `max_completion_tokens` kwarg is accepted as an alias.
`response_format`	`dict`	`None`	Output format spec, e.g. `{"type": "json_object"}` or a `json_schema` for Structured Outputs.
`stop`	`str`	`None`	Up to 4 stop sequences at which generation halts.
`user`	`str`	`None`	Stable end-user identifier for abuse monitoring.

Sampling

Parameter	Type	Default	Description
`top_p`	`float`	`None`	Nucleus-sampling probability cutoff.
`frequency_penalty`	`float`	`None`	Penalizes frequent tokens, reducing repetition.
`presence_penalty`	`float`	`None`	Penalizes tokens already present, encouraging new topics.
`seed`	`int`	`None`	Seed for deterministic sampling.
`parallel_tool_calls`	`bool`	`None`	Allow the model to call multiple tools in one turn.

Reasoning models

Parameter	Type	Default	Description
`reasoning_effort`	`str`	`None`	Reasoning depth (`"minimal"`, `"low"`, `"medium"`, `"high"`) for reasoning / GPT-5 models. `"none"` requires `gpt-5.1` or later.
`verbosity`	`str`	`None`	Output verbosity (`"low"`, `"medium"`, `"high"`) for reasoning / GPT-5 models.

Streaming & client

Parameter	Type	Default	Description
`streaming`	`bool`	`False`	Use the WebSocket Responses API for lower per-turn latency.
`store`	`bool`	`True`	Persist responses server-side for later retrieval.
`wss_url`	`str`	`None`	Override the WebSocket Responses URL.

Import paths

SDK	Import	Constructor
Python	`from zrt.plugins import OpenAILLM`	`OpenAILLM(...)`
JavaScript	`from '@zrt/js-sdk/plugins/openai'`	`OpenAILLM({ ... })`
Go	`.../zrt-golang-sdk/plugins/openai`	`openai.NewLLM(openai.LLMOptions{...})`

Azure OpenAI is provided by the separate AzureOpenAILLM plugin (from zrt.plugins import AzureOpenAILLM), currently available in the Python SDK only.

The reply is streamed to the text-to-speech plugin, which synthesizes the agent’s voice.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Low-latency streaming

Azure OpenAI

Configuration Options

Core

Sampling

Reasoning models

Streaming & client

Import paths

​Setup

​Usage

​Low-latency streaming

​Azure OpenAI

​Configuration Options

​Core

​Sampling

​Reasoning models

​Streaming & client

​Import paths

Setup

Usage

Low-latency streaming

Azure OpenAI

Configuration Options

Core

Sampling

Reasoning models

Streaming & client

Import paths