Azure OpenAI

Azure OpenAI is an LLM plugin. It generates the agent’s text responses from the caller’s transcribed speech, with support for tool calling and streaming.

Setup

Set your Azure OpenAI API key in the worker environment. Generate a key from the Azure AI Foundry portal:

export AZURE_OPENAI_API_KEY=<key>

Usage

Import the plugin and pass it to the pipeline’s llm slot.

from zrt.plugins import AzureOpenAILLM

llm = AzureOpenAILLM(
    azure_endpoint="https://<resource>.openai.azure.com/",
    deployment="gpt-4o",
)

# Pipeline(llm=llm, ...)

Parameters

Constructor parameters for the Python SDK (AzureOpenAILLM).

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Azure OpenAI API key. Falls back to the `AZURE_OPENAI_API_KEY` environment variable when unset.
`azure_endpoint`	`str \| None`	`None`	Full Azure OpenAI resource endpoint URL, e.g. `"https://<resource>.openai.azure.com/"`. Falls back to the `AZURE_OPENAI_ENDPOINT` environment variable when unset.
`deployment`	`AzureOpenAILLMModel \| str \| None`	`None`	Name of the Azure OpenAI deployment (not the underlying model name) - the custom name chosen when deploying a model in the Azure portal or AI Foundry, e.g. `"gpt-4o-deployment"`. Must be provided.
`api_version`	`str`	`"2024-10-21"`	Azure OpenAI REST API version date string.
`temperature`	`float`	`0.7`	Sampling temperature in `[0.0, 2.0]`. Lower values produce more deterministic output.
`max_output_tokens`	`int`	`1024`	Maximum tokens the model may generate per response.
`top_p`	`float \| None`	`None`	Nucleus sampling probability mass in `(0.0, 1.0]`. Mutually exclusive with `temperature`.
`frequency_penalty`	`float \| None`	`None`	Float in `[-2.0, 2.0]`. Penalizes token repetition based on cumulative frequency so far.
`presence_penalty`	`float \| None`	`None`	Float in `[-2.0, 2.0]`. Penalizes tokens that have already appeared, encouraging topic diversity.
`seed`	`int \| None`	`None`	Integer seed for deterministic sampling.
`stop`	`str \| None`	`None`	String (or up to four strings) at which the model stops generating further tokens.
`user`	`str \| None`	`None`	Opaque end-user identifier forwarded to Azure for abuse monitoring.
`tool_choice`	`str \| None`	`None`	Controls how the model selects tools: `"none"`, `"auto"`, `"required"`, or a named tool dict. Defaults to `"auto"` when tools are present.
`parallel_tool_calls`	`bool \| None`	`None`	When `True`, the model may emit multiple tool calls in a single response turn.
`response_format`	`dict \| None`	`None`	Enforce a structured output schema - `{"type": "json_object"}` for JSON mode, or a JSON Schema dict for strict structured outputs.
`reasoning_effort`	`str \| None`	`None`	Controls reasoning token budget for o-series (reasoning) models: `"low"`, `"medium"`, or `"high"`. Ignored for non-reasoning models.

The LLM’s response is synthesized to speech by the TTS plugin configured in the pipeline.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Parameters

​Setup

​Usage

​Parameters

Setup

Usage

Parameters