Azure Voice Live

Azure Voice Live is a speech-to-speech model. It unifies speech recognition, generative AI, and text-to-speech into a single Microsoft Azure endpoint, so it goes in the pipeline’s llm slot with no separate STT or TTS. The Pipeline auto-detects Realtime mode when you pass it.

Setup

Set your Azure Voice Live API key in the worker environment. Generate a key from the Azure AI Foundry portal:

export AZURE_VOICE_LIVE_API_KEY=<key>

Azure Voice Live also needs a service endpoint. Pass it explicitly as endpoint (or set it on AzureVoiceLiveConfig), or export it as AZURE_VOICE_LIVE_ENDPOINT and it will be picked up automatically.

export AZURE_VOICE_LIVE_ENDPOINT=<your-azure-ai-service-endpoint>

Usage

Pass the realtime model to the pipeline’s llm slot, no stt or tts needed.

from zrt import Pipeline
from zrt.plugins import AzureVoiceLive, AzureVoiceLiveConfig

llm = AzureVoiceLive(
    config=AzureVoiceLiveConfig(
        model="gpt-4o-realtime-preview",
        voice="en-US-AvaNeural",
        modalities=["text", "audio"],
    ),
)

pipeline = Pipeline(llm=llm)

Parameters

Constructor parameters for the Python SDK (AzureVoiceLive). Model behavior is configured through the nested AzureVoiceLiveConfig object.

Parameter	Type	Default	Description
`api_key`	`str`	`None`	Azure Voice Live API key. Falls back to the `AZURE_VOICE_LIVE_API_KEY` environment variable when unset.
`config`	`AzureVoiceLiveConfig`	`None`	Model behavior configuration (see below). When omitted, a default `AzureVoiceLiveConfig` is used.
`endpoint`	`str`	`None`	Azure Voice Live service endpoint. Falls back to `config.endpoint`, then the `AZURE_VOICE_LIVE_ENDPOINT` environment variable, then an empty string.

`AzureVoiceLiveConfig`

Field	Type	Default	Description
`model`	`str`	`"gpt-4o-realtime-preview"`	Voice Live model ID.
`voice`	`str`	`"en-US-AvaNeural"`	Output voice: an Azure neural voice (e.g. `"en-US-AvaNeural"`) or an OpenAI voice (e.g. `"alloy"`).
`endpoint`	`str`	`None`	Azure Voice Live service endpoint.
`modalities`	`list[str]`	`["text", "audio"]`	Enabled response modalities. Drop `"audio"` for text-only.
`temperature`	`float`	`None`	Sampling randomness.
`max_response_output_tokens`	`int \| str`	`None`	Caps the length of each response.
`turn_detection`	`TurnDetectionConfig`	server VAD	Turn detection (`server_vad`, threshold `0.5`, `prefix_padding_ms` `300`, `silence_duration_ms` `500`, `create_response` `True`, `interrupt_response` `True`).
`input_audio_transcription`	`InputAudioTranscriptionConfig`	`gpt-4o-mini-transcribe`	Transcription model for the caller’s audio.
`tool_choice`	`str`	`"auto"`	How the model decides when to call tools.

To pair the realtime model with an external STT or TTS instead, see the plugins overview.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Parameters

`AzureVoiceLiveConfig`

​Setup

​Usage

​Parameters

​AzureVoiceLiveConfig

Setup

Usage

Parameters

`AzureVoiceLiveConfig`