Azure Voice Live

Azure Voice Live is a speech-to-speech model. It unifies speech recognition, generative AI, and text-to-speech into a single Microsoft Azure endpoint, so it goes in the pipeline’s llm slot with no separate STT or TTS. The Pipeline auto-detects Realtime mode when you pass it.

Setup

Set your Azure Voice Live API key in the worker environment. Generate a key from the Azure AI Foundry portal:

export AZURE_VOICE_LIVE_API_KEY=<key>

Azure Voice Live also needs a service endpoint. Pass it explicitly as endpoint (or set an endpoint key in config), or export it as AZURE_VOICE_LIVE_ENDPOINT and it will be picked up automatically.

export AZURE_VOICE_LIVE_ENDPOINT=<your-azure-ai-service-endpoint>

Usage

Pass the realtime model to the pipeline’s llm slot, no stt or tts needed.

from zeroruntime import Pipeline
from zeroruntime.plugins import AzureVoiceLive

llm = AzureVoiceLive(
    model="gpt-4o-realtime-preview",
    config={
        "voice": "en-US-AvaNeural",
        "modalities": ["text", "audio"],
    },
)

pipeline = Pipeline(llm=llm)

import { Pipeline } from '@zeroruntime/js-sdk';
import { AzureVoiceLive } from '@zeroruntime/js-sdk/plugins';

const llm = AzureVoiceLive({
  model: 'gpt-4o-realtime-preview',
  config: {
    voice: 'en-US-AvaNeural',
    modalities: ['text', 'audio'],
  },
});

const pipeline = Pipeline({ llm });

Parameters

Constructor parameters for AzureVoiceLive. The Python and Node JS SDKs share these field names. Model behavior is configured through the config mapping.

Parameter	Type	Default	Description
`model`	`str`	`"gpt-4o-realtime-preview"`	Voice Live model ID.
`api_key`	`str`	`None`	Azure Voice Live API key. Falls back to the `AZURE_VOICE_LIVE_API_KEY` environment variable when unset.
`config`	`dict`	`None`	Model behavior configuration (see below). When omitted, provider defaults are used.
`endpoint`	`str`	`None`	Azure Voice Live service endpoint. Falls back to `config.endpoint`, then the `AZURE_VOICE_LIVE_ENDPOINT` environment variable, then an empty string.

`config` keys

Field	Type	Default	Description
`voice`	`str`	`"en-US-AvaNeural"`	Output voice: an Azure neural voice (e.g. `"en-US-AvaNeural"`) or an OpenAI voice (e.g. `"alloy"`).
`endpoint`	`str`	`None`	Azure Voice Live service endpoint.
`modalities`	`list[str]`	`["text", "audio"]`	Enabled response modalities. Drop `"audio"` for text-only.
`temperature`	`float`	`None`	Sampling randomness.
`max_response_output_tokens`	`int \| str`	`None`	Caps the length of each response.
`turn_detection`	`TurnDetectionConfig`	server VAD	Turn detection (`server_vad`, threshold `0.5`, `prefix_padding_ms` `300`, `silence_duration_ms` `500`, `create_response` `True`, `interrupt_response` `True`).
`input_audio_transcription`	`InputAudioTranscriptionConfig`	`gpt-4o-mini-transcribe`	Transcription model for the caller’s audio.
`tool_choice`	`str`	`"auto"`	How the model decides when to call tools.

To pair the realtime model with an external STT or TTS instead, see the plugins overview.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Parameters

`config` keys

​Setup

​Usage

​Parameters

​config keys

Setup

Usage

Parameters

`config` keys