Gemini Live

Gemini Live is a speech-to-speech model. It handles transcription, reasoning, and voice synthesis end-to-end, so it goes in the pipeline’s llm slot with no separate STT or TTS. The Pipeline auto-detects Realtime mode when you pass it.

Setup

Set your Google API key in the worker environment. Generate a key from the Google AI Studio:

export GOOGLE_API_KEY=<key>

To run Gemini Live through Vertex AI, set vertexai=True and authenticate with a service account (GOOGLE_APPLICATION_CREDENTIALS), as with the Gemini LLM plugin.

Usage

Pass the realtime model to the pipeline’s llm slot, no stt or tts needed.

from zrt import Pipeline
from zrt.plugins import GeminiRealtime, GeminiLiveConfig

model = GeminiRealtime(
    config=GeminiLiveConfig(
        model="gemini-3.1-flash-live-preview",
        voice="Puck",
        response_modalities=["AUDIO"],
    ),
)

pipeline = Pipeline(llm=model)

The default model is gemini-3.1-flash-live-preview in every SDK. Pass model explicitly for consistency.

Configuration

GeminiRealtime constructor and GeminiLiveConfig fields for the Python SDK. The JavaScript SDK passes the same settings via the flat config object; the Go SDK via RealtimeOptions.

Constructor

Parameter	Type	Default	Description
`api_key`	`str`	`None`	Gemini API key. Falls back to the `GOOGLE_API_KEY` environment variable when unset.
`config`	`GeminiLiveConfig`	`None`	Model behavior configuration (see below).

`GeminiLiveConfig`

Field	Type	Default	Description
`model`	`str`	`"gemini-3.1-flash-live-preview"`	Gemini Live model ID.
`voice`	`str`	`"Puck"`	Output voice: `Puck`, `Charon`, `Kore`, `Fenrir`, or `Aoede`.
`language_code`	`str`	`None`	Language for speech synthesis.
`response_modalities`	`list`	`["AUDIO"]`	Enabled response types (`"TEXT"`, `"AUDIO"`).
`top_p`	`float`	`None`	Nucleus-sampling cutoff.
`top_k`	`int`	`None`	Top-k sampling cutoff.
`max_output_tokens`	`int`	`None`	Caps the length of each response.
`thinking_budget`	`int`	`None`	Thinking budget; `0` disables it for low-latency voice (native-audio models only).
`include_thoughts`	`bool`	`False`	Include the model’s thought summaries in the response.
`vad_start_sensitivity`	`str`	`None`	Start-of-speech sensitivity for automatic activity detection.
`vad_end_sensitivity`	`str`	`None`	End-of-speech sensitivity for automatic activity detection.
`vad_prefix_padding_ms`	`int`	`None`	Audio padding retained before detected speech, in milliseconds.
`vad_silence_duration_ms`	`int`	`None`	Silence duration that ends a turn, in milliseconds.
`context_compression_trigger_tokens`	`int`	`None`	Token count that triggers context-window compression to extend sessions past the connection cap.
`session_resumption_handle`	`str`	`None`	Handle used to resume a previous session on reconnect.
`vertexai`	`bool`	`False`	Route through Vertex AI instead of the Gemini API.
`vertex_project_id`	`str`	`None`	Google Cloud project ID for Vertex AI (required when `vertexai=True`).
`vertex_location`	`str`	`"us-central1"`	Vertex AI region.
`vertex_service_account_json`	`str` or `dict`	`None`	Inline service-account credentials for Vertex AI.
`vertex_service_account_path`	`str`	`None`	Path to a service-account JSON for Vertex AI. Falls back to `GOOGLE_APPLICATION_CREDENTIALS` when unset.

Import paths

SDK	Import	Constructor
Python	`from zrt.plugins import GeminiRealtime, GeminiLiveConfig`	`GeminiRealtime(config=GeminiLiveConfig(...))`
JavaScript	`from '@zrt/js-sdk/plugins/gemini_realtime'`	`GeminiRealtime({ config: { ... } })`
Go	`.../zrt-golang-sdk/plugins/gemini_realtime`	`gemini_realtime.NewRealtime(gemini_realtime.RealtimeOptions{...})`

To pair the realtime model with an external STT or TTS, see Hybrid mode.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Configuration

Constructor

`GeminiLiveConfig`

Import paths

​Setup

​Usage

​Configuration

​Constructor

​GeminiLiveConfig

​Import paths

Setup

Usage

Configuration

Constructor

`GeminiLiveConfig`

Import paths