OpenAI Realtime

OpenAI Realtime is a speech-to-speech model. It handles transcription, reasoning, and voice synthesis end-to-end in a single model, so it goes in the pipeline’s llm slot with no separate STT or TTS. The Pipeline auto-detects Realtime mode when you pass it.

Setup

Set your OpenAI API key in the worker environment. Generate a key from the OpenAI dashboard:

export OPENAI_API_KEY=<key>

Usage

Pass the realtime model to the pipeline’s llm slot, no stt or tts needed.

from zeroruntime import Pipeline
from zeroruntime.plugins import OpenAIRealtime

model = OpenAIRealtime(
    model="gpt-4o-realtime-preview",
    config={
        "voice": "alloy",
        "modalities": ["text", "audio"],
    },
)

pipeline = Pipeline(llm=model)

import { Pipeline } from '@zeroruntime/js-sdk';
import { OpenAIRealtime } from '@zeroruntime/js-sdk/plugins';

const model = OpenAIRealtime({
  model: 'gpt-4o-realtime-preview',
  config: {
    voice: 'alloy',
    modalities: ['text', 'audio'],
  },
});

const pipeline = Pipeline({ llm: model });

Configuration

OpenAIRealtime constructor and config keys in both SDKs. The Python and Node JS SDKs share these field names.

Constructor

Parameter	Type	Default	Description
`model`	`str`	`"gpt-4o-realtime-preview"`	Realtime model ID.
`api_key`	`str`	`None`	OpenAI API key. Falls back to the `OPENAI_API_KEY` environment variable when unset.
`config`	`dict`	`None`	Model behavior configuration (see below).

`config` keys

Field	Type	Default	Description
`voice`	`str`	`"alloy"`	Output voice (e.g. `alloy`, `ash`, `marin`, `cedar`, `verse`).
`temperature`	`float`	`0.8`	Sampling randomness.
`turn_detection`	`TurnDetectionConfig`	server VAD	Turn detection (`server_vad`, threshold `0.5`, `prefix_padding_ms` `300`, `silence_duration_ms` `200`). Pass `None` to disable.
`input_audio_transcription`	`InputAudioTranscriptionConfig`	`gpt-4o-mini-transcribe`	Transcription model for the user’s audio.
`tool_choice`	`str`	`"auto"`	How the model decides when to call tools.
`modalities`	`list[str]`	`["text", "audio"]`	Enabled response modalities. Drop `"audio"` for text-only.

Import paths

SDK	Import	Constructor
Python	`from zeroruntime.plugins import OpenAIRealtime`	`OpenAIRealtime(model=..., config={...})`
Node JS	`import { OpenAIRealtime } from '@zeroruntime/js-sdk/plugins'`	`OpenAIRealtime({ ... })`

To pair the realtime model with an external STT or TTS, see Hybrid mode.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Configuration

Constructor

`config` keys

Import paths

​Setup

​Usage

​Configuration

​Constructor

​config keys

​Import paths

Setup

Usage

Configuration

Constructor

`config` keys

Import paths