> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipeline

> Python API reference for Pipeline.

The voice stack an agent runs on: speech-to-text, LLM, and text-to-speech, plus
voice-activity and turn detection. `Pipeline` wires these components together and
exposes hooks for observing and shaping each turn.

## Pipeline

The voice stack an agent runs on: how it hears, thinks, and speaks.

Compose a cascade by passing components (`stt`, `llm`, `tts` plus `vad` and
`turn_detector`), or a speech-to-speech setup via `realtime_config`. Also carries
behavior configuration like interruption, end-of-utterance, the context window, DTMF,
and wake-up. Register pipeline-event hooks with the `@pipeline.on(...)` decorator.

### Constructor

```python theme={null}
Pipeline(stt: 'Any' = None, llm: 'Any' = None, tts: 'Any' = None, vad: 'Any' = None, turn_detector: 'Any' = None, avatar: 'Any' = None, denoise: 'Any' = None, eou_config: 'EOUConfig | None' = None, interrupt_config: 'InterruptConfig | None' = None, conversational_graph: 'Any' = None, context_window: 'ContextWindow | Any' = None, voice_mail_detector: 'Any' = None, dtmf_handler: 'Any' = None, wake_up: 'int | None' = None, wake_up_max_attempts: 'int | None' = None, realtime_config: 'RealtimeConfig | None' = None, llm_stream_hook_enabled: 'bool' = False, llm_stream_hook_timeout_ms: 'int' = 100, playback_grace_ms: 'int' = 0, stt_filter_patterns: 'list[str] | None' = None, stt_word_substitutions: 'dict[str, str] | None' = None, inactivity_timeout_seconds: 'int | None' = None) -> 'None'
```

<ParamField path="stt" type="Any">
  Speech-to-text component (cascade mode).
</ParamField>

<ParamField path="llm" type="Any">
  Language model. A realtime/speech-to-speech model enables realtime mode.
</ParamField>

<ParamField path="tts" type="Any">
  Text-to-speech component (cascade mode).
</ParamField>

<ParamField path="vad" type="Any">
  Voice-activity detector (e.g. Silero).
</ParamField>

<ParamField path="turn_detector" type="Any">
  End-of-turn detector deciding when the caller has finished.
</ParamField>

<ParamField path="avatar" type="Any">
  Optional visual avatar component.
</ParamField>

<ParamField path="denoise" type="Any">
  Optional input noise-cancellation component.
</ParamField>

<ParamField path="eou_config" type="EOUConfig | None">
  End-of-utterance tuning (`EOUConfig`).
</ParamField>

<ParamField path="interrupt_config" type="InterruptConfig | None">
  Barge-in / interruption tuning (`InterruptConfig`).
</ParamField>

<ParamField path="conversational_graph" type="Any">
  Optional structured conversation flow.
</ParamField>

<ParamField path="context_window" type="ContextWindow | Any">
  Conversation-history budget (`ContextWindow`).
</ParamField>

<ParamField path="voice_mail_detector" type="Any">
  Optional voicemail detector for outbound calls.
</ParamField>

<ParamField path="dtmf_handler" type="Any">
  Handler for touch-tone (DTMF) keypad input.
</ParamField>

<ParamField path="wake_up" type="int | None">
  Seconds of caller silence before a wake-up prompt fires.
</ParamField>

<ParamField path="wake_up_max_attempts" type="int | None">
  Maximum wake-up prompts before giving up.
</ParamField>

<ParamField path="realtime_config" type="RealtimeConfig | None">
  Speech-to-speech configuration (`RealtimeConfig`).
</ParamField>

<ParamField path="llm_stream_hook_enabled" type="bool" default="False">
  Enable the streaming `@pipeline.on("llm")` hook.
</ParamField>

<ParamField path="llm_stream_hook_timeout_ms" type="int" default="100">
  Per-chunk timeout for the LLM stream hook. Default 100.
</ParamField>

<ParamField path="playback_grace_ms" type="int" default="0">
  Grace period before treating playback as finished. Default 0.
</ParamField>

<ParamField path="stt_filter_patterns" type="list[str] | None">
  Regexes whose matching transcript fragments are dropped.
</ParamField>

<ParamField path="stt_word_substitutions" type="dict[str, str] | None">
  Literal word replacements applied to transcripts.
</ParamField>

<ParamField path="inactivity_timeout_seconds" type="int | None">
  End the session after this much silence.
</ParamField>

### change\_component

```python theme={null}
def change_component(self, *, stt: 'Any' = NO_CHANGE, llm: 'Any' = NO_CHANGE, tts: 'Any' = NO_CHANGE, vad: 'Any' = NO_CHANGE, turn_detector: 'Any' = NO_CHANGE, denoise: 'Any' = NO_CHANGE) -> 'None'
```

Hot-swap pipeline components mid-session, without tearing the call down.

<ParamField path="stt" type="Any" default="NO_CHANGE" />

<ParamField path="llm" type="Any" default="NO_CHANGE" />

<ParamField path="tts" type="Any" default="NO_CHANGE" />

<ParamField path="vad" type="Any" default="NO_CHANGE" />

<ParamField path="turn_detector" type="Any" default="NO_CHANGE" />

<ParamField path="denoise" type="Any" default="NO_CHANGE" />

### change\_pipeline

```python theme={null}
def change_pipeline(self, *, stt: 'Any' = None, llm: 'Any' = None, tts: 'Any' = None, vad: 'Any' = None, turn_detector: 'Any' = None, interrupt_config: 'Any' = None, **unsupported: 'Any') -> 'None'
```

Reconfigure several pipeline pieces mid-session in one call.

<ParamField path="stt" type="Any" />

<ParamField path="llm" type="Any" />

<ParamField path="tts" type="Any" />

<ParamField path="vad" type="Any" />

<ParamField path="turn_detector" type="Any" />

<ParamField path="interrupt_config" type="Any" />

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### get\_latest\_frames

```python theme={null}
def get_latest_frames(self, num_frames: 'int' = 1) -> 'list'
```

Return the most recent buffered vision frames.

<ParamField path="num_frames" type="int" default="1">
  Number of trailing frames to return. Default 1.
</ParamField>

<ResponseField name="returns" type="list">
  Up to `num_frames` most recent frames in chronological order, or an empty list when `num_frames` is non-positive.
</ResponseField>

### get\_session\_metrics\_snapshot

```python theme={null}
def get_session_metrics_snapshot(self) -> 'dict'
```

Return a snapshot of session metrics.

<ResponseField name="returns" type="dict">
  An empty dictionary. Metrics collection is reported through the `PipelineMetricsHooks` hooks rather than this snapshot.
</ResponseField>

### interrupt

```python theme={null}
def interrupt(self) -> 'None'
```

Interrupt the agent's current speech.

No-op when no agent or active session is attached.

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'str', callback: 'Callable | None' = None) -> 'Callable'
```

Register a pipeline-event hook.

Usable either as a decorator (`@pipeline.on("event")`) or by passing
the callback directly.

<ParamField path="event" type="str" required>
  Name of the pipeline event to subscribe to.
</ParamField>

<ParamField path="callback" type="Callable | None">
  Callback to register. If omitted, a decorator is returned.
</ParamField>

<ResponseField name="returns" type="Callable">
  The registered callback when `callback` is provided, otherwise a decorator that registers the callable it wraps.
</ResponseField>

### process\_text

```python theme={null}
def process_text(self, text: 'str') -> 'None'
```

Feed text into the session as if it were user input.

No-op when no agent or active session is attached.

<ParamField path="text" type="str" required>
  The text to process through the session.
</ParamField>

### push\_audio\_frame

```python theme={null}
def push_audio_frame(self, frame: 'dict') -> 'None'
```

Notify `audio_delta` hooks with an incoming audio frame.

<ParamField path="frame" type="dict" required>
  The audio frame to dispatch to registered hooks.
</ParamField>

### push\_vision\_frame

```python theme={null}
def push_vision_frame(self, frame: 'dict') -> 'None'
```

Buffer a vision frame and notify `vision_frame` hooks.

<ParamField path="frame" type="dict" required>
  The vision frame to append to the buffer and dispatch.
</ParamField>

### reply\_with\_context

```python theme={null}
def reply_with_context(self, instructions: 'str', **kwargs) -> 'None'
```

Generate an agent reply guided by the given instructions.

No-op when no agent or active session is attached.

<ParamField path="instructions" type="str" required>
  Instructions steering the generated reply.
</ParamField>

### send\_message

```python theme={null}
def send_message(self, text: 'str', **kwargs) -> 'None'
```

Have the agent speak the given text.

No-op when no agent or active session is attached.

<ParamField path="text" type="str" required>
  The text for the agent to say.
</ParamField>

### set\_agent

```python theme={null}
def set_agent(self, agent: 'Any') -> 'None'
```

Attach the agent that owns this pipeline.

<ParamField path="agent" type="Any" required>
  The agent instance to associate with the pipeline.
</ParamField>

***

## PipelineHooks

Registry of callbacks bound to pipeline events.

Tracks single-slot stream hooks (`stt`, `tts`, `llm_stream`) and
multi-subscriber event hooks. Callbacks are registered through the
`on` decorator and retrieved by the pipeline at the appropriate
points in the conversation lifecycle.

### Constructor

```python theme={null}
PipelineHooks() -> 'None'
```

### clear\_all\_hooks

```python theme={null}
def clear_all_hooks(self) -> 'None'
```

Remove every registered hook, resetting the registry to empty.

### get\_hooks

```python theme={null}
def get_hooks(self, event: 'str') -> 'list[Callable]'
```

Return the callbacks registered for a multi-subscriber event.

<ParamField path="event" type="str" required>
  Name of the event to look up.
</ParamField>

<ResponseField name="returns" type="list[Callable]">
  The list of registered callbacks, or an empty list if none.
</ResponseField>

### has\_llm\_observation\_hooks

```python theme={null}
def has_llm_observation_hooks(self) -> 'bool'
```

Return `True` if any non-streaming `llm` observation hooks exist.

<ResponseField name="returns" type="bool" />

### has\_llm\_stream\_hook

```python theme={null}
def has_llm_stream_hook(self) -> 'bool'
```

Return `True` if an LLM stream hook is registered.

<ResponseField name="returns" type="bool" />

### has\_stt\_stream\_hook

```python theme={null}
def has_stt_stream_hook(self) -> 'bool'
```

Return `True` if a speech-to-text stream hook is registered.

<ResponseField name="returns" type="bool" />

### has\_tts\_stream\_hook

```python theme={null}
def has_tts_stream_hook(self) -> 'bool'
```

Return `True` if a text-to-speech stream hook is registered.

<ResponseField name="returns" type="bool" />

### on

```python theme={null}
def on(self, event: 'str') -> 'Callable'
```

Create a decorator that registers a callback for an event.

<ParamField path="event" type="str" required>
  Name of the pipeline event to subscribe to.
</ParamField>

<ResponseField name="returns" type="Callable">
  A decorator that registers the wrapped callable and returns it unchanged.
</ResponseField>

### registered\_names

```python theme={null}
def registered_names(self) -> 'list[str]'
```

List the names of all events that currently have a hook registered.

<ResponseField name="returns" type="list[str]">
  Event names with at least one registered callback, including the single-slot `stt`, `tts`, and `llm_stream` hooks when set.
</ResponseField>

***

## PipelineHookError

Raised when a requested `@pipeline.on(...)` hook is not supported.

### Constructor

```python theme={null}
PipelineHookError(hook_name: 'str', reason: 'str', alternative: 'Optional[str]' = None) -> 'None'
```

<ParamField path="hook_name" type="str" required>
  Name of the unsupported hook.
</ParamField>

<ParamField path="reason" type="str" required>
  Explanation of why the hook is unsupported.
</ParamField>

<ParamField path="alternative" type="Optional[str]">
  Optional suggested alternative, appended to the message.
</ParamField>

***

## PipelineComponent

Individual components that can participate in the processing pipeline.

```python theme={null}
class PipelineComponent(Enum):
    STT = 'stt'
    LLM = 'llm'
    TTS = 'tts'
    VAD = 'vad'
    TURN_DETECTOR = 'turn_detector'
    AVATAR = 'avatar'
    DENOISE = 'denoise'
    REALTIME_MODEL = 'realtime_model'
```

***

## PipelineConfig

Immutable description of a resolved pipeline configuration.

### Fields

<ParamField path="pipeline_mode" type="PipelineMode" required>
  The high-level operating mode of the pipeline.
</ParamField>

<ParamField path="realtime_mode" type="RealtimeMode | None" required>
  The realtime model mode, or `None` if not in a realtime mode.
</ParamField>

<ParamField path="is_realtime" type="bool" required>
  Whether the pipeline operates in a realtime mode.
</ParamField>

<ParamField path="active_components" type="frozenset[PipelineComponent]" required>
  The set of components active in this pipeline.
</ParamField>

### has\_component

```python theme={null}
def has_component(self, component: 'PipelineComponent') -> 'bool'
```

Check whether a component is active in this pipeline.

<ParamField path="component" type="PipelineComponent" required>
  The pipeline component to look for.
</ParamField>

<ResponseField name="returns" type="bool">
  `True` if the component is part of the active components.
</ResponseField>

***

## PipelineMode

Operating modes of the processing pipeline, defining which components are active.

```python theme={null}
class PipelineMode(Enum):
    REALTIME = 'realtime'
    FULL_CASCADING = 'full_cascading'
    LLM_TTS_ONLY = 'llm_tts_only'
    STT_LLM_ONLY = 'stt_llm_only'
    LLM_ONLY = 'llm_only'
    STT_ONLY = 'stt_only'
    TTS_ONLY = 'tts_only'
    STT_TTS_ONLY = 'stt_tts_only'
    HYBRID = 'hybrid'
    PARTIAL_CASCADING = 'partial_cascading'
```

***

## RealtimeMode

Modes for a realtime model, ranging from full speech-to-speech to hybrid configurations.

```python theme={null}
class RealtimeMode(Enum):
    FULL_S2S = 'full_s2s'
    HYBRID_STT = 'hybrid_stt'
    HYBRID_TTS = 'hybrid_tts'
    LLM_ONLY = 'llm_only'
```

***

## STT

Abstract base class for speech-to-text providers.

Implementations convert audio input into transcription events and may
emit an `'error'` event when a failure occurs.

### Constructor

```python theme={null}
STT() -> 'None'
```

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

### on\_stt\_transcript

```python theme={null}
def on_stt_transcript(self, callback: 'Callable[[STTResponse], Awaitable[None]]') -> 'None'
```

Register the coroutine invoked when a transcription event is produced.

<ParamField path="callback" type="Callable[[STTResponse], Awaitable[None]]" required>
  An async callable that receives each `STTResponse`.
</ParamField>

### process\_audio

```python theme={null}
def process_audio(self, audio_frames: 'bytes', language: 'Optional[str]' = None, **kwargs: 'Any') -> 'None'
```

Process a chunk of audio for transcription.

Implementations should consume the audio frames and emit transcription
results through the registered transcript callback.

<ParamField path="audio_frames" type="bytes" required>
  Raw audio bytes to transcribe.
</ParamField>

<ParamField path="language" type="Optional[str]">
  Optional language code to guide recognition.
</ParamField>

### stream\_transcribe

```python theme={null}
def stream_transcribe(self, audio_stream: 'AsyncIterator[bytes]', **kwargs: 'Any') -> 'AsyncIterator[STTResponse]'
```

Transcribe a continuous audio stream.

<ParamField path="audio_stream" type="AsyncIterator[bytes]" required>
  An async iterator yielding raw audio byte chunks.
</ParamField>

<ResponseField name="returns" type="AsyncIterator[STTResponse]">
  STTResponse: Transcription events produced as audio is consumed.
</ResponseField>

***

## STTResponse

Event emitted by a speech-to-text provider.

### Fields

<ParamField path="event_type" type="SpeechEventType" required>
  The type of speech event this response represents.
</ParamField>

<ParamField path="data" type="SpeechData" required>
  The transcription payload associated with the event.
</ParamField>

<ParamField path="metadata" type="Optional[dict[str, Any]]">
  Optional provider-specific metadata.
</ParamField>

***

## SpeechData

Transcription payload produced by a speech-to-text provider.

### Fields

<ParamField path="text" type="str" required>
  The recognized text.
</ParamField>

<ParamField path="confidence" type="float" default="0.0">
  Recognition confidence score in the range 0.0 to 1.0.
</ParamField>

<ParamField path="language" type="Optional[str]">
  Detected or configured language code, if available.
</ParamField>

<ParamField path="start_time" type="float" default="0.0">
  Start time of the recognized segment, in seconds.
</ParamField>

<ParamField path="end_time" type="float" default="0.0">
  End time of the recognized segment, in seconds.
</ParamField>

<ParamField path="duration" type="float" default="0.0">
  Duration of the recognized segment, in seconds.
</ParamField>

***

## LLM

Abstract base class for language model providers.

Implementations generate streamed chat responses and may emit an
`'error'` event when a failure occurs.

### Constructor

```python theme={null}
LLM() -> 'None'
```

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### cancel\_current\_generation

```python theme={null}
def cancel_current_generation(self) -> 'None'
```

Cancel the in-progress generation for the active session, if any.

### chat

```python theme={null}
def chat(self, messages: 'Any', tools: 'Any' = None, conversational_graph: 'Any' = None, **kwargs: 'Any') -> 'AsyncIterator[LLMResponse]'
```

Generate a streamed chat completion.

<ParamField path="messages" type="Any" required>
  The conversation history to send to the model.
</ParamField>

<ParamField path="tools" type="Any">
  Optional tool definitions the model may call.
</ParamField>

<ParamField path="conversational_graph" type="Any">
  Optional conversational graph guiding the exchange.
</ParamField>

<ResponseField name="returns" type="AsyncIterator[LLMResponse]">
  LLMResponse: Response chunks produced as the model generates output.
</ResponseField>

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

***

## LLMResponse

A single response chunk produced by a language model provider.

### Fields

<ParamField path="content" type="str" required>
  The text content of the response.
</ParamField>

<ParamField path="role" type="ChatRole" required>
  The chat role associated with the content.
</ParamField>

<ParamField path="metadata" type="Optional[dict[str, Any]]">
  Optional provider-specific metadata.
</ParamField>

***

## TTS

Abstract base class for text-to-speech providers.

Implementations synthesize text into audio output and may emit an
`'error'` event when a failure occurs.

### Constructor

```python theme={null}
TTS(sample_rate: 'int' = 24000, num_channels: 'int' = 1) -> 'None'
```

<ParamField path="sample_rate" type="int" default="24000">
  Sample rate of the synthesized audio, in Hz.
</ParamField>

<ParamField path="num_channels" type="int" default="1">
  Number of audio channels in the synthesized output.
</ParamField>

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### interrupt

```python theme={null}
def interrupt(self) -> 'None'
```

Interrupt and cancel the current synthesis for the active session, if any.

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

### on\_first\_audio\_byte

```python theme={null}
def on_first_audio_byte(self, callback: "'Callable[[int, int], Awaitable[None] | None]'") -> 'None'
```

Register the callback invoked when the first audio byte is produced.

<ParamField path="callback" type="'Callable[[int, int], Awaitable[None] | None]'" required>
  A callable, optionally async, invoked with synthesis timing information when the first audio byte is emitted.
</ParamField>

### synthesize

```python theme={null}
def synthesize(self, text: 'Any', **kwargs: 'Any') -> 'None'
```

Synthesize the given text into speech.

Implementations should convert the text to audio. The default behavior
forwards the text to the active session for playback when one is present.

<ParamField path="text" type="Any" required>
  The text to synthesize; coerced to a string if needed.
</ParamField>

***

## VAD

Abstract base class for voice activity detection providers.

Implementations analyze audio to detect speech and silence boundaries and
may emit `'error'` and `'info'` events.

### Constructor

```python theme={null}
VAD(sample_rate: 'int' = 16000, threshold: 'float' = 0.5, min_speech_duration: 'float' = 0.5, min_silence_duration: 'float' = 0.5) -> 'None'
```

<ParamField path="sample_rate" type="int" default="16000">
  Sample rate of the input audio, in Hz.
</ParamField>

<ParamField path="threshold" type="float" default="0.5">
  Speech probability threshold above which audio is treated as speech.
</ParamField>

<ParamField path="min_speech_duration" type="float" default="0.5">
  Minimum duration, in seconds, required to confirm a speech segment.
</ParamField>

<ParamField path="min_silence_duration" type="float" default="0.5">
  Minimum duration, in seconds, required to confirm a silence segment.
</ParamField>

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### flush

```python theme={null}
def flush(self) -> 'None'
```

Flush any buffered audio and finalize pending detection state.

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

### on\_vad\_event

```python theme={null}
def on_vad_event(self, callback: 'Callable[[VADResponse], Awaitable[None]]') -> 'None'
```

Register the coroutine invoked when a voice activity event occurs.

<ParamField path="callback" type="Callable[[VADResponse], Awaitable[None]]" required>
  An async callable that receives each `VADResponse`.
</ParamField>

### process\_audio

```python theme={null}
def process_audio(self, audio_frames: 'bytes', **kwargs: 'Any') -> 'None'
```

Analyze a chunk of audio for voice activity.

Implementations should detect speech and silence transitions and emit
results through the registered voice activity callback.

<ParamField path="audio_frames" type="bytes" required>
  Raw audio bytes to analyze.
</ParamField>

***

## VADResponse

Event emitted by a voice activity detection provider.

### Fields

<ParamField path="event_type" type="VADEventType" required>
  The type of voice activity event this response represents.
</ParamField>

<ParamField path="data" type="VADData" required>
  The voice activity payload associated with the event.
</ParamField>

<ParamField path="metadata" type="Optional[dict[str, Any]]">
  Optional provider-specific metadata.
</ParamField>

***

## VADData

Voice activity detection payload describing the current speech state.

### Fields

<ParamField path="is_speech" type="bool" required>
  Whether speech is currently detected.
</ParamField>

<ParamField path="confidence" type="float" default="0.0">
  Detection confidence score in the range 0.0 to 1.0.
</ParamField>

<ParamField path="timestamp" type="float" default="0.0">
  Time of the detection, in seconds.
</ParamField>

<ParamField path="speech_duration" type="float" default="0.0">
  Accumulated duration of detected speech, in seconds.
</ParamField>

<ParamField path="silence_duration" type="float" default="0.0">
  Accumulated duration of detected silence, in seconds.
</ParamField>

***

## EOU

Abstract base class for end-of-utterance detection providers.

Implementations estimate the likelihood that a speaker has finished their
turn and may emit an `'error'` event when a failure occurs.

### Constructor

```python theme={null}
EOU(threshold: 'float' = 0.7) -> 'None'
```

<ParamField path="threshold" type="float" default="0.7">
  Probability threshold above which an utterance is considered complete.
</ParamField>

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### cleanup

```python theme={null}
def cleanup(self) -> 'None'
```

Perform any additional teardown required by the provider.

### detect\_end\_of\_utterance

```python theme={null}
def detect_end_of_utterance(self, chat_context: 'Any', threshold: 'Optional[float]' = None) -> 'bool'
```

Determine whether the current utterance has ended.

<ParamField path="chat_context" type="Any" required>
  The conversation context used to evaluate the utterance.
</ParamField>

<ParamField path="threshold" type="Optional[float]">
  Optional probability threshold overriding the configured one.
</ParamField>

<ResponseField name="returns" type="bool">
  `True` if the end-of-utterance probability meets or exceeds the threshold, otherwise `False`.
</ResponseField>

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### get\_eou\_probability

```python theme={null}
def get_eou_probability(self, chat_context: 'Any') -> 'float'
```

Return the estimated probability that the current utterance has ended.

Reads the most recent detection result from the active session, falling
back to `0.5` when no result is available or the value cannot be parsed.

<ParamField path="chat_context" type="Any" required>
  The conversation context used to evaluate the utterance.
</ParamField>

<ResponseField name="returns" type="float">
  The end-of-utterance probability in the range 0.0 to 1.0.
</ResponseField>

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

### set\_threshold

```python theme={null}
def set_threshold(self, threshold: 'float') -> 'None'
```

Update the end-of-utterance probability threshold.

<ParamField path="threshold" type="float" required>
  The new probability threshold.
</ParamField>

***

## RealtimeBaseModel

Base class for realtime models supporting asynchronous lifecycle management.

Provides an asynchronous context manager interface and a no-op cleanup
contract that subclasses may override to release resources.

### Constructor

```python theme={null}
RealtimeBaseModel()
```

### aclose

```python theme={null}
def aclose(self) -> None
```

Release resources held by the model.

The base implementation performs no work. Subclasses should override
this method to close connections or free resources.

<ResponseField name="returns">
  `None`.
</ResponseField>

### cleanup

```python theme={null}
def cleanup(self) -> None
```

Clean up the model by delegating to `aclose`.

<ResponseField name="returns">
  `None`.
</ResponseField>

***

## RealtimeLLMAdapter

```python theme={null}
def RealtimeLLMAdapter(realtime_model, agent=None)
```

Mark a realtime model so the pipeline treats it as a realtime LLM.

Tags the given model with the internal flag the pipeline checks to route it
through the realtime (speech-to-speech) path, then returns the same object.

<ParamField path="realtime_model" required>
  The realtime model instance to adapt.
</ParamField>

<ParamField path="agent">
  Optional agent the model is associated with. Currently unused.
</ParamField>

<ResponseField name="returns">
  The same `realtime_model` instance, tagged as a realtime model.
</ResponseField>

***

## FallbackSTT

Speech-to-text provider that falls back across an ordered list of providers.

The first provider is used as the primary. When a provider fails, the
fallback machinery temporarily disables it and routes to the next available
provider, optionally disabling it permanently after repeated failures.

### Constructor

```python theme={null}
FallbackSTT(providers: 'List[STT]', *, temporary_disable_sec: 'int' = 60, permanent_disable_after_attempts: 'int' = 3, recovery_check_interval_sec: 'int' = 300, **kwargs: 'Any') -> 'None'
```

<ParamField path="providers" type="List[STT]" required>
  Ordered list of STT providers; the first is the primary.
</ParamField>

<ParamField path="temporary_disable_sec" type="int" default="60">
  Duration in seconds a failing provider is temporarily disabled before being retried.
</ParamField>

<ParamField path="permanent_disable_after_attempts" type="int" default="3">
  Number of failed attempts after which a provider is permanently disabled.
</ParamField>

<ParamField path="recovery_check_interval_sec" type="int" default="300">
  Interval in seconds between checks for recovery of a disabled provider.
</ParamField>

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

### on\_stt\_transcript

```python theme={null}
def on_stt_transcript(self, callback: 'Callable[[STTResponse], Awaitable[None]]') -> 'None'
```

Register the coroutine invoked when a transcription event is produced.

<ParamField path="callback" type="Callable[[STTResponse], Awaitable[None]]" required>
  An async callable that receives each `STTResponse`.
</ParamField>

### process\_audio

```python theme={null}
def process_audio(self, audio_frames: 'bytes', language: 'Optional[str]' = None, **kwargs: 'Any') -> 'None'
```

Process a chunk of audio for transcription.

Implementations should consume the audio frames and emit transcription
results through the registered transcript callback.

<ParamField path="audio_frames" type="bytes" required>
  Raw audio bytes to transcribe.
</ParamField>

<ParamField path="language" type="Optional[str]">
  Optional language code to guide recognition.
</ParamField>

### stream\_transcribe

```python theme={null}
def stream_transcribe(self, audio_stream: 'AsyncIterator[bytes]', **kwargs: 'Any') -> 'AsyncIterator[STTResponse]'
```

Transcribe a continuous audio stream.

<ParamField path="audio_stream" type="AsyncIterator[bytes]" required>
  An async iterator yielding raw audio byte chunks.
</ParamField>

<ResponseField name="returns" type="AsyncIterator[STTResponse]">
  STTResponse: Transcription events produced as audio is consumed.
</ResponseField>

***

## FallbackLLM

Language model provider that falls back across an ordered list of providers.

The first provider is used as the primary. When a provider fails, the
fallback machinery temporarily disables it and routes to the next available
provider, optionally disabling it permanently after repeated failures.

### Constructor

```python theme={null}
FallbackLLM(providers: 'List[LLM]', *, temporary_disable_sec: 'int' = 60, permanent_disable_after_attempts: 'int' = 3, recovery_check_interval_sec: 'int' = 300, **kwargs: 'Any') -> 'None'
```

<ParamField path="providers" type="List[LLM]" required>
  Ordered list of LLM providers; the first is the primary.
</ParamField>

<ParamField path="temporary_disable_sec" type="int" default="60">
  Duration in seconds a failing provider is temporarily disabled before being retried.
</ParamField>

<ParamField path="permanent_disable_after_attempts" type="int" default="3">
  Number of failed attempts after which a provider is permanently disabled.
</ParamField>

<ParamField path="recovery_check_interval_sec" type="int" default="300">
  Interval in seconds between checks for recovery of a disabled provider.
</ParamField>

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### cancel\_current\_generation

```python theme={null}
def cancel_current_generation(self) -> 'None'
```

Cancel the in-progress generation for the active session, if any.

### chat

```python theme={null}
def chat(self, messages: 'Any', tools: 'Any' = None, conversational_graph: 'Any' = None, **kwargs: 'Any') -> 'AsyncIterator[LLMResponse]'
```

Generate a streamed chat completion.

<ParamField path="messages" type="Any" required>
  The conversation history to send to the model.
</ParamField>

<ParamField path="tools" type="Any">
  Optional tool definitions the model may call.
</ParamField>

<ParamField path="conversational_graph" type="Any">
  Optional conversational graph guiding the exchange.
</ParamField>

<ResponseField name="returns" type="AsyncIterator[LLMResponse]">
  LLMResponse: Response chunks produced as the model generates output.
</ResponseField>

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

***

## FallbackTTS

Text-to-speech provider that falls back across an ordered list of providers.

The first provider is used as the primary. When a provider fails, the
fallback machinery temporarily disables it and routes to the next available
provider, optionally disabling it permanently after repeated failures.

### Constructor

```python theme={null}
FallbackTTS(providers: 'List[TTS]', *, temporary_disable_sec: 'int' = 60, permanent_disable_after_attempts: 'int' = 3, recovery_check_interval_sec: 'int' = 300, **kwargs: 'Any') -> 'None'
```

<ParamField path="providers" type="List[TTS]" required>
  Ordered list of TTS providers; the first is the primary.
</ParamField>

<ParamField path="temporary_disable_sec" type="int" default="60">
  Duration in seconds a failing provider is temporarily disabled before being retried.
</ParamField>

<ParamField path="permanent_disable_after_attempts" type="int" default="3">
  Number of failed attempts after which a provider is permanently disabled.
</ParamField>

<ParamField path="recovery_check_interval_sec" type="int" default="300">
  Interval in seconds between checks for recovery of a disabled provider.
</ParamField>

### aclose

```python theme={null}
def aclose(self) -> 'None'
```

Release any resources held by the provider.

### emit

```python theme={null}
def emit(self, event: 'T', *args: 'Any') -> 'None'
```

Emit an event, invoking all registered handlers with the given arguments.

Handlers are called in registration order. Coroutine handlers are
scheduled on the running event loop. If the emitter is closed, the call
is a no-op.

<ParamField path="event" type="T" required>
  The event to emit.
</ParamField>

### interrupt

```python theme={null}
def interrupt(self) -> 'None'
```

Interrupt and cancel the current synthesis for the active session, if any.

### off

```python theme={null}
def off(self, event: 'T', callback: 'Callable[..., Any]') -> 'None'
```

Remove a previously registered handler for an event.

If the handler is not registered for the event, the call is a no-op.

<ParamField path="event" type="T" required>
  The event the handler was registered for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any]" required>
  The handler to remove.
</ParamField>

### on

```python theme={null}
def on(self, event: 'T', callback: 'Callable[..., Any] | None' = None) -> 'Callable[..., Any]'
```

Register a handler for an event.

Can be used directly by passing a callback, or as a decorator when
`callback` is omitted.

<ParamField path="event" type="T" required>
  The event to listen for.
</ParamField>

<ParamField path="callback" type="Callable[..., Any] | None">
  The handler to invoke when the event is emitted. If `None`, a decorator is returned that registers the decorated function.
</ParamField>

<ResponseField name="returns" type="Callable[..., Any]">
  The registered callback when `callback` is provided, otherwise a decorator that registers and returns the function it wraps.
</ResponseField>

### on\_first\_audio\_byte

```python theme={null}
def on_first_audio_byte(self, callback: "'Callable[[int, int], Awaitable[None] | None]'") -> 'None'
```

Register the callback invoked when the first audio byte is produced.

<ParamField path="callback" type="'Callable[[int, int], Awaitable[None] | None]'" required>
  A callable, optionally async, invoked with synthesis timing information when the first audio byte is emitted.
</ParamField>

### synthesize

```python theme={null}
def synthesize(self, text: 'Any', **kwargs: 'Any') -> 'None'
```

Synthesize the given text into speech.

Implementations should convert the text to audio. The default behavior
forwards the text to the active session for playback when one is present.

<ParamField path="text" type="Any" required>
  The text to synthesize; coerced to a string if needed.
</ParamField>

***

## UtteranceHandle

Tracks the lifecycle of a single spoken utterance.

Provides identity and interruption state for an utterance, lets callers
register callbacks for when the final audio byte is emitted, and can be
awaited to block until the utterance completes or is interrupted.

### Constructor

```python theme={null}
UtteranceHandle(utterance_id: 'str' = None, interruptible: 'bool' = True)
```

<ParamField path="utterance_id" type="str">
  Identifier for the utterance. A random UUID string is generated when not provided.
</ParamField>

<ParamField path="interruptible" type="bool" default="True">
  Whether the utterance may be interrupted without forcing. Defaults to `True`.
</ParamField>

### done

```python theme={null}
def done(self) -> 'bool'
```

Report whether the utterance has finished or been interrupted.

<ResponseField name="returns" type="bool">
  `True` if the utterance has completed or been interrupted, otherwise `False`.
</ResponseField>

### interrupt

```python theme={null}
def interrupt(self, *, force: 'bool' = False) -> 'None'
```

Interrupt the utterance, marking it done and firing callbacks.

When the utterance is interruptible, or when `force` is set, the
utterance is marked as interrupted and completed and any registered
last-audio-byte callbacks are invoked. Otherwise the call is a no-op.

<ParamField path="force" type="bool" default="False">
  Interrupt even when the utterance is not interruptible. Defaults to `False`.
</ParamField>

### on\_last\_audio\_byte

```python theme={null}
def on_last_audio_byte(self, callback: 'Callable[[], Any]') -> 'Callable[[], Any]'
```

Register a callback for when the final audio byte is emitted.

If the utterance is already done, the callback is invoked immediately;
otherwise it is queued and invoked when the utterance completes or is
interrupted. The callback may be a regular function or a coroutine
function.

<ParamField path="callback" type="Callable[[], Any]" required>
  Zero-argument callable invoked when the last audio byte is reached.
</ParamField>

<ResponseField name="returns" type="Callable[[], Any]">
  The same `callback`, allowing use as a decorator.
</ResponseField>