> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipeline

> Python API reference for Pipeline.

The voice stack an agent runs on: speech-to-text, LLM, and text-to-speech, plus
voice-activity and turn detection. `Pipeline` wires these components together and
exposes hooks for observing and shaping each turn.

## Pipeline

The providers a call runs on: what hears, what thinks, what speaks.

Two shapes are valid. A cascade names `stt`, `llm` and `tts`
separately; a realtime pipeline names a speech-to-speech model in `llm`
alone, because a realtime model is the language model. `realtime=` is
accepted as the older spelling of the same slot; naming both is rejected.

A fallback slot -- `stt`, `llm`, `tts` -- also takes a list, in which
case the head serves and the tail stands by. Wrap that list in `FallbackSTT`,
`FallbackLLM` or `FallbackTTS` to say how it demotes as well as what it
falls back to; each slot carries its own settings.

### Fields

<ParamField path="stt" type="Any">
  Speech-to-text, or a `FallbackSTT` chain. Omit it on a realtime pipeline.
</ParamField>

<ParamField path="llm" type="Any">
  The language model -- a text LLM for a cascade, a `FallbackLLM` chain of them, or a speech-to-speech model for a realtime pipeline.
</ParamField>

<ParamField path="tts" type="Any">
  Text-to-speech, or a `FallbackTTS` chain. Omit it on a realtime pipeline.
</ParamField>

<ParamField path="vad" type="Any">
  Voice activity detection -- what notices the caller is talking.
</ParamField>

<ParamField path="turn_detector" type="Any">
  End-of-turn detection, for deciding when they finished.
</ParamField>

<ParamField path="denoise" type="Any">
  Noise cancellation applied to the inbound audio.
</ParamField>

<ParamField path="realtime" type="Any">
  The older spelling of a speech-to-speech `llm`.
</ParamField>

<ParamField path="avatar" type="Any">
  A video avatar to render the agent's speech through.
</ParamField>

<ParamField path="eou_config" type="Any">
  `EOUConfig` -- how patiently the agent waits before answering.
</ParamField>

<ParamField path="interrupt_config" type="Any">
  `InterruptConfig` -- what counts as the caller barging in.
</ParamField>

<ParamField path="pronunciations" type="list[PronunciationRule]" default="…">
  `PronunciationRule` substitutions applied to generated text on its way to TTS.
</ParamField>

<ParamField path="context_window" type="Optional[ContextWindow]">
  `ContextWindow` -- how a long conversation is kept inside the model's limit.
</ParamField>

<ParamField path="dtmf_handler" type="Any">
  `DTMFHandler` -- deliver keypad tones to the agent. `None` drops them.
</ParamField>

<ParamField path="voice_mail_detector" type="Any">
  `VoiceMailDetector` -- answering-machine detection, for outbound calls that may be picked up by one.
</ParamField>

<ParamField path="_chains" type="dict[str, tuple[ProviderSpec, ...]]" default="…" />

<ParamField path="_fallback_settings" type="dict[str, _Fallback]" default="…" />

<ParamField path="_hooks" type="dict[str, list]" default="…" />

### on

```python theme={null}
def on(self, event: 'str', callback: 'Any' = None) -> 'Any'
```

Register a hook that runs in this process, mid-turn.

Decorator or direct call. The events are the transcript hooks `stt`
and `llm`, and the turn hooks `user_turn_start`, `user_turn_end`,
`agent_turn_start` and `agent_turn_end`.

An async generator registered on `llm` is filed as `llm_stream`: it
is handed the model's output as a stream and yields what should be
spoken, so it can buffer, rewrite or drop the turn. A plain coroutine on
the same event sees the finished text instead.

Per-component latency is a separate family, registered through its own
namespace rather than by spelling the prefix here::

@pipeline.metrics.on("stt")
def on\_stt(data: dict) -> None:
...

<ParamField path="event" type="str" required>
  Which hook to attach to.
</ParamField>

<ParamField path="callback" type="Any">
  The handler. Omitted, this returns a decorator.
</ParamField>

<ResponseField name="returns" type="Any">
  The decorator, or the handler it registered.
</ResponseField>

### hooks

```python theme={null}
def hooks(self, event: 'str') -> 'list'
```

The handlers registered for one event, in registration order.

<ParamField path="event" type="str" required />

<ResponseField name="returns" type="list" />

### chain

```python theme={null}
def chain(self, slot: 'str') -> 'tuple[ProviderSpec, ...]'
```

Everything configured for one slot: the primary, then its fallbacks.

Empty for a slot nothing was named for, and for a slot that is not one.

<ParamField path="slot" type="str" required />

<ResponseField name="returns" type="tuple[ProviderSpec, ...]" />

### primary

```python theme={null}
def primary(self, slot: 'str') -> 'ProviderSpec | None'
```

The provider a slot uses first, or `None` if it has none.

<ParamField path="slot" type="str" required />

<ResponseField name="returns" type="ProviderSpec | None" />

### fallbacks

```python theme={null}
def fallbacks(self, slot: 'str') -> 'tuple[ProviderSpec, ...]'
```

What a slot falls back to, in the order they are tried.

<ParamField path="slot" type="str" required />

<ResponseField name="returns" type="tuple[ProviderSpec, ...]" />

### fallback\_settings

```python theme={null}
def fallback_settings(self, slot: 'str') -> 'Optional[_Fallback]'
```

How one slot demotes, or `None` where nothing was tuned.

`fallbacks` is who a slot falls back to; this is how.

<ParamField path="slot" type="str" required />

<ResponseField name="returns" type="Optional[_Fallback]" />

### providers

```python theme={null}
def providers(self) -> 'Iterable[ProviderSpec]'
```

Every provider in the pipeline, fallbacks included.

<ResponseField name="returns" type="Iterable[ProviderSpec]" />

***

## NO\_CHANGE

```python theme={null}
NO_CHANGE = NO_CHANGE
```

"Leave this slot as it is." Distinct from None, which is a real answer:
`change_component(denoise=None)` takes denoising off a running call, while
leaving `denoise` out keeps whatever the call is already denoising with.

***

## PronunciationRule

Rewrite generated text on its way to TTS.

### Fields

<ParamField path="find" type="str" required>
  The text to look for.
</ParamField>

<ParamField path="replace" type="str" required>
  What to say instead.
</ParamField>

<ParamField path="case_sensitive" type="bool" default="False">
  Match case exactly. Off by default, because the model capitalises the same word differently at the start of a sentence.
</ParamField>

***

## ContextWindow

Keep a long conversation inside the model's context, automatically.

### Fields

<ParamField path="max_tokens" type="Optional[int]">
  Compress once the context exceeds this. `None` for no token ceiling.
</ParamField>

<ParamField path="max_context_items" type="Optional[int]">
  Or bound by item count. `None` for none.
</ParamField>

<ParamField path="keep_recent_turns" type="int" default="3">
  Recent user turns kept verbatim. The SDK's default is 3, and it is worth keeping several -- summarising the sentence the caller just said is how an agent starts answering the wrong question.
</ParamField>

<ParamField path="max_tool_calls_per_turn" type="int" default="10">
  The SDK's default is 10.
</ParamField>

<ParamField path="summary_llm" type="Any">
  The model that compresses. A second LLM, described here and built in the agent process like every other provider.
</ParamField>

***

## EOUConfig

When the caller is judged to have finished speaking.

### Fields

<ParamField path="mode" type="str" default="DEFAULT">
  `DEFAULT` waits a fixed time after speech stops. `ADAPTIVE` varies it with how certain the turn detector is -- longer when the sentence sounds unfinished.
</ParamField>

<ParamField path="min_max_speech_wait_timeout" type="List[float]" default="…">
  `[min, max]` seconds to wait before treating silence as the end of a turn. `[0.0, 0.0]` answers as soon as the detector says the turn is complete, which is snappy but cuts off anyone who pauses to think.
</ParamField>

<ParamField path="eou_certainty_threshold" type="float" default="0.75">
  How sure the detector must be. Lower answers sooner and interrupts more.
</ParamField>

<ParamField path="backchannel_classification" type="Optional[bool]">
  Treat "mhm", "haan", "right" as listening noises rather than turns. `None` takes the runtime's default; the SDK's own default is on for detectors that support it.
</ParamField>

***

## InterruptConfig

What counts as the caller interrupting, rather than just making a noise.

### Fields

<ParamField path="mode" type="str" default="HYBRID">
  `VAD_ONLY` reacts to any speech-like audio -- fastest, and the most easily fooled by a cough or a door. `STT_ONLY` waits for words. `HYBRID` uses both.
</ParamField>

<ParamField path="interrupt_min_duration" type="float" default="0.0">
  Seconds of speech before it counts. Raising this is the usual fix for an agent that stops every time someone breathes.
</ParamField>

<ParamField path="interrupt_min_words" type="int" default="0">
  Words required before it counts. `2` ignores a stray "yeah".
</ParamField>

<ParamField path="interrupt_min_confidence" type="float" default="0.0">
  STT confidence floor for those words.
</ParamField>

<ParamField path="false_interrupt_pause_duration" type="float" default="0.0">
  How long to stay paused after an interruption that turned out to be nothing.
</ParamField>

<ParamField path="resume_on_false_interrupt" type="bool" default="False">
  Pick the sentence back up when the interruption proves false, rather than dropping it.
</ParamField>

<ParamField path="interrupt_fade_duration" type="float" default="0.0">
  Seconds to fade the agent's audio out over. An instant cut sounds like a dropped call.
</ParamField>

***

## FallbackSTT

Speech-to-text providers to try in order, and how they demote.

### Fields

<ParamField path="providers" type="Sequence[ProviderSpec]" default="()" />

<ParamField path="temporary_disable_sec" type="Optional[float]" />

<ParamField path="permanent_disable_after_attempts" type="Optional[int]" />

<ParamField path="latency_threshold_ms" type="Optional[float]" />

<ParamField path="consecutive_latency_hits" type="Optional[int]" />

***

## FallbackLLM

Language models to try in order, and how they demote.

### Fields

<ParamField path="providers" type="Sequence[ProviderSpec]" default="()" />

<ParamField path="temporary_disable_sec" type="Optional[float]" />

<ParamField path="permanent_disable_after_attempts" type="Optional[int]" />

<ParamField path="latency_threshold_ms" type="Optional[float]" />

<ParamField path="consecutive_latency_hits" type="Optional[int]" />

***

## FallbackTTS

Text-to-speech providers to try in order, and how they demote.

### Fields

<ParamField path="providers" type="Sequence[ProviderSpec]" default="()" />

<ParamField path="temporary_disable_sec" type="Optional[float]" />

<ParamField path="permanent_disable_after_attempts" type="Optional[int]" />

<ParamField path="latency_threshold_ms" type="Optional[float]" />

<ParamField path="consecutive_latency_hits" type="Optional[int]" />

***

## DTMFHandler

Deliver the caller's keypad tones to the agent instead of dropping them.

### Fields

<ParamField path="callback" type="Any">
  What runs per keypress, taking `(key)` or `(key, payload)`. Left `None`, the agent's `on_dtmf` method is called instead -- which is the usual shape, because a keypad menu almost always wants the agent's own state to accumulate a multi-digit entry into. The runtime delivers one key per call either way; a PIN is accumulated on your side, not handed over whole.
</ParamField>

***

## VoiceMailDetector

Detect an answering machine on an outbound call.

The detector runs in the agent process the ZeroRuntime starts: it buffers
the opening speech for `duration` seconds and asks `llm` whether it is a
person or a greeting.

### Fields

<ParamField path="llm" type="Any">
  The classifier. Required: without one there is nothing to classify with, and the detector is simply not installed.
</ParamField>

<ParamField path="callback" type="Any">
  What runs on detection. Left `None`, the agent's `on_voicemail` method is called instead. Either is awaited, so anything said in it finishes before the call is ended.
</ParamField>

<ParamField path="duration" type="float" default="2.0">
  Seconds of speech to buffer before deciding. The default is 2.0 -- long enough for "Hi, you've reached...".
</ParamField>

<ParamField path="custom_prompt" type="Optional[str]">
  Replace the built-in classification prompt.
</ParamField>

<ParamField path="enabled" type="bool" default="True">
  Set False to configure it without turning it on.
</ParamField>

***

## PipelineMode

Which shape a pipeline resolved to.

Derived from the slots that were filled rather than set by hand; read it
off `Pipeline.mode`.

```python theme={null}
class PipelineMode(Enum):
    REALTIME = 'realtime'
    FULL_CASCADING = 'full_cascading'
    LLM_TTS_ONLY = 'llm_tts_only'
    STT_LLM_ONLY = 'stt_llm_only'
    LLM_ONLY = 'llm_only'
    STT_ONLY = 'stt_only'
    TTS_ONLY = 'tts_only'
    STT_TTS_ONLY = 'stt_tts_only'
    HYBRID = 'hybrid'
    PARTIAL_CASCADING = 'partial_cascading'
```