> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Modes
> Cascade, Hybrid, and Realtime, the three pipeline modes the Pipeline auto-detects from the components you pass in.
The `Pipeline` is self-configuring: based on the components you hand it, it automatically runs in **Cascade**, **Hybrid**, or **Realtime** mode. Switch between the tabs below to compare each mode's architecture, when to reach for it, and the code that selects it.
#### Supported Modes
There are three modes supported by the Pipeline.
##### 1. Cascade (STT-LLM-TTS)
##### 2. Realtime (LLM)
##### 3. Hybrid (Cascade + Realtime)
Cascade mode runs the pipeline as separate stages: VAD → STT → Turn Detector → LLM → TTS. The `Pipeline` auto-detects it when you pass all these components, giving you full control over each stage.
**Architecture**
Audio flows: VAD detects speech, STT transcribes, the turn detector signals end of turn, the LLM generates a reply, and TTS synthesizes audio.
**When to use it**
* You need granular control over each stage (STT, LLM, TTS, VAD, turn detection).
* You want to mix and match providers for individual stages.
* You rely on custom hooks for turn-taking, RAG, or content filtering.
* You can trade higher latency for maximum flexibility.
**Example**
```python title="main.py" Python theme={null}
import zrt
from zrt import Agent, Pipeline
from zrt.plugins import DeepgramSTT, OpenAILLM, ElevenLabsTTS, SileroVAD, TurnDetector
from zrt.inference import AICousticsDenoise
AGENT_ID = "assistant"
pipeline = Pipeline(
stt=DeepgramSTT(),
llm=OpenAILLM(),
tts=ElevenLabsTTS(),
vad=SileroVAD(),
turn_detector=TurnDetector(model="echo-large"),
denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
)
class Assistant(Agent):
def __init__(self):
super().__init__(
agent_id=AGENT_ID,
instructions="You are a helpful voice assistant.",
pipeline=pipeline,
)
if __name__ == "__main__":
# Pass the class itself (not an instance): serve() builds a fresh Assistant +
# pipeline per call, which is required for correct per-call state under concurrent calls.
zrt.serve(Assistant, on_ready=lambda: zrt.invoke(AGENT_ID, room=zrt.Room(playground=True)))
```
Realtime mode runs a single end-to-end speech-to-speech model (for example, OpenAI Realtime, Google Gemini Live, or AWS Nova Sonic). The Pipeline auto-detects it when you pass a realtime model as `llm`, so no separate STT or TTS stages are required.
**Architecture**
User audio streams to the realtime model, which transcribes, reasons, synthesizes, and streams audio back.
**When to use it**
* Latency is your top priority.
* You want the simplest setup with the fewest moving parts.
* You need fast, natural conversational flow.
* You don't need to swap individual STT or TTS providers.
**Example**
```python title="main.py" Python theme={null}
from zrt import Pipeline
from zrt.plugins import OpenAIRealtime, OpenAIRealtimeConfig
model = OpenAIRealtime(
config=OpenAIRealtimeConfig(model="gpt-4o-realtime-preview", voice="alloy", modalities=["audio", "text"])
)
# A realtime model goes in the LLM slot - the Pipeline auto-detects Realtime mode.
pipeline = Pipeline(llm=model)
```
Hybrid mode pairs a realtime model with an external STT or TTS. The Pipeline auto-detects the sub-mode from the extra component: `hybrid_stt` (external STT) or `hybrid_tts` (external TTS).
**Architecture**
* **Hybrid STT**: your STT produces a transcript, then both audio and transcript go to the realtime model. Use this when you need searchable text (for KB lookups).
* **Hybrid TTS**: the realtime model handles input and reasoning, and its text output is routed to your TTS for the final voice.
**When to use it**
| sub‑mode | Recommended use cases |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hybrid STT (`hybrid_stt`) | When you need a searchable transcript for knowledge‑base lookups or RAG, or when the realtime model lacks the language support or accuracy you require. |
| Hybrid TTS (`hybrid_tts`) | When you need a specific custom voice or advanced TTS features (for example, voice tuning or SSML) that the realtime model doesn't provide. |
**Example**
Provide an external STT and the Pipeline auto-detects `hybrid_stt`:
```python title="main.py" Python theme={null}
from zrt import Pipeline
from zrt.plugins import GeminiRealtime, GeminiLiveConfig, SarvamAISTT, SileroVAD
from zrt.inference import AICousticsDenoise
model = GeminiRealtime(
config=GeminiLiveConfig(
model="gemini-3.1-flash-live-preview",
voice="Puck",
response_modalities=["AUDIO"]
)
)
pipeline = Pipeline(
stt=SarvamAISTT(),
llm=model,
vad=SileroVAD(),
denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz")
)
```
Provide an external TTS and the Pipeline auto-detects `hybrid_tts`:
```python title="main.py" Python theme={null}
from zrt import Pipeline
from zrt.plugins import OpenAIRealtime, OpenAIRealtimeConfig, ElevenLabsTTS
from zrt.inference import AICousticsDenoise
model = OpenAIRealtime(
config=OpenAIRealtimeConfig(model="gpt-4o-realtime-preview", voice="alloy")
)
pipeline = Pipeline(
llm=model,
tts=ElevenLabsTTS(),
denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz")
)
```
## Other Pipeline Combinations
The `Pipeline` is not limited to the named modes. It accepts any subset of components and configures itself accordingly.
| Combination | Components | Use case |
| :---------- | :--------------- | :------------------------------------------------------------------------- |
| LLM + TTS | `llm` and `tts` | Text in, voice out. The agent receives text input and replies with speech. |
| STT + LLM | `stt` and `llm` | Voice in, text out. The user speaks and the agent replies with text. |
| Partial | Any other subset | Custom setups where you only need part of the conversational loop. |
```python title="main.py" Python theme={null}
# Text in, voice out
pipeline = Pipeline(llm=OpenAILLM(), tts=ElevenLabsTTS())
# Voice in, text out
pipeline = Pipeline(stt=DeepgramSTT(), llm=OpenAILLM())
```
## What's Next
Tap into the pipeline at runtime to inspect or transform each stage.
Observe metrics and traces at each pipeline stage.
Add automatic provider failover for any mode.
## References
#### Examples
Run STT, LLM and TTS as a cascade.
Run a speech-to-speech realtime model.
#### SDK Reference
`Pipeline` in the Python API reference.