Speech & Audio

Speech is the default modality. The caller speaks, speech-to-text transcribes, the LLM replies, and text-to-speech voices the answer, or a single realtime model does it end to end. Beyond the core loop, this modality covers how the agent sounds and the audio it plays around the conversation.

Audio Customization

Choose the voice and tune speed, pitch, and emotion.

Background Audio

Play thinking sounds and ambient music during a call.

STT & TTS Plugins

Browse the speech-to-text and text-to-speech providers.

The voice loop

from zrt import Pipeline
from zrt.plugins import DeepgramSTT, OpenAILLM, CartesiaTTS, SileroVAD, TurnDetector
from zrt.inference import AICousticsDenoise

pipeline = Pipeline(
    stt=DeepgramSTT(),
    llm=OpenAILLM(),
    tts=CartesiaTTS(),
    vad=SileroVAD(),
    turn_detector=TurnDetector(model="echo-large"),
    denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
)

To swap voice providers, change the stt/tts plugins. To collapse the loop into one low-latency model, pass a realtime model as llm. See Pipeline Modes.

References

Python

Examples

Background Audio

Play ambient or hold audio during a call.

Enhanced Pronunciation

Tune how the agent pronounces words.

SDK Reference

Speech Generation

Speech Generation in the Python API reference.

TTS

TTS in the Python API reference.

Overview Audio Customization

Get started

Modalities

Configure Pipeline

Tools and Capabilities

Turn Detection and Interruptions

Telephony Agents

Context Management

Observability and Monitoring

Audio Customization

Background Audio

STT & TTS Plugins

The voice loop

References

Examples

Background Audio

Enhanced Pronunciation

SDK Reference

Speech Generation

TTS

Audio Customization

Background Audio

STT & TTS Plugins

​The voice loop

​References

​Examples

Background Audio

Enhanced Pronunciation

​SDK Reference

Speech Generation

TTS

The voice loop

References

Examples

SDK Reference