Pipeline

The providers a call runs on: what hears, what thinks, what speaks. Two shapes are valid. A cascade names stt, llm and tts separately; a realtime pipeline names a speech-to-speech model in llm alone, because a realtime model is the language model. realtime is accepted as the older spelling of the same slot; naming both is rejected. A fallback slot — stt, llm, tts — also takes an array, in which case the head serves and the tail stands by. Wrap that array in FallbackSTT, FallbackLLM or FallbackTTS to say how it demotes as well as what it falls back to; each slot carries its own settings.

Options

Pipeline(options: PipelineOptions)

SlotValue

A video avatar to render the agent’s speech through.

ContextWindow | null

How a long conversation is kept inside the model’s limit.

SlotValue

Noise cancellation applied to the inbound audio.

DTMFHandler | null

Deliver keypad tones to the agent. null drops them.

EOUConfig | null

How patiently the agent waits before answering.

InterruptConfig | null

What counts as the caller barging in.

SlotValue

The language model — a text LLM for a cascade, a FallbackLLM chain of them, or a speech-to-speech model for a realtime pipeline.

PronunciationRule[]

PronunciationRule substitutions applied to generated text on its way to TTS.

SlotValue

The older spelling of a speech-to-speech llm.

SlotValue

Speech-to-text, or a FallbackSTT chain. Omit it on a realtime pipeline.

SlotValue

Text-to-speech, or a FallbackTTS chain. Omit it on a realtime pipeline.

SlotValue

End-of-turn detection, for deciding when they finished.

SlotValue

Voice activity detection — what notices the caller is talking.

VoiceMailDetector | null

Answering-machine detection, for outbound calls that may be picked up by one.

is_realtime

is_realtime(): void

mode

mode(): void

chain

chain(slot: string): void

Everything configured for one slot: the primary, then its fallbacks. Empty for a slot nothing was named for, and for a slot that is not one.

string

required

fallback_settings

fallback_settings(slot: string): _Fallback | null

How one slot demotes, or null where nothing was tuned. fallbacks is who a slot falls back to; this is how.

string

required

_Fallback | null

fallbacks

fallbacks(slot: string): void

What a slot falls back to, in the order they are tried.

string

required

hooks

hooks(event: string): function[]

The handlers registered for one event, in registration order.

string

required

function[]

on

on(event: PipelineHookEvent, callback: H): H

Register a hook that runs in this process, mid-turn. The events are the transcript hooks stt and llm, and the turn hooks user_turn_start, user_turn_end, agent_turn_start and agent_turn_end. An async generator function registered on llm is filed as llm_stream: it is handed the model’s output as a stream and yields what should be spoken, so it can buffer, rewrite or drop the turn. A plain async function on the same event sees the finished text instead. Per-component latency is a separate family, registered through its own namespace rather than by spelling the prefix here:

pipeline.metrics.on('stt', (data) =&gt; &#123; ... &#125;);

PipelineHookEvent

required

Which hook to attach to.

required

The handler.

primary

primary(slot: string): ProviderSpec | null

The provider a slot uses first, or null if it has none.

string

required

ProviderSpec | null

providers

providers(): Iterable<ProviderSpec>

Every provider in the pipeline, fallbacks included.

Iterable<ProviderSpec>

toString

toString(): string

string

NO_CHANGE

const NO_CHANGE: any

Annotated any on purpose: it lets tts: any = NO_CHANGE type-check while the honest annotation stays visible in the signature a reader sees.

PronunciationRule

Rewrite generated text on its way to TTS. Positional rather than an options object, because two strings in a fixed order read better than four words of keys: PronunciationRule('nginx', 'engine x').

Constructor

PronunciationRule(find: string, replace: string, case_sensitive: boolean)

string

required

string

required

boolean

default:"false"

ContextWindow

Keep a long conversation inside the model’s context, automatically.

Options

ContextWindow(options: ContextWindowOptions)

number

Recent user turns kept verbatim. The SDK’s default is 3, and it is worth keeping several — summarising the sentence the caller just said is how an agent starts answering the wrong question.

number | null

Or bound by item count. null for none.

number | null

Compress once the context exceeds this. null for no token ceiling.

number

The SDK’s default is 10.

ProviderSpec | null

The model that compresses. A second LLM, described here and built in the agent process like every other provider.

EOUConfig

When the caller is judged to have finished speaking.

Options

EOUConfig(options: EOUConfigOptions)

boolean | null

Treat “mhm”, “haan”, “right” as listening noises rather than turns. null takes the runtime’s default; the SDK’s own default is on for detectors that support it.

number

How sure the detector must be. Lower answers sooner and interrupts more.

number[]

[min, max] seconds to wait before treating silence as the end of a turn. [0.0, 0.0] answers as soon as the detector says the turn is complete, which is snappy but cuts off anyone who pauses to think.

string

DEFAULT waits a fixed time after speech stops. ADAPTIVE varies it with how certain the turn detector is — longer when the sentence sounds unfinished.

InterruptConfig

What counts as the caller interrupting, rather than just making a noise.

Options

InterruptConfig(options: InterruptConfigOptions)

number

How long to stay paused after an interruption that turned out to be nothing.

number

Seconds to fade the agent’s audio out over. An instant cut sounds like a dropped call.

number

STT confidence floor for those words.

number

Seconds of speech before it counts. Raising this is the usual fix for an agent that stops every time someone breathes.

number

Words required before it counts. 2 ignores a stray “yeah”.

string

VAD_ONLY reacts to any speech-like audio — fastest, and the most easily fooled by a cough or a door. STT_ONLY waits for words. HYBRID uses both.

boolean

Pick the sentence back up when the interruption proves false, rather than dropping it.

FallbackSTT

Speech-to-text providers to try in order, and how they demote.

Constructor

FallbackSTT(providers: , options: FallbackOptions)

default:"[]"

FallbackOptions

default:"{}"

SLOT

SLOT(): void

FallbackLLM

Language models to try in order, and how they demote.

Constructor

FallbackLLM(providers: , options: FallbackOptions)

default:"[]"

FallbackOptions

default:"{}"

SLOT

SLOT(): void

FallbackTTS

Text-to-speech providers to try in order, and how they demote.

Constructor

FallbackTTS(providers: , options: FallbackOptions)

default:"[]"

FallbackOptions

default:"{}"

SLOT

SLOT(): void

DTMFHandler

Deliver the caller’s keypad tones to the agent instead of dropping them.

Options

DTMFHandler(options: DTMFHandlerOptions)

DTMFCallback | null

What runs per keypress, taking (key) or (key, payload). Left null, the agent’s on_dtmf method is called instead — which is the usual shape, because a keypad menu almost always wants the agent’s own state to accumulate a multi-digit entry into.The runtime delivers one key per call either way; a PIN is accumulated on your side, not handed over whole.

VoiceMailDetector

Detect an answering machine on an outbound call. The detector runs in the agent process the ZeroRuntime starts: it buffers the opening speech for duration seconds and asks llm whether it is a person or a greeting.

Options

VoiceMailDetector(options: VoiceMailDetectorOptions)

VoiceMailCallback | null

What runs on detection. Left null, the agent’s on_voicemail method is called instead. Either is awaited, so anything said in it finishes before the call is ended.

string | null

Replace the built-in classification prompt.

number

Seconds of speech to buffer before deciding. The default is 2.0 — long enough for “Hi, you’ve reached…”.

boolean

Set false to configure it without turning it on.

ProviderSpec | null

The classifier. Required: without one there is nothing to classify with, and the detector is simply not installed.

PipelineMode

Which shape a pipeline resolved to. Derived from the slots that were filled rather than set by hand; read it off Pipeline.mode.

enum PipelineMode {
  FULL_CASCADING = 'full_cascading',
  HYBRID = 'hybrid',
  LLM_ONLY = 'llm_only',
  LLM_TTS_ONLY = 'llm_tts_only',
  PARTIAL_CASCADING = 'partial_cascading',
  REALTIME = 'realtime',
  STT_LLM_ONLY = 'stt_llm_only',
  STT_ONLY = 'stt_only',
  STT_TTS_ONLY = 'stt_tts_only',
  TTS_ONLY = 'tts_only',
}

​Pipeline

​Options

​is_realtime

​mode

​chain

​fallback_settings

​fallbacks

​hooks

​on

​primary

​providers

​toString

​NO_CHANGE

​PronunciationRule

​Constructor

​ContextWindow

​Options

​EOUConfig

​Options

​InterruptConfig

​Options

​FallbackSTT

​Constructor

​SLOT

​FallbackLLM

​Constructor

​SLOT

​FallbackTTS

​Constructor

​SLOT

​DTMFHandler

​Options

​VoiceMailDetector

​Options

​PipelineMode

Pipeline

Options

is_realtime

mode

chain

fallback_settings

fallbacks

hooks

on

primary

providers

toString

NO_CHANGE

PronunciationRule

Constructor

ContextWindow

Options

EOUConfig

Options

InterruptConfig

Options

FallbackSTT

Constructor

SLOT

FallbackLLM

Constructor

SLOT

FallbackTTS

Constructor

SLOT

DTMFHandler

Options

VoiceMailDetector

Options

PipelineMode