Pipeline

Create a Pipeline from the given component PipelineOptions.

Constructor

new Pipeline(opts: PipelineOptions)

stt

any

Speech-to-text provider. Defaults to null.

llm

any

Language-model provider. Defaults to null.

tts

any

Text-to-speech provider. Defaults to null.

vad

any

Voice-activity-detection provider. Defaults to null.

turnDetector

any

End-of-turn detector. Defaults to null.

avatar

any

Avatar/video provider. Defaults to null.

denoise

any

Audio denoiser. Defaults to null.

eouConfig

EOUConfig | null

End-of-utterance config. Defaults to EOUConfig().

interruptConfig

InterruptConfig | null

Interruption-handling config. Defaults to InterruptConfig().

conversationalGraph

any

Conversational-graph routing config. Defaults to null.

contextWindow

ContextWindow | any

Context-window trimming policy. Defaults to null.

voiceMailDetector

any

Voicemail detector. Defaults to null.

dtmfHandler

any

DTMF (touch-tone) handler. Defaults to null.

realtimeConfig

RealtimeConfig | null

Realtime speech-to-speech config. Defaults to null.

llmStreamHookEnabled

boolean

Enable the streaming LLM token-rewrite hook. Defaults to false.

llmStreamHookTimeoutMs

number

Per-token timeout for the LLM stream hook, in milliseconds. Defaults to 100.

playbackGraceMs

number

Grace period in milliseconds before considering playback finished. Defaults to 0.

sttFilterPatterns

string[] | null

Regex patterns to drop from transcripts. Defaults to DEFAULT_STT_FILTER_PATTERNS.

sttWordSubstitutions

Record<string, string> | null

Word replacements applied to transcripts ({ from: to }). Defaults to {}.

inactivityTimeoutSeconds

number | null

Seconds of inactivity before the session times out, or null to disable. Defaults to null.

wakeUp

number | null

Seconds of caller silence before the agent’s onWakeUp hook fires, or null to disable. Defaults to null.

wakeUpMaxAttempts

number | null

Maximum wake-up nudges before the call ends, or null for no cap. Defaults to null.

filterFillerWords

boolean

Drop a transcript that consists solely of a filler word (“okay”, “yeah”, …) without an LLM turn. Defaults to false (every transcript reaches the model).

fillerWords

string[] | null

Filler words used when PipelineOptions.filterFillerWords is on. When unset, the runtime supplies its own defaults.

changeComponent

changeComponent(opts: ChangeComponentOptions = {}): Promise<void>

Hot-swap components that already exist in the pipeline, in place. Invalid swaps warn and are skipped; passContext: false clears the LLM’s history (only after the runtime confirms the swap succeeded).

opts

ChangeComponentOptions

changePipeline

changePipeline(opts: ChangePipelineOptions = {}): Promise<void>

Replace the whole pipeline mid-session (live mode switch): pass the complete target pipeline; the mode is inferred from the llm/components you pass.

opts

ChangePipelineOptions

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

getLatestFrames

getLatestFrames(numFrames = 1): any[]

Return the most recent captured vision frames.

numFrames

Number of frames to return (most recent). Defaults to 1.

returns

any[]

getSessionMetricsSnapshot

getSessionMetricsSnapshot(): Record<string, any>

Snapshot of session metrics (empty in this SDK build).

returns

Record<string, any>

interrupt

interrupt(): void

Interrupt the bound agent’s current turn.

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, callback?: PipelineHookFn): any

Register a hook for event. Usable directly (on(event, fn)) or as a decorator factory (on(event)). Delegates to Pipeline.hooks.

event

string

required

callback

PipelineHookFn

returns

any

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

processText

processText(text: string): Promise<void>

Feed text into the bound agent’s session as model input.

text

string

required

pushAudioFrame

pushAudioFrame(frame: any): void

Fire audio_delta hooks for an incoming audio frame.

frame

any

required

pushVisionFrame

pushVisionFrame(frame: any): void

Buffer a vision frame (capped at VISION_FRAME_BUFFER_MAX) and fire vision_frame hooks.

frame

any

required

removeAllListeners

removeAllListeners(event?: string): void

event

string

replyWithContext

replyWithContext(instructions: string, extras: Record<string, any> = {}): Promise<void>

Generate a reply to instructions on the bound agent’s session.

instructions

string

required

extras

Record<string, any>

sendMessage

sendMessage(text: string, extras: Record<string, any> = {}): Promise<void>

Speak text on the bound agent’s session.

text

string

required

extras

Record<string, any>

setAgent

setAgent(agent: any): void

Bind the agent that runs on this pipeline.

agent

any

required

PipelineHooks

Registry of pipeline hooks. Accessed via Pipeline.hooks or Pipeline.on(...); lets you observe or transform audio, transcripts, LLM tokens, and lifecycle events.

Constructor

new PipelineHooks()

clearAllHooks

clearAllHooks(): void

Remove all registered hooks.

getHooks

getHooks(event: string): PipelineHookFn[]

All multi-handler callbacks registered for event.

event

string

required

returns

PipelineHookFn[]

hasLlmObservationHooks

hasLlmObservationHooks(): boolean

Whether any observe-only llm hooks are registered.

returns

boolean

hasLlmStreamHook

hasLlmStreamHook(): boolean

Whether an LLM stream hook is registered.

returns

boolean

hasSttStreamHook

hasSttStreamHook(): boolean

Whether an STT stream hook is registered.

returns

boolean

hasTtsStreamHook

hasTtsStreamHook(): boolean

Whether a TTS stream hook is registered.

returns

boolean

on

on(event: string, fn?: PipelineHookFn): PipelineHookFn | ((fn: PipelineHookFn) => PipelineHookFn)

event

string

required

PipelineHookFn

returns

PipelineHookFn | ((fn: PipelineHookFn) => PipelineHookFn)

registeredNames

registeredNames(): string[]

Names of all hooks that currently have at least one handler.

returns

string[]

PipelineHookError

Thrown when a registered pipeline hook is not supported.

Constructor

new PipelineHookError(hookName: string, reason: string, alternative: string | null)

hookName

string

required

reason

string

required

alternative

string | null

default:"null"

PipelineComponent

A processing component that can be present in a Pipeline.

enum PipelineComponent {
  STT = 'stt',
  LLM = 'llm',
  TTS = 'tts',
  VAD = 'vad',
  TURN_DETECTOR = 'turn_detector',
  AVATAR = 'avatar',
  DENOISE = 'denoise',
  REALTIME_MODEL = 'realtime_model',
}

PipelineConfig

Create a PipelineConfig.

Constructor

new PipelineConfig(opts: { /** Overall pipeline shape. */ pipelineMode: PipelineMode; /** Realtime wiring, or `null` for a non-realtime pipeline. */ realtimeMode: RealtimeMode | null; /** Whether a realtime model is in use. */ isRealtime: boolean; /** Components present in the pipeline. */ activeComponents: Iterable<PipelineComponent>; })

pipelineMode

PipelineMode

required

Overall pipeline shape.

realtimeMode

RealtimeMode | null

required

Realtime wiring, or null for a non-realtime pipeline.

isRealtime

boolean

required

Whether a realtime model is in use.

activeComponents

Iterable<PipelineComponent>

required

Components present in the pipeline.

hasComponent

hasComponent(component: PipelineComponent): boolean

Whether the given component is part of this pipeline.

component

PipelineComponent

required

returns

boolean

PipelineMode

The shape of the configured pipeline, derived from which components (PipelineComponent) are present. Determined automatically from the Pipeline you build; not normally set by hand.

enum PipelineMode {
  REALTIME = 'realtime',
  FULL_CASCADING = 'full_cascading',
  LLM_TTS_ONLY = 'llm_tts_only',
  STT_LLM_ONLY = 'stt_llm_only',
  LLM_ONLY = 'llm_only',
  STT_ONLY = 'stt_only',
  TTS_ONLY = 'tts_only',
  STT_TTS_ONLY = 'stt_tts_only',
  HYBRID = 'hybrid',
  PARTIAL_CASCADING = 'partial_cascading',
}

RealtimeMode

How a realtime speech-to-speech model is wired into the pipeline.

enum RealtimeMode {
  FULL_S2S = 'full_s2s',
  HYBRID_STT = 'hybrid_stt',
  HYBRID_TTS = 'hybrid_tts',
  LLM_ONLY = 'llm_only',
}

STT

Base class for speech-to-text providers. Provider plugins (e.g. DeepgramSTT({...})) extend this; subclass it only to integrate a custom STT.

Constructor

new STT()

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

Provider configuration sent to the Zero Runtime. Override to supply settings.

returns

Record<string, any>

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

onSttTranscript

onSttTranscript(callback: SttTranscriptCallback): void

callback

SttTranscriptCallback

required

processAudio

processAudio(_audioFrames: Uint8Array, _language?: string | null, ..._extra: any[]): Promise<void>

Process a buffer of audio frames. Override in a custom provider.

removeAllListeners

removeAllListeners(event?: string): void

event

string

streamTranscribe

streamTranscribe(_audioStream: AsyncIterable<Uint8Array>, ..._extra: any[]): AsyncIterator<STTResponse>

Transcribe a streaming audio source, yielding STTResponses. Override in a custom provider.

returns

AsyncIterator<STTResponse>

STTResponse

Create an STTResponse.

Constructor

new STTResponse(init: { eventType: SpeechEventType; data: SpeechData; metadata?: Record<string, any> | null; })

eventType

SpeechEventType

required

data

SpeechData

required

metadata

Record<string, any> | null

SpeechData

Create SpeechData.

Constructor

new SpeechData(init: { text: string; confidence?: number; language?: string | null; startTime?: number; endTime?: number; duration?: number; })

text

string

required

confidence

number

language

string | null

startTime

number

endTime

number

duration

number

LLM

Base class for language-model providers. Provider plugins (e.g. GoogleLLM({...})) extend this; subclass it only to integrate a custom LLM.

Constructor

new LLM()

cancelCurrentGeneration

cancelCurrentGeneration(): Promise<void>

Cancel the in-flight generation for the current session, if any.

chat

chat(_messages: any, _opts?: LLMChatOptions): AsyncIterator<LLMResponse>

Stream a completion for the given messages, yielding LLMResponse chunks.

returns

AsyncIterator<LLMResponse>

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

Provider configuration sent to the Zero Runtime. Override to supply settings.

returns

Record<string, any>

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

removeAllListeners

removeAllListeners(event?: string): void

event

string

LLMResponse

Create an LLMResponse.

Constructor

new LLMResponse(init: { content: string; role: ChatRole; metadata?: Record<string, any> | null; })

content

string

required

role

ChatRole

required

metadata

Record<string, any> | null

TTS

Base class for text-to-speech providers. Provider plugins (e.g. CartesiaTTS({...})) extend this; subclass it only to integrate a custom TTS.

Constructor

new TTS(opts: TTSOptions)

sampleRate

number

Output sample rate in Hz. Defaults to 24000.

numChannels

number

Number of output channels. Defaults to 1.

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

Provider configuration sent to the Zero Runtime. Override to supply settings.

returns

Record<string, any>

interrupt

interrupt(): Promise<void>

Stop the current synthesis.

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

onFirstAudioByte

onFirstAudioByte(callback: FirstAudioByteCallback): void

callback

FirstAudioByteCallback

required

removeAllListeners

removeAllListeners(event?: string): void

event

string

synthesize

synthesize(text: any, ..._extra: any[]): Promise<void>

Synthesize and speak text on the active session.

text

any

required

VAD

Base class for voice-activity-detection providers. Provider plugins extend this; subclass it only to integrate a custom VAD.

Constructor

new VAD(opts: VADOptions)

sampleRate

number

Input sample rate in Hz. Defaults to 16000.

threshold

number

Speech-probability threshold 0-1 above which audio counts as speech. Defaults to 0.5.

minSpeechDuration

number

Minimum speech duration in seconds to register a start-of-speech. Defaults to 0.5.

minSilenceDuration

number

Minimum silence duration in seconds to register an end-of-speech. Defaults to 0.5.

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

flush

flush(): Promise<void>

Flush any buffered audio.

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

Provider configuration (threshold and durations) sent to the Zero Runtime.

returns

Record<string, any>

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

onVadEvent

onVadEvent(callback: VadEventCallback): void

callback

VadEventCallback

required

processAudio

processAudio(_audioFrames: Uint8Array, ..._extra: any[]): Promise<void>

Process a buffer of audio frames. Override in a custom provider.

removeAllListeners

removeAllListeners(event?: string): void

event

string

VADResponse

Create a VADResponse.

Constructor

new VADResponse(init: { eventType: VADEventType; data: VADData; metadata?: Record<string, any> | null; })

eventType

VADEventType

required

data

VADData

required

metadata

Record<string, any> | null

VADData

Create VADData.

Constructor

new VADData(init: { isSpeech: boolean; confidence?: number; timestamp?: number; speechDuration?: number; silenceDuration?: number; })

isSpeech

boolean

required

confidence

number

timestamp

number

speechDuration

number

silenceDuration

number

EOU

Base class for end-of-utterance (turn-detection) providers. Provider plugins extend this; subclass it only to integrate a custom turn detector.

Constructor

new EOU(opts: EOUOptions)

threshold

number

End-of-utterance probability threshold 0-1. Defaults to 0.7.

cleanup

cleanup(): Promise<void>

Tear down provider state.

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

detectEndOfUtterance

detectEndOfUtterance(chatContext?: any, threshold?: number): boolean

Whether the current EOU probability meets threshold (or the configured one).

chatContext

any

threshold

number

returns

boolean

emit

emit(event: string, ...args: any[]): void

event

string

required

getEouProbability

getEouProbability(_chatContext?: any): number

Latest end-of-utterance probability 0-1 (defaults to 0.5 when unknown).

returns

number

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

Provider configuration (threshold) sent to the Zero Runtime.

returns

Record<string, any>

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

removeAllListeners

removeAllListeners(event?: string): void

event

string

setThreshold

setThreshold(threshold: number): void

Update the end-of-utterance threshold.

threshold

number

required

FallbackSTT

Create an STT provider that tries each provider in order, failing over to the next when one is unavailable.

Constructor

new FallbackSTT(providers: STT[], opts: FallbackOptions)

providers

STT[]

required

Providers in priority order; the first is the primary.

temporaryDisableSec

number

Seconds to skip a failing provider before retrying. Defaults to DEFAULT_TEMPORARY_DISABLE_SEC.

permanentDisableAfterAttempts

number

Failed attempts after which a provider is disabled for the session. Defaults to DEFAULT_PERMANENT_DISABLE_AFTER_ATTEMPTS.

recoveryCheckIntervalSec

number

Seconds between health checks of a disabled provider. Defaults to DEFAULT_RECOVERY_CHECK_INTERVAL_SEC.

latencyThresholdMs

number | null

Latency ceiling in ms for the active provider. When exceeded for consecutiveLatencyHits turns in a row, the chain fails over. Omitted/0 = disabled.

consecutiveLatencyHits

number

Consecutive turns over latencyThresholdMs before failing over. Defaults to 3.

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

returns

Record<string, any>

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

onSttTranscript

onSttTranscript(callback: SttTranscriptCallback): void

callback

SttTranscriptCallback

required

processAudio

processAudio(_audioFrames: Uint8Array, _language?: string | null, ..._extra: any[]): Promise<void>

Process a buffer of audio frames. Override in a custom provider.

removeAllListeners

removeAllListeners(event?: string): void

event

string

streamTranscribe

streamTranscribe(_audioStream: AsyncIterable<Uint8Array>, ..._extra: any[]): AsyncIterator<STTResponse>

Transcribe a streaming audio source, yielding STTResponses. Override in a custom provider.

returns

AsyncIterator<STTResponse>

FallbackLLM

Create an LLM provider that tries each provider in order, failing over to the next when one is unavailable. The primary’s temperature and max_output_tokens settings carry through.

Constructor

new FallbackLLM(providers: LLM[], opts: FallbackOptions)

providers

LLM[]

required

Providers in priority order; the first is the primary.

temporaryDisableSec

number

Seconds to skip a failing provider before retrying. Defaults to DEFAULT_TEMPORARY_DISABLE_SEC.

permanentDisableAfterAttempts

number

Failed attempts after which a provider is disabled for the session. Defaults to DEFAULT_PERMANENT_DISABLE_AFTER_ATTEMPTS.

recoveryCheckIntervalSec

number

Seconds between health checks of a disabled provider. Defaults to DEFAULT_RECOVERY_CHECK_INTERVAL_SEC.

latencyThresholdMs

number | null

Latency ceiling in ms for the active provider. When exceeded for consecutiveLatencyHits turns in a row, the chain fails over. Omitted/0 = disabled.

consecutiveLatencyHits

number

Consecutive turns over latencyThresholdMs before failing over. Defaults to 3.

cancelCurrentGeneration

cancelCurrentGeneration(): Promise<void>

Cancel the in-flight generation for the current session, if any.

chat

chat(_messages: any, _opts?: LLMChatOptions): AsyncIterator<LLMResponse>

Stream a completion for the given messages, yielding LLMResponse chunks.

returns

AsyncIterator<LLMResponse>

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

returns

Record<string, any>

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

removeAllListeners

removeAllListeners(event?: string): void

event

string

FallbackTTS

Create a TTS provider that tries each provider in order, failing over to the next when one is unavailable.

Constructor

new FallbackTTS(providers: TTS[], opts: FallbackOptions)

providers

TTS[]

required

Providers in priority order; the first is the primary.

temporaryDisableSec

number

Seconds to skip a failing provider before retrying. Defaults to DEFAULT_TEMPORARY_DISABLE_SEC.

permanentDisableAfterAttempts

number

Failed attempts after which a provider is disabled for the session. Defaults to DEFAULT_PERMANENT_DISABLE_AFTER_ATTEMPTS.

recoveryCheckIntervalSec

number

Seconds between health checks of a disabled provider. Defaults to DEFAULT_RECOVERY_CHECK_INTERVAL_SEC.

latencyThresholdMs

number | null

Latency ceiling in ms for the active provider. When exceeded for consecutiveLatencyHits turns in a row, the chain fails over. Omitted/0 = disabled.

consecutiveLatencyHits

number

Consecutive turns over latencyThresholdMs before failing over. Defaults to 3.

close

close(): Promise<void>

Release any resources held by the provider.

closeEmitter

closeEmitter(): Promise<void>

Close the emitter: drop all handlers and stop emitting. Waits up to 2 seconds for in-flight async handlers to settle before returning.

emit

emit(event: string, ...args: any[]): void

event

string

required

getRuntimeConfig

getRuntimeConfig(): Record<string, any>

returns

Record<string, any>

interrupt

interrupt(): Promise<void>

Stop the current synthesis.

listenerCount

listenerCount(event: string): number

event

string

required

returns

number

off

off(event: string, handler: EventHandler): void

event

string

required

handler

EventHandler

required

on

on(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

once

once(event: string, handler: EventHandler): EventHandler

event

string

required

handler

EventHandler

required

returns

EventHandler

onFirstAudioByte

onFirstAudioByte(callback: FirstAudioByteCallback): void

callback

FirstAudioByteCallback

required

removeAllListeners

removeAllListeners(event?: string): void

event

string

synthesize

synthesize(text: any, ..._extra: any[]): Promise<void>

Synthesize and speak text on the active session.

text

any

required

UtteranceHandle

See the class-level description for usage.

Constructor

new UtteranceHandle(opts: UtteranceHandleOptions)

utteranceId

string

Identifier for the utterance. Default: a generated UUID.

interruptible

boolean

Whether the utterance can be interrupted without force. Default: true.

done

done(): boolean

Whether the utterance has finished (completed or interrupted).

returns

boolean

interrupt

interrupt(opts: { force?: boolean } = {}): void

Interrupt the utterance, marking it done and firing last-audio-byte callbacks. Has no effect on a non-interruptible utterance unless force is true.

opts

{ force?: boolean }

markDone

markDone(): void

Mark the utterance as completed normally, firing last-audio-byte callbacks.

onLastAudioByte

onLastAudioByte(cb: () => any): () => any

Register a callback that fires when the utterance’s final audio byte has played. If the utterance is already done, the callback runs immediately; otherwise it is queued and fires once when the utterance finishes. Returns the same callback you passed in. Errors thrown by the callback are caught and logged, never propagated.

() => any

required

returns

() => any

then

then(onfulfilled?: ((value: void) => TResult1 | PromiseLike<TResult1>) | null, onrejected?: ((reason: any) => TResult2 | PromiseLike<TResult2>) | null): Promise<TResult1 | TResult2>

Makes the handle awaitable; resolves when the utterance finishes.

onfulfilled

((value: void) => TResult1 | PromiseLike<TResult1>) | null

onrejected

((reason: any) => TResult2 | PromiseLike<TResult2>) | null

returns

Promise<TResult1 | TResult2>

wait

wait(): Promise<void>

Returns a promise that resolves when the utterance finishes.

Agent & Session Events & Observability

​Pipeline

​Constructor

​changeComponent

​changePipeline

​closeEmitter

​emit

​getLatestFrames

​getSessionMetricsSnapshot

​interrupt

​listenerCount

​off

​on

​once

​processText

​pushAudioFrame

​pushVisionFrame

​removeAllListeners

​replyWithContext

​sendMessage

​setAgent

​PipelineHooks

​Constructor

​clearAllHooks

​getHooks

​hasLlmObservationHooks

​hasLlmStreamHook

​hasSttStreamHook

​hasTtsStreamHook

​on

​registeredNames

​PipelineHookError

​Constructor

​PipelineComponent

​PipelineConfig

​Constructor

​hasComponent

​PipelineMode

​RealtimeMode

​STT

​Constructor

​close

​closeEmitter

​emit

​getRuntimeConfig

​listenerCount

​off

​on

​once

​onSttTranscript

​processAudio

​removeAllListeners

​streamTranscribe

​STTResponse

​Constructor

​SpeechData

​Constructor

​LLM

​Constructor

​cancelCurrentGeneration

​chat

​close

​closeEmitter

​emit

​getRuntimeConfig

​listenerCount

​off

​on

​once

​removeAllListeners

​LLMResponse

​Constructor

​TTS

​Constructor

​close

​closeEmitter

​emit

​getRuntimeConfig

​interrupt

​listenerCount

​off

Pipeline

Constructor

changeComponent

changePipeline

closeEmitter

emit

getLatestFrames

getSessionMetricsSnapshot

interrupt

listenerCount

off

on

once

processText

pushAudioFrame

pushVisionFrame

removeAllListeners

replyWithContext

sendMessage

setAgent

PipelineHooks

Constructor

clearAllHooks

getHooks

hasLlmObservationHooks

hasLlmStreamHook

hasSttStreamHook

hasTtsStreamHook

on

registeredNames

PipelineHookError

Constructor

PipelineComponent

PipelineConfig

Constructor

hasComponent

PipelineMode

RealtimeMode

STT

Constructor

close

closeEmitter

emit

getRuntimeConfig

listenerCount

off

on

once

onSttTranscript

processAudio

removeAllListeners

streamTranscribe

STTResponse

Constructor

SpeechData

Constructor

LLM

Constructor

cancelCurrentGeneration

chat

close

closeEmitter

emit

getRuntimeConfig

listenerCount

off

on

once

removeAllListeners

LLMResponse

Constructor

TTS

Constructor

close

closeEmitter

emit

getRuntimeConfig

interrupt

listenerCount

off