Agent & Session - Zero Runtime

Agent

What the agent is, what it can do, and how it behaves on a call. Subclass it to give the agent behaviour: fields holding function_tool callables are registered automatically, and the on_* hooks below are called by the session as the call progresses. An instance is inert until serve registers it and a session is started against it. The pipeline is the other half. This class carries the prompt, the tools and the call-shaped options; Pipeline carries the providers that hear and speak.

Options

Agent(options: AgentOptions)

string

required

Required. The name serve registers under and the runtime routes sessions to.

Agent[]

Other agents this one hands off to, kept on alternates. Nothing registers them for you: serve registers one agent_id per call, and Session.add_handoff names the target by that id.

CallSummary | null

Summarise the conversation at teardown and POST it somewhere.

string | null

Spoken on the way out, when the agent ends the call itself.

string | null

Spoken as soon as the agent joins, before the caller says anything.

boolean

Carry the chat history over when another agent hands off to this one. Off means it starts the conversation fresh.

string

required

The system prompt. Persona, task, and the rules the model is expected to hold to.

number | null

Hard ceiling on the call. The runtime ends the session when it is reached.

MCPServer[]

MCPServerStdio and MCPServerHTTP whose tools join the agent’s own. Connected on the first call to initialize_mcp, not here.

string | null

Display name. Defaults to agent_id.

Pipeline | null

The Pipeline this agent runs on.

number | null

How long the runtime waits for one of this agent’s tools to return before handing the model a timeout. null keeps the runtime’s own default of 30 seconds.Raise it only for an agent whose tools genuinely run long. The case it exists for is a tool that awaits Session.warm_transfer — that runs for supervisor_join_timeout + briefing_timeout, minutes rather than seconds, and the default would kill the round trip while the transfer carried on underneath. The cost is symmetric: a tool that hangs holds the turn for exactly this long.

FunctionTool<any, any>[]

Tools defined elsewhere. Tools declared as fields on the agent are found on their own, so this is for the ones that are not.

number

Seconds of caller silence before the agent nudges. 0 disables it; negative is rejected.

string

What the nudge says.

agent_id

agent_id(): void

instructions

instructions(): void

session

session(): void

tools

tools(): void

add_server

add_server(mcp_server: MCPServer): Promise<void>

Connect one MCP server and append what it publishes to tools. Ignores anything that is not an MCPServerStdio or MCPServerHTTP.

MCPServer

required

The server to connect.

Promise<void>

cleanup

cleanup(): Promise<void>

Close the MCP connections and drop the tools they contributed. Called for you at teardown. Safe to call twice, and safe with no MCP servers configured.

Promise<void>

emit

emit(event: 'agent_started', args: unknown[]): void

Call every handler subscribed to event. Returns as soon as the handlers have been started, not when they finish, and never throws: an exception inside a handler is logged against the event name. Ignored once the emitter has been closed.

'agent_started'

required

The event name.

unknown[]

required

Passed through to each handler.

hangup

hangup(reason: string, farewell: string): Promise<void>

End the call from the agent’s side. A no-op with no live session.

string

default:"'agent ended the call'"

Recorded against the session.

string

default:"''"

Spoken before hanging up. Empty uses the agent’s own farewell.

Promise<void>

initialize_mcp

initialize_mcp(): Promise<void>

Connect every configured MCP server and adopt its tools. Runs once; later calls return immediately. serve calls it while building the agent, so a server that is down fails the build rather than the first tool call.

Promise<void>

off

off(event: 'agent_started', callback: EventHandler): void

Unsubscribe one handler. Matched by identity, so it has to be the same object that was registered — a fresh arrow function will not match. Removing something that was never subscribed does nothing.

'agent_started'

required

EventHandler

required

on

on(event: 'agent_started', callback: EventHandler): EventHandler

Subscribe to an event. Handlers fire in the order they were registered, and registering the same one twice calls it twice.

'agent_started'

required

The event name.

EventHandler

required

The handler.

EventHandler

on_enter

on_enter(): Promise<void>

Called once the agent has joined and the session is live. Override it to open the conversation, prime the context, or set up standing configuration such as Session.set_thinking_audio.

Promise<void>

on_exit

on_exit(): Promise<void>

Called as the call is ending, before teardown finishes.

Promise<void>

on_participant_joined

on_participant_joined(participant: Participant): Promise<void>

Called when somebody joins the room the agent is in.

Participant

required

Promise<void>

on_participant_left

on_participant_left(participant: Participant): Promise<void>

Called when somebody leaves. The agent is not one of them.

Participant

required

Promise<void>

register_tools

register_tools(): void

Collect the function_tool callables declared on this agent. Called for you the first time tools is read. Call it again after adding one at runtime; it never registers the same tool twice.

toString

toString(): string

Returns a string representation of an object.

string

update_tools

update_tools(tools: FunctionTool<any, any>[]): void

Replace the tool list outright. This drops the tools that were found on the agent and anything MCP added, so pass the full set you want.

FunctionTool<any, any>[]

required

AgentContext

What the runtime knows about a job, handed to the agent factory. serve passes one to your factory if it takes an argument, so the agent can be built around the call it is about to take — a caller’s number out of metadata, a different prompt per room.

Options

AgentContext(options: AgentContextOptions)

string

Which registration the job was routed to.

string

The runtime’s id for this dispatch.

Record<string, any>

Whatever the dispatcher attached — the caller’s number, a tenant id, anything the job was started with.

string

The room the agent is joining.

Room

The Room the runtime resolved for this job.

string

The auth token for this job.

toString

toString(): string

string

CallSummary

Summarise the conversation when it ends, and POST the result.

Options

CallSummary(options: CallSummaryOptions)

boolean

Set false to carry the configuration without summarising.

string | null

Where the summary is POSTed. Without one it is generated and logged but sent nowhere.

Record<string, string>

Sent with the POST — an auth header, usually.

string | null

Replaces the built-in summarising prompt.

ProviderSpec | null

A second model for the summary alone. null reuses the session’s. Worth pointing at something cheap: it runs once, over the whole transcript, while the process is already shutting down.

number

Seconds the summariser gets. 0 takes the runtime’s default. Summarising is an LLM round trip over the whole conversation, so it is a different order of work from the rest of teardown. The ceiling is the runtime’s SIGTERM grace, since this runs while the agent is shutting down — asking for more than that gets the process killed mid-request rather than granting the time.

serve

serve(agent: AgentFactory, options: ServeOptions): Promise<void>

Register agent and serve every call dispatched to it. Every option left here describes this worker — how many calls it takes, what it logs, where it answers probes. Nothing here describes the session: that is Room’s job, and the twelve options which used to say it a second time are gone, because two spellings of recording that could disagree is a question with no right answer. The returned promise settles when the worker is interrupted or shut down. The agent is registered with the runtime’s registry, and every call dispatched to that id builds a fresh agent from the factory — one instance per call, never one shared across them.

AgentFactory

required

A callable returning an Agent. Your subclass is usually the callable. An instance is rejected: concurrent calls would share one conversation.

ServeOptions

default:"{}"

Promise<void>

invoke

invoke(agent_id: string, options: InvokeOptions): Promise<Record<string, string>>

Start a session against an agent this process is already serving. Resolves once the runtime has accepted the session, which is the point at which the room is joinable. An ordinary async function: await it from on_ready, or from anywhere else once serve() is running. The agent has to be registered in this process. There is no dispatch RPC to route a start to another one: the pipeline, credentials and tool schemas all live where serve was called.

string

required

The id passed to serve.

InvokeOptions

default:"{}"

Promise<Record<string, string>>

Session

One live call, and everything you can do to it while it runs. You are handed one rather than constructing one: invoke returns it, and inside a function_tool or an on_* hook it is the session the agent is bound to. Every method here is a message to the agent process, so they are all awaited and none of them block the call. The object stays usable after the call ends — ended goes true and the send methods become no-ops — so a hook can read ended_reason without guarding first.

Constructor

Session(stream: any, accepted: ProtoMessage, outbound: Outbound, options: object)

any

required

ProtoMessage

required

Outbound

required

object

default:"{}"

agent

agent(): void

ended

ended(): void

playground_url

playground_url(): void

[asyncDispose]

[asyncDispose](): Promise<void>

Promise<void>

aclose

aclose(): Promise<void>

Release everything this call holds. Idempotent, and safe after any ending. Every way a call can finish routes here. Closing the transport is the part that cannot be skipped: the outbound queue feeds a writer this SDK holds, and a live writer keeps the stream — and the process — alive.

Promise<void>

add_handoff

add_handoff(to_agent: string, __namedParameters: object): Promise<void>

Hand the conversation to another agent, by id. The target has to be registered — serve registers one agent_id per call — and whether it inherits the history is that agent’s inherit_context.

string

required

The agent_id taking over.

object

default:"{}"

Promise<void>

add_message

add_message(role: string, content: string, __namedParameters: object): Promise<void>

Write a message into the conversation without speaking it. Useful for handing the model something it should know — a CRM lookup, a verification result — without a turn being spent saying it out loud.

string

required

system, developer, user or assistant. A ChatRole works too.

string

required

The message body.

object

default:"{}"

Promise<void>

change_component

change_component(options: ChangeComponentOptions): Promise<string>

Swap the named components and leave the rest of the call alone.

ChangeComponentOptions

default:"{}"

Promise<string>

change_pipeline

change_pipeline(pipeline: Pipeline, __namedParameters: object): Promise<string>

Replace the whole pipeline mid-call. Everything not named is rebuilt from the pipeline you pass, so state a complete one. To change a provider or two and leave the rest alone, use change_component.

Pipeline

required

The Pipeline to switch to.

object

default:"{}"

Promise<string>

destroy

destroy(reason: string): Promise<void>

Tear the session down at the runtime, without a goodbye. The blunt one: no farewell, no teardown hooks worth waiting for. end is the polite version.

string

default:"'client requested destroy'"

Promise<void>

detach

detach(): Promise<void>

Stop watching this session without ending the call. The agent keeps running runtime-side; this process just stops listening. For a call that should actually stop, use end or destroy.

Promise<void>

emit

emit(event: string, args: unknown[]): void

string

required

The event name.

unknown[]

required

Passed through to each handler.

end

end(reason: string, farewell: string): Promise<void>

Ask the runtime to end the call. A no-op once it has ended.

string

default:"'client ended the session'"

Recorded against the session.

string

default:"''"

Spoken before hanging up.

Promise<void>

events

events(): AsyncGenerator<Event>

Iterate everything the runtime reports, until the call ends. Keeps yielding briefly past the ended event so the last transcripts and metrics are not cut off mid-teardown.

AsyncGenerator<Event>

get_context_history

get_context_history(__namedParameters: object): Promise<any[]>

Fetch the conversation as the agent process holds it.

object

default:"{}"

Promise<any[]>

get_metrics

get_metrics(__namedParameters: object): Promise<any[]>

Fetch the per-turn latency metrics collected so far.

object

default:"{}"

Promise<any[]>

get_participants

get_participants(__namedParameters: object): Promise<any[]>

Who is in the room right now, as Participant records.

object

default:"{}"

Promise<any[]>

interrupt

interrupt(__namedParameters: object): Promise<void>

Stop the agent talking.

object

default:"{}"

Promise<void>

log_playground

log_playground(): void

Announce the join URL on stdout, at most once per session. Both the direct-start path and the registry worker path call this, and invoke adds a third; the guard is what keeps one session to one URL. It prints rather than logs because the URL is the one line the operator is waiting to copy, and a log record buries it behind level and module columns that are noise for a value you paste into a browser.

off

off(event: string, callback: EventHandler): void

Unsubscribe one handler. Matched by identity, so it has to be the same object that was registered — a fresh arrow function will not match. Removing something that was never subscribed does nothing.

string

required

EventHandler

required

on

on(event: string, callback: EventHandler): EventHandler

Subscribe to an event. Handlers fire in the order they were registered, and registering the same one twice calls it twice.

string

required

The event name.

EventHandler

required

The handler.

EventHandler

play_background_audio

play_background_audio(file: string | object | null, __namedParameters: object): Promise<void>

Start a sound bed under the call. Needs the room’s mixing track, Room({ background_audio: true }).

string | object | null

default:"null"

A path, or a BackgroundAudio carrying one along with its volume and looping. A disabled BackgroundAudio plays nothing.

object

default:"{}"

Promise<void>

process_text

process_text(text: string): Promise<void>

Feed text to the agent as though the caller had said it. The full turn runs: the model answers and the answer is spoken.

string

required

Promise<void>

publish_to_pubsub

publish_to_pubsub(pubsub_config: PubSubPublishConfig): Promise<void>

Publish one pubsub frame into the room.

PubSubPublishConfig

required

The topic, body and options to publish with.

Promise<void>

reply

reply(instructions: string, options: ReplyOptions): Promise<UtteranceHandle>

Ask the model for a line, generated against instructions. Unlike say, this is a generation: the model sees the conversation so far plus these instructions, and what it produces is spoken.

string

required

What to tell the model for this turn alone.

ReplyOptions

default:"{}"

Promise<UtteranceHandle>

say

say(text: string, options: SayOptions): Promise<UtteranceHandle>

Speak an exact line. No model involved.

string

required

What to say.

SayOptions

default:"{}"

Promise<UtteranceHandle>

set_thinking_audio

set_thinking_audio(file: string | null, __namedParameters: object): Promise<void>

Play a sound while the agent is generating its reply. Standing configuration rather than a one-shot: once set, the session starts it at the top of every generation and stops it when the answer begins, for as long as the session lasts. Set it from on_enter. file omitted takes the runtime’s own thinking sound. Any file libav can decode works — wav, mp3, ogg, flac, m4a. Needs the room’s mixing track, Room({ background_audio: true }), like all audio that is not speech.

string | null

default:"null"

object

default:"{}"

Promise<void>

stop

stop(reason: string): Promise<void>

End the call and release it. What await using calls.

string

default:"'client requested stop'"

Promise<void>

stop_background_audio

stop_background_audio(): Promise<void>

Stop the sound bed. Leaves the thinking sound alone.

Promise<void>

subscribe_to_pubsub

subscribe_to_pubsub(pubsub_config: PubSubSubscribeConfig): Promise<void>

Listen on one pubsub topic for the rest of the call.

PubSubSubscribeConfig

required

The topic and its handler.

Promise<void>

toString

toString(): string

Returns a string representation of an object.

string

transfer_call

transfer_call(transfer_to: string, __namedParameters: object): Promise<Record<string, any>>

Cold-transfer the call and leave. The caller is handed to transfer_to with no introduction and the agent drops out.

string

required

Where to send them — a SIP URI or a phone number.

object

default:"{}"

Promise<Record<string, any>>

wait

wait(__namedParameters: object): Promise<void>

Consume events until the call ends, logging as it goes. The usual last line of a script that started a session and has nothing else to do.

object

default:"{}"

Promise<void>

current_session

current_session(): Session

The session whose callback is currently running. Valid anywhere the SDK invokes your code on a session’s behalf: a function_tool, a lifecycle hook, a pipeline hook. Useful for code that runs on the call without holding a reference to it — a module-level pipeline.on handler, say. Anything defined on the agent should reach the call through this.session instead, and a pubsub handler subscribed with Session.subscribe_to_pubsub already has the session in hand.

Session

Participant

Constructor

Participant(options: object)

object

required

PubSubSubscribeConfig

One topic to listen on, and what to call for each frame. Passed to Session.subscribe_to_pubsub and Room.subscribe_to_pubsub.

Options

PubSubSubscribeConfig(options: PubSubSubscribeConfigOptions)

PubSubHandler | null

Called once per frame with the raw frame as an object — the whole thing as the transport delivered it, since pubsub promises no schema beyond message. Sync or async; a promise is awaited.A handler that declares a second parameter is additionally told whether the frame is backlog:

function cb(frame, backlog) &#123;
  if (backlog) return;          // already in the topic before we joined
&#125;

Subscribing replays whatever the topic already held, so a one-argument handler sees that history as ordinary traffic — which is the transport’s own shape, and worth knowing about before replying to it.

string

required

The topic to subscribe to.

PubSubPublishConfig

One frame to publish into the room. Passed to Session.publish_to_pubsub and Room.publish_to_pubsub.

Options

PubSubPublishConfig(options: PubSubPublishConfigOptions)

string

The text body.

Record<string, any>

Transport options. Only sendOnly — a list of participant ids to deliver to — is carried; anything else is dropped, because the wire has nowhere to put it. Omitted, everyone subscribed to the topic gets the frame.

Record<string, any> | null

Structured data alongside it, JSON-encoded on the way out.

string

required

The topic to publish on.

UtteranceHandle

One thing the agent is saying, and how it went. Returned by Session.say and Session.reply. Call wait() to block until the line finishes — it resolves whether the line played out or was cut off, so check interrupted rather than assuming it was heard.

const handle = await session.say('One moment while I look that up.');
await handle.wait();
if (handle.interrupted) &#123;
  // ...
&#125;

The wait is a method rather than the handle itself being awaitable: a thenable would be unwrapped by await session.say(...), so that line would not return until the audio had finished playing. As it is, say() hands the handle back the moment the request is on its way. Waiting is optional. Ignore the handle and the line still plays.

Constructor

UtteranceHandle(utterance_id: string)

string

required

interrupted

interrupted(): void

state

state(): void

done

done(): boolean

Whether the utterance has settled, either way. Never blocks.

boolean

toString

toString(): string

string

wait

wait(): Promise<UtteranceHandle>

Resolve once the line has settled, either way. Resolves to this handle.

Promise<UtteranceHandle>

Room

Which room the agent joins. room_id: null asks for a new one.

Options

Room(options: RoomOptions)

string | null

The 0.1.2 spelling of name. Folded into it when set.

string | null

Publish the agent under a fixed participant id. null lets the room mint one, which is what a single-agent session wants.

string | null

Codec the room negotiates — "opus" (the default when null), "pcmu", "pcma", "g722". This is what tells a meeting from a phone call: opus is 48 kHz, pcmu/pcma are 8 kHz and g722 is 16 kHz, and that rate reaches the VAD and STT. Leave it null for meetings; set "pcmu" or "pcma" for SIP so the pipeline sizes itself to the narrowband audio instead of resampling it up to 48 kHz and back down again.

string | null

VideoSDK token for the room. Falls back to the environment. It travels in StartSession.credentials, never in params_json.

boolean | null

End the session when the last participant leaves. null leaves the runtime’s default (TRUE) in place.

boolean

Open the room’s mixing audio track. Nothing plays by itself — this is the track, not the sound. Start audio on it with session.play_background_audio({ file, volume, looping }) and stop it with session.stop_background_audio().Required before either call: without the track the SDK refuses, and the failure is a silent one from the caller’s side.

number | null

Seconds of silence before the session is considered idle.

boolean | null

Whether the agent joins the meeting. null leaves the runtime’s default (TRUE) in place.

string

Display name the agent publishes under.

number | null

Seconds with nobody attached before the session is reclaimed. Carried in SessionLimits.inactivity_timeout_seconds, same as above.

Observability | null

Where this session’s telemetry goes. null takes the runtime’s defaults.

boolean

Ask the runtime for a playground URL for this session.

boolean

Record the session. Off unless asked for — recording a call is a decision about the person on the other end, so the default has to be the safe one and the same runtime serves recorded and unrecorded sessions side by side. Audio is always captured when this is on; the two below add tracks to it.

Also record the screen share. Needs recording and vision — there is no screen-share track on a session that is not receiving video.

boolean

Also record camera video. Needs recording.

string | null

Existing room to join. A new room is created when null.

number | null

Hard cap on session length, in seconds. Carried in SessionLimits.max_session_duration_seconds rather than in RoomSpec, so it is read by the transport and not by to_proto. null inherits the runtime’s own ceiling.

string | null

Override the signaling base URL.

boolean

Deliver camera frames to the agent process.

boolean | null

Whether the agent waits for someone to join before speaking. null leaves the runtime’s default (TRUE) in place.

explicitly_set

explicitly_set(): void

Sip

The telephony leg of a session — who is being called, and from what.

Options

Sip(options: SipOptions)

string | null

Caller id presented on the call.

string | null

Identifier correlating the call.

string | null

Destination number/address for an outbound call.

string | null

Call direction/type.

Record<string, string>

Additional key/values merged into the metadata map.

string | null

URL to receive call-event callbacks.

to_metadata

to_metadata(): Record<string, string>

This leg as the metadata dict the room is created with.

Record<string, string>

UNSET

const UNSET: any

Annotated any so recording: any = UNSET type-checks while the honest default stays visible in the signature a reader sees.

ZeroRuntimeChannel

A connection to one runtime, and the sessions started over it. serve and invoke own one for you. Hold one directly to start sessions from async code, to reach a runtime other than the configured one, or to ask a runtime what capacity it has. Connect explicitly with await channel.connect(); every method connects on first use anyway. TLS is decided by the target unless secure says otherwise: local hosts are plaintext, port 443 is TLS. ZERORUNTIME_INSECURE=1 forces plaintext.

Options

ZeroRuntimeChannel(options: ZeroRuntimeChannelOptions)

string | null

Record<string, any> | null

boolean | null

auth_token

auth_token(): void

connected

connected(): void

stub

stub(): void

aclose

aclose(): Promise<void>

Close the connection. Sessions started over it stop being watched.

Promise<void>

attach

attach(session_id: string, __namedParameters: object): Promise<Session>

Attach to a session that is already running.

string

required

The session to attach to.

object

default:"{}"

Promise<Session>

connect

connect(__namedParameters: object): Promise<ZeroRuntimeChannel>

Open the connection. Returns immediately if it is already open.

object

default:"{}"

Promise<ZeroRuntimeChannel>

destroy

destroy(session_id: string, reason: string): Promise<ProtoMessage>

Tear a session down at the runtime, by id.

string

required

The session to destroy.

string

default:"'client requested destroy'"

Recorded against it.

Promise<ProtoMessage>

session_info

session_info(session_id: string): Promise<ProtoMessage>

Look up one session’s state, pid, room and uptime. Works for any session on this runtime, including ones this process did not start.

string

required

Promise<ProtoMessage>

start

start(options: StartOptions): Promise<Session>

Start a session and return it, ready to run. The async counterpart to invoke, and what to reach for inside async code, where invoke cannot be called.

StartOptions

default:"{}"

Promise<Session>

status

status(): Promise<ZeroRuntimeStatus>

Ask the runtime what capacity it has.

Promise<ZeroRuntimeStatus>

use_tls

use_tls(): boolean

Whether this channel will speak TLS. secure wins if it was set, then ZERORUNTIME_INSECURE, then the target itself: a local host is plaintext and port 443 is TLS.

boolean

​Agent

​Options

​agent_id

​instructions

​session

​tools

​add_server

​cleanup

​emit

​hangup

​initialize_mcp

​off

​on

​on_enter

​on_exit

​on_participant_joined

​on_participant_left

​register_tools

​toString

​update_tools

​AgentContext

​Options

​toString

​CallSummary

​Options

​serve

​invoke

​Session

​Constructor

​agent

​ended

​playground_url

​[asyncDispose]

​aclose

​add_handoff

​add_message

​change_component

​change_pipeline

​destroy

​detach

​emit

​end

​events

​get_context_history

​get_metrics

​get_participants

​interrupt

​log_playground

​off

​on

​play_background_audio

​process_text

​publish_to_pubsub

​reply

​say

​set_thinking_audio

​stop

​stop_background_audio

​subscribe_to_pubsub

​toString

​transfer_call

​wait

​current_session

​Participant

​Constructor

​PubSubSubscribeConfig

​Options

​PubSubPublishConfig

​Options

​UtteranceHandle

​Constructor

​interrupted

​state

​done

​toString

​wait

​Room

​Options

​explicitly_set

​Sip

Agent

Options

agent_id

instructions

session

tools

add_server

cleanup

emit

hangup

initialize_mcp

off

on

on_enter

on_exit

on_participant_joined

on_participant_left

register_tools

toString

update_tools

AgentContext

Options

toString

CallSummary

Options

serve

invoke

Session

Constructor

agent

ended

playground_url

[asyncDispose]

aclose

add_handoff

add_message

change_component

change_pipeline

destroy

detach

emit

end

events

get_context_history

get_metrics

get_participants

interrupt

log_playground

off

on

play_background_audio

process_text

publish_to_pubsub

reply

say

set_thinking_audio

stop

stop_background_audio

subscribe_to_pubsub

toString

transfer_call

wait

current_session

Participant

Constructor

PubSubSubscribeConfig

Options

PubSubPublishConfig

Options

UtteranceHandle

Constructor

interrupted

state

done

toString

wait

Room

Options

explicitly_set

Sip