Agent you
subclass to give it instructions and tools, and the AgentSession that runs it,
speaking, listening, and managing the call lifecycle. serve and invoke register
an agent and start sessions against it.
Agent
A voice agent: instructions, tools, and lifecycle behavior. Subclass to add behavior: define@function_tool methods for tools and override
on_enter/on_exit for lifecycle hooks. The agent carries its
Pipeline and is registered for incoming sessions by zrt.serve.
Constructor
System prompt defining the agent’s behavior and persona.
Display name. Defaults to
agent_id.The
Pipeline (STT/LLM/TTS + VAD/turn detection) to run on.Extra
@function_tool callables. Tools defined as methods on the agent are auto-registered, so use this only for tools defined elsewhere.Stable id the agent is registered and routed under. Required.
MCP servers whose tools are exposed to the agent.
Carry chat history over when handed off from another agent.
Optional line spoken automatically when the agent starts.
Disallow barge-in during the greeting.
Text appended to the agent’s voice id (provider-specific).
Whether to apply
voice_suffix. Default True.Alternate agents this one can hand off to.
Hard cap on session length for this agent.
add_server
The MCP server to add.
capture_frames
How many recent frames to return. Default 1.
A list of frames, empty when vision is not active.
cleanup
emit
The event to emit.
hangup
initialize_mcp
off
The event the handler was registered for.
The handler to remove.
on
callback is omitted.
The event to listen for.
The handler to invoke when the event is emitted. If
None, a decorator is returned that registers the decorated function.The registered callback when
callback is provided, otherwise a decorator that registers and returns the function it wraps.on_enter
await self.session.say(...).
on_exit
play_background_audio
Audio file URL/path.
Playback volume. Default 1.0.
Loop the clip. Default False.
Replace any thinking-audio while this plays. Default True.
preload_background_audio
Audio file URL/path.
Playback volume. Default 1.0.
register_a2a
The agent card describing this agent for A2A.
register_tools
@function_tool callables to the agent’s tool set, skipping tools
already registered. Called automatically during initialization.
set_thinking_audio
Audio file URL/path. Pass a falsy value to clear it.
Playback volume, 0.0–1.0. Default 0.3.
stop_background_audio
unregister_a2a
update_tools
The new list of
@function_tool callables.AgentSession
A running agent session, your handle to interact with the live conversation. Available asself.session inside an agent. Use it to speak (say,
reply), steer the turn (interrupt, generate), read or edit the
conversation (get_context_history, update_instructions,
update_tools), hot-swap components (update_provider), play audio, and
end the call (hangup, leave).
Constructor
The agent whose instructions, tools and callbacks drive the session.
The media pipeline (STT/LLM/TTS and hooks) for the session.
Idle timeout in seconds after which the wake-up callback fires.
Maximum number of wake-up callbacks before the session is closed;
None means unlimited.Background audio configuration for the session.
Handler dispatched when DTMF digits are received.
Detector invoked when voicemail is detected.
Recording configuration applied to the session.
active_agent
add_shutdown_callback
close; coroutine callbacks are awaited.
A callable taking no arguments; may be sync or async.
call_transfer
Auth token for the transfer.
Destination number/address to transfer to.
cancel_generation
close
Optional reason recorded for the close.
emit
The event to emit.
fetch_context_history
get_context_history: always retrieves the latest
messages for the live session.
Return only the most recent
last_n messages (0 = all).Include tool-call/tool-result messages. Default False.
Include system messages. Default False.
A list of message dicts.
generate
The user-turn text to inject and generate a reply for.
get_context_history
await the
result to fetch a fresh copy of the live conversation. Each item is a dict with
role, content, message_id and related fields.
Return only the most recent
last_n messages (0 = all).Include tool-call/tool-result messages. Default False.
Include system messages. Default False.
An awaitable list of message dicts.
handoffs
A copy of the handoff records, each a dict with
from, to and reason fields.hangup
Reason recorded on the close (defaults to “manual_hangup”).
inject_context
An object exposing
messages() or an iterable of messages, each injected via inject_message.True if every message was injected successfully, False otherwise.inject_message
role,
content, message_id, images, tool_calls and tool_call_id.
The message to inject.
Optional placement hint for where to insert the message.
True if the message was accepted, False otherwise (e.g. no active session or transport).interrupt
Interrupt even when the current utterance is non-interruptible.
last_handoff_reason
leave
off
The event the handler was registered for.
The handler to remove.
on
callback is omitted.
The event to listen for.
The handler to invoke when the event is emitted. If
None, a decorator is returned that registers the decorated function.The registered callback when
callback is provided, otherwise a decorator that registers and returns the function it wraps.play_background_audio
file_url (or file_path/url), volume, looping and a playback
mode. Local files are read and sent inline. A config whose enabled flag is
False is ignored.
Audio source as a string path/URL, dict, or config object.
Whether this playback may override active thinking audio.
preload_background_audio
play_background_audio. A config whose
enabled flag is False is ignored.
Audio source as a string path/URL, dict, or config object.
publish_message
An object with a
topic and message, plus optional options and payload. Non-string messages, options and payloads are JSON-encoded.push_audio_frame
Raw PCM audio bytes; empty input is ignored.
Sample rate of the PCM data in Hz. Default 48000.
register_handoff_agent
The agent to register; indexed by its
agent_id or id.The registered agent’s id, or an empty string if it has none.
remove_message
Id of the message to remove (from
get_context_history).True if the message was removed.reply
say (which speaks fixed text), this asks the LLM to produce the
response from instructions.
Prompt telling the agent what to say.
Await until playback finishes before returning. Default True.
Image frames to include with the prompt (vision).
Instead of
frames, attach this many most-recent camera frames.Allow the caller to barge in. Default True.
An
UtteranceHandle you can await for playback completion.say
The exact text to speak.
Allow the caller to barge in over it. Default True.
Optional one-off voice override for this utterance.
Reserved; pre-synthesized audio injection is not supported yet.
Reserved; transcript handling is automatic.
An
UtteranceHandle you can await for playback completion.send_a2a
Id of the agent to deliver the message to; required.
The message body to deliver.
Optional id to correlate request and response messages.
send_image
mime_type; other inputs are
coerced to JPEG bytes. Empty input is ignored.
Image data as bytes-like or any object coercible to JPEG.
MIME type to use when
data is already bytes. Default "image/jpeg".send_message_with_frames
(payload, mime_type) tuple, a dict with data and an
optional mime_type, or a raw image payload; non-bytes payloads are coerced
to JPEG. The call is skipped when there is no message, no frames, and
num_latest_frames is not positive.
Text to send alongside the frames; may be
None.The image frames to attach.
MIME type used when a frame does not specify one.
Number of most-recent camera frames to attach instead of (or in addition to)
frames.set_user_input_enabled
False to ignore the caller, True to resume.start
on_enter. Optionally holds startup until a participant (and audio stream)
is present and can block until shutdown.
Hold
on_enter until a participant joins.Block until the session is closed, then clean up.
Hang up the SIP call when the session shuts down.
How long to wait for a participant, in ms.
Also wait for the participant’s audio stream before proceeding; defaults to enabled for SIP calls.
Optional runtime configuration overrides.
Caller-supplied session identifier to associate with the call.
start_recording
Recording configuration; falls back to the session-level
recording= config if omitted. If neither is set, the call is ignored.start_thinking_audio
agent.set_thinking_audio(...)); otherwise the call is ignored.
stop_background_audio
stop_recording
stop_thinking_audio
subscribe_a2a
a2a_message event
for each well-formed envelope received. Re-subscribing to the same topic is a
no-op; switching agents resubscribes to the new agent’s topic.
subscribe_pubsub
The topic to subscribe to; must be non-empty.
Optional callable invoked with each message dict matching the topic.
The internal handler registered for the callback, or
None if no callback was supplied or the topic was empty.update_instructions
The new system prompt.
update_interrupt_config
Detection mode:
"VAD_ONLY", "STT_ONLY" or "HYBRID".Minimum speech duration (seconds) to count as a barge-in.
Minimum words before an interruption is honored.
Minimum gap between interruptions, in ms.
Pause length treated as a false interrupt, in ms.
Resume the agent’s turn after a false interrupt.
update_provider
Which stage to change, e.g.
"stt", "llm", "tts".The provider to switch to.
Optional provider-specific settings (model, voice, …).
update_tools
The new list of
@function_tool callables.wait_for_synthesis_done
Maximum time to wait in seconds;
None waits indefinitely.True if synthesis completed (or none was in progress), False if the wait timed out.warm_transfer
Room
Per-session room configuration passed toinvoke.
Fields
Existing room to join. A new room is created when
None.Auth token for the room. Falls back to the ZRT auth env vars.
Display name to publish the agent under. Defaults to the agent id.
Return a
playground_url for opening the session. Default True.Enable camera-frame capture for the agent. Default False.
Record the session. Default False.
Enable background-audio playback. Default False.
Deliver raw input audio to listeners. Default False.
End the session when the caller leaves. Default True.
Hard cap on session duration, in seconds.
End the session if nobody joins in time.
Fixed participant id for the agent.
Override the signaling base URL.
Stream session logs to the dashboard. Default True.
Log level for dashboard logs. Default
"INFO".Sip
Telephony (SIP) details for a phone-call session passed toinvoke.
Fields
Destination number/address for an outbound call.
Caller id presented on the call.
Call direction/type (e.g. inbound/outbound).
Identifier correlating the call.
URL to receive call-event callbacks.
Additional provider-specific key/values forwarded with the call.