AWS Bedrock

AWS Bedrock is an LLM plugin. It takes the transcribed conversation and generates the reply using any Bedrock-hosted model (Amazon Nova, Anthropic Claude, Meta Llama, Mistral, and more) through the unified Converse API.

Setup

Set your AWS region in the worker environment. Credentials resolve as explicit constructor argument, then environment variable, then the default AWS credential chain (IAM role or shared profile). Get your access keys from the AWS console:

export AWS_DEFAULT_REGION=<region>
export AWS_ACCESS_KEY_ID=<key-id>
export AWS_SECRET_ACCESS_KEY=<secret-key>

Usage

Import the plugin and pass it to the pipeline’s llm slot.

Python

from zrt.plugins import AWSBedrockLLM

llm = AWSBedrockLLM(
    model="amazon.nova-lite-v1:0",
    region="us-east-1",
    temperature=0.7,
)

# Pipeline(llm=llm, ...)

Parameters

Parameter	Type	Default	Description
`model`	`str`	`"amazon.nova-lite-v1:0"`	Bedrock model id or inference profile ARN. Falls back to the `BEDROCK_INFERENCE_PROFILE_ARN` environment variable when unset.
`region`	`str \| None`	`None`	AWS region for Bedrock Runtime. Falls back to `AWS_DEFAULT_REGION`, then `AWS_REGION`, then `"us-east-1"`.
`aws_access_key_id`	`str \| None`	`None`	AWS access key ID. Falls back to the `AWS_ACCESS_KEY_ID` environment variable.
`aws_secret_access_key`	`str \| None`	`None`	AWS secret access key. Falls back to the `AWS_SECRET_ACCESS_KEY` environment variable.
`aws_session_token`	`str \| None`	`None`	Session token for temporary credentials. Falls back to the `AWS_SESSION_TOKEN` environment variable.
`temperature`	`float`	`0.7`	Sampling randomness; lower is more deterministic.
`max_output_tokens`	`int`	`1024`	Maximum tokens generated per response. The legacy `max_tokens` keyword argument is also accepted.
`top_p`	`float \| None`	`None`	Nucleus-sampling probability mass.
`top_k`	`int \| None`	`None`	Restricts sampling to the top-k most probable tokens. Sent via additional model request fields; support varies by model.
`stop_sequences`	`list[str] \| str \| None`	`None`	Sequence or list of sequences that stop generation.
`tool_choice`	`str`	`"auto"`	How the model decides when to call tools: `"auto"`, `"required"`, `"none"`, or a specific tool name.
`cache_system`	`bool \| None`	`None`	Add a prompt-cache checkpoint after the system prompt to reduce input token usage.
`cache_tools`	`bool \| None`	`None`	Add a prompt-cache checkpoint after the tool definitions.
`strip_thinking`	`bool \| None`	`None`	Remove `<thinking>...</thinking>` spans from the streamed text. Amazon Nova models emit chain-of-thought in these tags, which would otherwise be read aloud by TTS.
`text_tool_calls`	`bool \| None`	`None`	Parse function calls a model prints as plain text instead of native Converse tool use. Auto-enabled for models that lack native tool use (e.g. Gemma) when left unset.
`additional_request_fields`	`dict \| None`	`None`	Extra fields merged into `additionalModelRequestFields` for model-specific parameters.

See the plugins overview for how the LLM slot fits into the rest of the pipeline.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Parameters

​Setup

​Usage

​Parameters

Setup

Usage

Parameters