Cerebras

Cerebras is an LLM plugin built for fast inference. It takes the transcribed conversation and generates the reply, running open models such as Llama and Qwen on Cerebras hardware.

Setup

Set your Cerebras API key in the worker environment. Generate a key from the Cerebras cloud:

export CEREBRAS_API_KEY=<key>

Usage

Import the plugin and pass it to the pipeline’s llm slot.

from zrt.plugins import CerebrasLLM

llm = CerebrasLLM(
    model="llama3.3-70b",
    temperature=0.7,
)

# Pipeline(llm=llm, ...)

Configuration Options

Constructor parameters for the Python SDK (CerebrasLLM). The JavaScript and Go SDKs expose equivalent options where supported, in their idiomatic form (camelCase / LLMOptions struct).

Parameter	Type	Default	Description
`api_key`	`str`	`None`	Cerebras API key. Falls back to the `CEREBRAS_API_KEY` environment variable when unset.
`model`	`str`	`"llama3.3-70b"`	Cerebras model used to generate replies (e.g. `gpt-oss-120b`, `zai-glm-4.7`, `llama3.3-70b`).
`temperature`	`float`	`0.7`	Sampling randomness; lower is more deterministic.
`tool_choice`	`str`	`"auto"`	How the model decides when to call tools (`"auto"`, `"none"`, or `"required"`).
`max_completion_tokens`	`int`	`None`	Caps the length of each reply.
`top_p`	`float`	`None`	Nucleus-sampling probability cutoff.
`seed`	`int`	`None`	Seed for reproducible sampling.
`stop`	`str`	`None`	Stop sequence that halts generation.
`user`	`str`	`None`	End-user identifier passed to the API.

Import paths

SDK	Import	Constructor
Python	`from zrt.plugins import CerebrasLLM`	`CerebrasLLM(...)`
JavaScript	`from '@zrt/js-sdk/plugins/cerebras'`	`CerebrasLLM({ ... })`
Go	`.../zrt-golang-sdk/plugins/cerebras`	`cerebras.NewLLM(cerebras.LLMOptions{...})`

The reply is streamed to the text-to-speech plugin, which synthesizes the agent’s voice.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Configuration Options

Import paths

​Setup

​Usage

​Configuration Options

​Import paths

Setup

Usage

Configuration Options

Import paths