Skip to main content

Usage guide

Setup, environment variables, and Python/JavaScript/Go usage examples.

​
XAILLM

xAI Grok LLM plugin for text and reasoning completions. Wraps the xAI chat completions API (OpenAI-compatible) to integrate Grok models for real-time voice agents. Supports tool/function calling, structured outputs, optional chain-of-thought reasoning, and prompt caching.

​
Constructor

​
api_key
str | None
xAI API key. Falls back to the XAI_API_KEY environment variable when None.
​
model
XAILLMModel | str
default:"grok-4-1-fast-non-reasoning"
Grok model ID. Defaults to 'grok-4-1-fast-non-reasoning'. Other notable IDs: 'grok-4.3' (latest general-purpose, supports reasoning_effort), 'grok-4.20-0309-reasoning', 'grok-4.20-0309-non-reasoning', 'grok-4.20-multi-agent-0309'.
​
base_url
str
default:"https://api.x.ai/v1"
xAI API base URL. Defaults to 'https://api.x.ai/v1'. Override for private deployments.
​
temperature
float
default:"0.7"
Sampling temperature controlling output randomness. Defaults to 0.7. Higher values increase creativity; lower values make output more deterministic.
​
tool_choice
str
default:"auto"
Controls how the model selects tools. Defaults to 'auto'. Other values: 'none', 'required', or a specific tool name dict.
​
max_completion_tokens
int | None
Maximum number of tokens the model may generate per response. None defaults to 1024.
​
top_p
float | None
Nucleus sampling probability mass. None uses the model default.
​
frequency_penalty
float | None
Penalises repeated tokens proportional to their frequency. Range -2.0–2.0; None uses the model default. Cannot be used with reasoning models.
​
presence_penalty
float | None
Penalises tokens that have already appeared, encouraging topic diversity. Range -2.0–2.0; None uses the model default. Cannot be used with reasoning models.
​
seed
int | None
Fixed integer seed for deterministic sampling. None disables seeding.
​
stop
str | None
Stop sequence string; generation halts when this token is produced. None disables early stopping. Cannot be used with reasoning models.
​
user
str | None
Opaque end-user identifier forwarded to the API for abuse monitoring. None omits the field.
​
parallel_tool_calls
bool | None
When True, allows the model to emit multiple tool calls in a single turn. None uses the model default.
​
response_format
dict | None
Structured-output schema dict (e.g. {'type': 'json_object'}). None returns plain text.
​
reasoning_effort
str | None
Chain-of-thought depth for models that support it (e.g. 'grok-4.3'). Valid values: 'none', 'low' (default when omitted), 'medium', 'high'. None lets the model decide. Cannot be combined with frequency_penalty, presence_penalty, or stop.
​
prompt_cache_key
str | None
Cache key string for prompt caching. Requests with the same key reuse cached KV state to reduce latency and cost. None disables caching.
​
service_tier
str | None
API service tier for routing and rate-limit bucketing. None uses the account default.

​
aclose

Release any resources held by the provider.

​
cancel_current_generation

Cancel the in-progress generation for the active session, if any.

​
chat

Generate a streamed chat completion.
​
messages
Any
required
The conversation history to send to the model.
​
tools
Any
Optional tool definitions the model may call.
​
conversational_graph
Any
Optional conversational graph guiding the exchange.
​
returns
AsyncIterator[LLMResponse]
LLMResponse: Response chunks produced as the model generates output.

​
emit

Emit an event, invoking all registered handlers with the given arguments. Handlers are called in registration order. Coroutine handlers are scheduled on the running event loop. If the emitter is closed, the call is a no-op.
​
event
T
required
The event to emit.

​
off

Remove a previously registered handler for an event. If the handler is not registered for the event, the call is a no-op.
​
event
T
required
The event the handler was registered for.
​
callback
Callable[..., Any]
required
The handler to remove.

​
on

Register a handler for an event. Can be used directly by passing a callback, or as a decorator when callback is omitted.
​
event
T
required
The event to listen for.
​
callback
Callable[..., Any] | None
The handler to invoke when the event is emitted. If None, a decorator is returned that registers the decorated function.
​
returns
Callable[..., Any]
The registered callback when callback is provided, otherwise a decorator that registers and returns the function it wraps.

​
XAILLMModel