Usage guide
Setup, environment variables, and Python/JavaScript/Go usage examples.
OpenAILLM
Language model using OpenAI’s GPT chat models with your own OpenAI key. Supports tool calling, reasoning controls and an optional persistent WebSocket mode for lower latency. Authenticates with OPENAI_API_KEY from your environment, or an explicitapi_key.
Fields
str
default:"gpt-5.4-nano"
Chat model to use, e.g. “gpt-5.4-nano” or “gpt-4o-mini”. Defaults to ‘gpt-5.4-nano’.
str | None
default:"UNSET"
Custom API base URL, for a proxy or an OpenAI-compatible server. Defaults to None (not set).
float
default:"UNSET"
Sampling temperature, 0 to 2. Lower is more focused and repeatable, higher is more varied. Defaults to 0.7.
Literal['auto', 'required', 'none']
default:"UNSET"
Whether the model may call tools: “auto” lets it decide, “required” forces a tool call, “none” disables tools. Defaults to ‘auto’.
int | None
default:"UNSET"
Maximum number of tokens the model may generate per reply. Defaults to None (not set).
float | None
default:"UNSET"
Nucleus sampling cutoff, 0 to 1. Usually tune this or temperature, not both. Defaults to None (not set).
float | None
default:"UNSET"
Penalizes tokens in proportion to how often they have appeared, -2.0 to 2.0. Positive values reduce repetition. Defaults to None (not set).
float | None
default:"UNSET"
Penalizes tokens that have already appeared at all, -2.0 to 2.0. Positive values push toward new topics. Defaults to None (not set).
int | None
default:"UNSET"
Integer seed for best-effort deterministic sampling across identical requests. Defaults to None (not set).
str | None
default:"UNSET"
OpenAI organization ID to bill requests to. Defaults to None (not set).
str | None
default:"UNSET"
OpenAI project ID to bill requests to. Defaults to None (not set).
bool | None
default:"UNSET"
Whether the model may call several tools in a single turn. Defaults to None (not set).
Any
default:"UNSET"
HTTP timeout for API calls, in seconds or as an httpx.Timeout. Defaults to None (not set).
dict | None
default:"UNSET"
Additional HTTP headers sent with every API request. Defaults to None (not set).
dict | None
default:"UNSET"
Additional query-string parameters sent with every API request. Defaults to None (not set).
dict | None
default:"UNSET"
Additional JSON body fields sent with every API request, for options not exposed here. Defaults to None (not set).
int
default:"UNSET"
Number of automatic retries on transient API errors. Defaults to 0.
Literal['none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max'] | None
default:"UNSET"
How much the model reasons before answering: “none”, “minimal”, “low”, “medium”, “high”, “xhigh” or “max”. Reasoning and GPT-5 models only. Defaults to None (not set).
Literal['low', 'medium', 'high'] | None
default:"UNSET"
Length and detail of replies: “low”, “medium” or “high”. Reasoning and GPT-5 models only. Defaults to None (not set).
bool
default:"UNSET"
Use OpenAI’s WebSocket Responses API, keeping one connection open across turns for lower per-turn latency. Defaults to False.
bool
default:"True"
Whether OpenAI stores responses server-side. Only applies when streaming is on; turn off for zero data retention. Defaults to True.
str | None
default:"UNSET"
Custom WebSocket URL for the Responses API when streaming is on. Defaults to None (not set).
ZRTLLM
Language model served from the hosted OpenAI-compatible endpoint, running open models such as Gemma. Choose it for open-weight models without managing your own server. Authenticates with OPENAI_API_KEY from your environment, or an explicitapi_key.
Fields
str
default:"google.gemma-4-31b"
Chat model to use, e.g. “google.gemma-4-31b”. Defaults to ‘google.gemma-4-31b’.
str | None
default:"UNSET"
Custom base URL for the OpenAI-compatible endpoint. Defaults to None (not set).
float
default:"UNSET"
Sampling temperature, 0 to 2. Lower is more focused and repeatable, higher is more varied. Defaults to 0.7.
Literal['auto', 'required', 'none']
default:"UNSET"
Whether the model may call tools: “auto” lets it decide, “required” forces a tool call, “none” disables tools. Defaults to ‘auto’.
int | None
default:"UNSET"
Maximum number of tokens the model may generate per reply. Defaults to None (not set).
float | None
default:"UNSET"
Nucleus sampling cutoff, 0 to 1. Usually tune this or temperature, not both. Defaults to None (not set).
float | None
default:"UNSET"
Penalizes tokens that have already appeared at all, -2.0 to 2.0. Positive values push toward new topics. Defaults to None (not set).
float | None
default:"UNSET"
Penalizes tokens in proportion to how often they have appeared, -2.0 to 2.0. Positive values reduce repetition. Defaults to None (not set).