Skip to main content

Usage guide

Setup, environment variables, and Python/JavaScript/Go usage examples.

​
OpenAILLM

OpenAI chat/completions LLM provider with streaming and tool-calling support. Wraps OpenAI’s GPT model families behind the SDK’s LLM contract. Note the two parameter regimes: gpt-5.x reasoning models are tuned via reasoning_effort and verbosity, while gpt-4.x non-reasoning models are tuned via temperature and top_p.

​
Constructor

​
api_key
str | None
OpenAI API key. Falls back to the OPENAI_API_KEY environment variable when omitted.
​
model
OpenAILLMModel | str
default:"gpt-5.4-nano"
Model id or OpenAILLMModel. Defaults to 'gpt-5.4-nano'. Use a gpt-5.x family model for reasoning workloads or a gpt-4.x family model for classic completions.
​
temperature
float
default:"0.7"
Sampling temperature (0–2); higher is more random, lower more deterministic. Defaults to 0.7. Adjust this or top_p, not both. Applies to non-reasoning (gpt-4.x) models.
​
max_output_tokens
int
default:"1024"
Upper bound on generated tokens, including reasoning tokens for reasoning models. Defaults to 1024. The legacy max_completion_tokens kwarg is accepted and mapped here.
​
top_p
float | None
Nucleus-sampling probability mass (0–1); the model considers only the top top_p of token probability. Defaults to None. Adjust this or temperature, not both.
​
frequency_penalty
float | None
-2.0–2.0; positive values discourage verbatim repetition. Defaults to None.
​
presence_penalty
float | None
-2.0–2.0; positive values encourage new topics. Defaults to None.
​
seed
int | None
Best-effort deterministic sampling seed (beta). Defaults to None.
​
response_format
Output format spec, e.g. {"type": "json_object"} or a json_schema for Structured Outputs. Defaults to None (text).
​
tool_choice
Tool-selection control: "none", "auto", "required", or a specific {"type": "function", ...} spec. Defaults to None.
​
parallel_tool_calls
bool | None
Whether to allow parallel function calls. Defaults to None (provider default of enabled).
​
stop
str | None
Up to 4 stop sequences at which generation halts. Defaults to None.
​
user
str | None
Stable end-user identifier for abuse monitoring (hash to avoid PII). Defaults to None.
​
reasoning_effort
str | None
Reasoning compute budget for gpt-5.x reasoning models. Common values: 'minimal', 'low', 'medium', 'high' (model-dependent; 'none' requires gpt-5.1 or later). Defaults to None.
​
verbosity
str | None
Response verbosity for reasoning models: 'low', 'medium', or 'high'. Defaults to None.
​
streaming
bool
default:"False"
Whether to stream the response incrementally. Defaults to False.
​
wss_url
str | None
Optional WebSocket endpoint override. Defaults to None.
​
store
bool
default:"True"
Whether OpenAI may store the request/response for later retrieval. Defaults to True.

​
aclose

Release any resources held by the provider.

​
cancel_current_generation

Cancel the in-progress generation for the active session, if any.

​
chat

Generate a streamed chat completion.
​
messages
Any
required
The conversation history to send to the model.
​
tools
Any
Optional tool definitions the model may call.
​
conversational_graph
Any
Optional conversational graph guiding the exchange.
​
returns
AsyncIterator[LLMResponse]
LLMResponse: Response chunks produced as the model generates output.

​
emit

Emit an event, invoking all registered handlers with the given arguments. Handlers are called in registration order. Coroutine handlers are scheduled on the running event loop. If the emitter is closed, the call is a no-op.
​
event
T
required
The event to emit.

​
off

Remove a previously registered handler for an event. If the handler is not registered for the event, the call is a no-op.
​
event
T
required
The event the handler was registered for.
​
callback
Callable[..., Any]
required
The handler to remove.

​
on

Register a handler for an event. Can be used directly by passing a callback, or as a decorator when callback is omitted.
​
event
T
required
The event to listen for.
​
callback
Callable[..., Any] | None
The handler to invoke when the event is emitted. If None, a decorator is returned that registers the decorated function.
​
returns
Callable[..., Any]
The registered callback when callback is provided, otherwise a decorator that registers and returns the function it wraps.

​
OpenAILLMModel