Skip to main content

Usage guide

Setup, environment variables, and Python/JavaScript/Go usage examples.

​
AzureOpenAILLM

Azure OpenAI LLM plugin. Wraps Azure-hosted OpenAI chat-completion models (GPT-4o, o-series, etc.) with streaming generation, tool/function calling, and structured-output support via a user-managed Azure deployment.

​
Constructor

​
api_key
str | None
Azure OpenAI API key. Falls back to the AZURE_OPENAI_API_KEY environment variable when omitted.
​
azure_endpoint
str | None
Full Azure OpenAI resource endpoint URL, e.g. "https://<resource>.openai.azure.com/". Falls back to the AZURE_OPENAI_ENDPOINT environment variable when omitted.
​
deployment
AzureOpenAILLMModel | str | None
Name of the Azure OpenAI deployment (not the underlying model name). This is the custom name you chose when deploying a model in the Azure portal or AI Foundry, e.g. "gpt-4o-deployment". Accepts any AzureOpenAILLMModel enum value or a raw string. Defaults to None (empty string, must be provided).
​
api_version
str
default:"2024-10-21"
Azure OpenAI REST API version date string. Defaults to "2024-10-21" (GA stable release for chat completions). Use "2025-03-01-preview" or later to access preview features.
​
temperature
float
default:"0.7"
Sampling temperature in [0.0, 2.0]. Lower values produce more deterministic output. Defaults to 0.7.
​
max_output_tokens
int
default:"1024"
Maximum tokens the model may generate per response. Defaults to 1024.
​
top_p
float | None
Nucleus sampling probability mass in (0.0, 1.0]. Mutually exclusive with temperature; set one or the other. Defaults to None (disabled).
​
frequency_penalty
float | None
Float in [-2.0, 2.0]. Penalises token repetition based on cumulative frequency in the output so far. Positive values reduce repetition. Defaults to None.
​
presence_penalty
float | None
Float in [-2.0, 2.0]. Penalises tokens that have already appeared at least once, encouraging topic diversity. Defaults to None.
​
seed
int | None
Integer seed for deterministic sampling. When set, the API makes a best effort to return the same result for identical inputs. Defaults to None.
​
stop
str | None
String (or up to four strings) at which the model stops generating further tokens. Defaults to None.
​
user
str | None
Opaque end-user identifier forwarded to Azure for abuse monitoring. Defaults to None.
​
tool_choice
str | None
Controls how the model selects tools. One of "none" (no tools), "auto" (model decides), or "required" (must call a tool). Also accepts a named tool dict. Defaults to None ("auto" when tools are present).
​
parallel_tool_calls
bool | None
When True, the model may emit multiple tool calls in a single response turn. Defaults to None (provider default, currently True).
​
response_format
Enforce a structured output schema. Pass {"type": "json_object"} for JSON mode or a JSON Schema dict for strict structured outputs. Defaults to None (plain text).
​
reasoning_effort
str | None
Controls reasoning token budget for o-series (reasoning) models. One of "low", "medium", or "high". Ignored for non-reasoning models. Defaults to None.

​
aclose

Release any resources held by the provider.

​
cancel_current_generation

Cancel the in-progress generation for the active session, if any.

​
chat

Generate a streamed chat completion.
​
messages
Any
required
The conversation history to send to the model.
​
tools
Any
Optional tool definitions the model may call.
​
conversational_graph
Any
Optional conversational graph guiding the exchange.
​
returns
AsyncIterator[LLMResponse]
LLMResponse: Response chunks produced as the model generates output.

​
emit

Emit an event, invoking all registered handlers with the given arguments. Handlers are called in registration order. Coroutine handlers are scheduled on the running event loop. If the emitter is closed, the call is a no-op.
​
event
T
required
The event to emit.

​
off

Remove a previously registered handler for an event. If the handler is not registered for the event, the call is a no-op.
​
event
T
required
The event the handler was registered for.
​
callback
Callable[..., Any]
required
The handler to remove.

​
on

Register a handler for an event. Can be used directly by passing a callback, or as a decorator when callback is omitted.
​
event
T
required
The event to listen for.
​
callback
Callable[..., Any] | None
The handler to invoke when the event is emitted. If None, a decorator is returned that registers the decorated function.
​
returns
Callable[..., Any]
The registered callback when callback is provided, otherwise a decorator that registers and returns the function it wraps.

​
AzureOpenAILLMModel