Skip to main content

Usage guide

Setup, environment variables, and Python/JavaScript/Go usage examples.

​
CometAPILLM

Unified OpenAI-compatible LLM gateway via CometAPI. Routes to many providers’ models behind a single endpoint and key, using the OpenAI chat-completions contract with streaming and tool calling.

​
Constructor

​
api_key
str | None
CometAPI key. Falls back to COMETAPI_API_KEY.
​
model
CometAPILLMModel | str
default:"gpt-4o-mini"
Any CometAPI-routed model id (e.g. "gpt-4o-mini", "claude-...", "gemini-..."). Default "gpt-4o-mini".
​
base_url
str | None
Override the API base URL. Falls back to COMETAPI_BASE_URL.
​
temperature
float
default:"0.7"
Sampling temperature. Default 0.7.
​
tool_choice
str
default:"auto"
Tool selection mode ("auto"/"none"/"required").
​
max_completion_tokens
int | None
Max tokens to generate. Default 1024.
​
top_p
float | None
Nucleus sampling probability mass. Default None.
​
frequency_penalty
float | None
Penalize token frequency, -2.0–2.0. Default None.
​
presence_penalty
float | None
Penalize token presence, -2.0–2.0. Default None.
​
seed
int | None
Seed for reproducible sampling. Default None.
​
stop
str | None
Stop sequence that halts generation. Default None.
​
user
str | None
Stable end-user identifier. Default None.
​
parallel_tool_calls
bool | None
Allow multiple tool calls per turn. Default None.
​
response_format
dict | None
Structured-output spec (e.g. {"type": "json_object"}).

​
aclose

Release any resources held by the provider.

​
cancel_current_generation

Cancel the in-progress generation for the active session, if any.

​
chat

Generate a streamed chat completion.
​
messages
Any
required
The conversation history to send to the model.
​
tools
Any
Optional tool definitions the model may call.
​
conversational_graph
Any
Optional conversational graph guiding the exchange.
​
returns
AsyncIterator[LLMResponse]
LLMResponse: Response chunks produced as the model generates output.

​
emit

Emit an event, invoking all registered handlers with the given arguments. Handlers are called in registration order. Coroutine handlers are scheduled on the running event loop. If the emitter is closed, the call is a no-op.
​
event
T
required
The event to emit.

​
off

Remove a previously registered handler for an event. If the handler is not registered for the event, the call is a no-op.
​
event
T
required
The event the handler was registered for.
​
callback
Callable[..., Any]
required
The handler to remove.

​
on

Register a handler for an event. Can be used directly by passing a callback, or as a decorator when callback is omitted.
​
event
T
required
The event to listen for.
​
callback
Callable[..., Any] | None
The handler to invoke when the event is emitted. If None, a decorator is returned that registers the decorated function.
​
returns
Callable[..., Any]
The registered callback when callback is provided, otherwise a decorator that registers and returns the function it wraps.

​
CometAPILLMModel