Skip to main content

Usage guide

Setup, environment variables, and Python/JavaScript/Go usage examples.

​
SarvamAILLM

Sarvam AI Large Language Model provider plugin. Wraps the Sarvam AI chat-completions API, which exposes models optimised for Indian languages (22 Indic languages + English). Unique capabilities include Sarvam-specific wiki_grounding (grounding responses in Wikipedia-derived knowledge) and configurable reasoning_effort for chain-of-thought depth control. The API is OpenAI-compatible and supports streaming, tool/function calling, and structured output.

​
Constructor

​
api_key
str | None
Sarvam AI API key. Falls back to the SARVAM_API_KEY environment variable when None.
​
model
SarvamAILLMModel | str
default:"sarvam-30b"
Sarvam AI model identifier. Defaults to "sarvam-30b". Available models: "sarvam-30b": 30 B MoE model (2.4 B active params); strong reasoning and conversational capabilities at a balanced performance-to-cost ratio. "sarvam-105b": 105 B flagship model; highest quality Indian-language understanding, reasoning, and generation.
​
temperature
float
default:"0.7"
Sampling temperature controlling output randomness. Range 0 – 2. Defaults to 0.7 (0.5 when reasoning is enabled, 0.2 when disabled on the API side). Lower values produce more deterministic output.
​
tool_choice
str
default:"auto"
Controls when the model invokes tools. Defaults to "auto". Mirrors the OpenAI tool_choice parameter ("auto", "none", "required", or a specific tool object).
​
max_completion_tokens
int | None
Maximum number of tokens to generate in the completion. Defaults to 1024 when not set. Tier limits: Starter 4 096, Pro 8 192, Business 64 000.
​
top_p
float | None
Nucleus-sampling probability mass cutoff. Range 0 – 1. None uses the provider default.
​
frequency_penalty
float | None
Penalises repeated tokens based on their cumulative frequency. Range –2.0 – 2.0. None disables.
​
presence_penalty
float | None
Penalises tokens that have already appeared at least once. Range –2.0 – 2.0. None disables.
​
seed
int | None
Fixed random seed for deterministic generation. None means no fixed seed.
​
stop
str | None
Up to 4 stop sequences that halt generation early. None disables.
​
user
str | None
An opaque end-user identifier forwarded to the API for abuse-detection purposes. None omits the field.
​
parallel_tool_calls
bool | None
When True, the model may emit multiple tool calls in a single response turn. None uses the provider default.
​
response_format
dict | None
Structured-output specification dict (e.g. {"type": "json_object"}). None returns plain text.
​
reasoning_effort
Optional[str]
Controls the depth of chain-of-thought reasoning. Valid values: "low", "medium", "high", or None to disable reasoning entirely. Higher effort increases both quality and latency/cost since reasoning tokens are billed as completion tokens.
​
wiki_grounding
bool
default:"False"
When True, the model grounds its responses using Wikipedia-derived knowledge, improving factual accuracy for knowledge-intensive queries. Defaults to False.

​
aclose

Release any resources held by the provider.

​
cancel_current_generation

Cancel the in-progress generation for the active session, if any.

​
chat

Generate a streamed chat completion.
​
messages
Any
required
The conversation history to send to the model.
​
tools
Any
Optional tool definitions the model may call.
​
conversational_graph
Any
Optional conversational graph guiding the exchange.
​
returns
AsyncIterator[LLMResponse]
LLMResponse: Response chunks produced as the model generates output.

​
emit

Emit an event, invoking all registered handlers with the given arguments. Handlers are called in registration order. Coroutine handlers are scheduled on the running event loop. If the emitter is closed, the call is a no-op.
​
event
T
required
The event to emit.

​
off

Remove a previously registered handler for an event. If the handler is not registered for the event, the call is a no-op.
​
event
T
required
The event the handler was registered for.
​
callback
Callable[..., Any]
required
The handler to remove.

​
on

Register a handler for an event. Can be used directly by passing a callback, or as a decorator when callback is omitted.
​
event
T
required
The event to listen for.
​
callback
Callable[..., Any] | None
The handler to invoke when the event is emitted. If None, a decorator is returned that registers the decorated function.
​
returns
Callable[..., Any]
The registered callback when callback is provided, otherwise a decorator that registers and returns the function it wraps.

​
SarvamAILLMModel