Skip to main content

Usage guide

Setup, environment variables, and Python/JavaScript/Go usage examples.

​
GoogleLLM

Google Gemini LLM via the Gemini API or Vertex AI. Supports the full Gemini model family (2.5 and 3.x series) including thinking/reasoning models, safety-settings control, and optional service-account-authenticated Vertex AI routing.

​
Constructor

​
api_key
str | None
Google AI Studio API key. Falls back to GOOGLE_API_KEY env var. Ignored when vertexai=True.
​
model
GoogleLLMModel | str
default:"gemini-2.5-flash-lite"
Gemini model ID. Known values (GoogleLLMModel): "gemini-2.5-flash-lite" (default), "gemini-2.5-flash", "gemini-2.5-pro", "gemini-3-flash-preview", "gemini-3.1-pro-preview", "gemini-3.5-flash", "gemini-3.1-flash-lite", "gemini-flash-latest". Any string is accepted for forward compatibility.
​
temperature
float
default:"0.7"
Sampling temperature controlling response randomness. Higher values produce more varied output. Default: 0.7.
​
max_output_tokens
int
default:"8192"
Maximum number of tokens in the generated response. Default: 8192.
​
thinking_budget
Optional[int]
default:"0"
Token budget reserved for the model’s internal reasoning chain (applicable to thinking-capable models). 0 disables thinking; higher values allow deeper reasoning at the cost of latency and tokens. Default: 0.
​
include_thoughts
bool
default:"False"
When True and thinking_budget > 0, the model’s reasoning trace is included in the response alongside the final answer. Default: False.
​
safety_settings
Optional[List[Dict[str, str]]]
List of {"category": ..., "threshold": ...} dicts overriding Gemini’s default harm-blocking thresholds. See Google AI safety-settings docs for valid category and threshold strings.
​
top_p
Optional[float]
Nucleus sampling probability mass cutoff (0.0–1.0). None uses the model default.
​
top_k
Optional[int]
Top-K sampling limit. None uses the model default.
​
presence_penalty
Optional[float]
Penalises tokens that have already appeared, encouraging topic diversity (range typically −2.0–2.0). None uses the model default.
​
frequency_penalty
Optional[float]
Penalises tokens proportional to how often they have appeared, reducing repetition (range typically −2.0–2.0). None uses the model default.
​
seed
Optional[int]
Fixed random seed for deterministic outputs when supported. None uses non-deterministic sampling.
​
vertexai
bool
default:"False"
When True, routes requests through Vertex AI instead of the Gemini API. Requires project_id. Default: False.
​
project_id
Optional[str]
Google Cloud project ID. Required when vertexai=True. Falls back to GOOGLE_CLOUD_PROJECT env var (resolved in the calling code).
​
location
str
default:"us-central1"
Vertex AI regional endpoint, e.g. "us-central1" (default) or "europe-west4". Ignored when vertexai=False.
​
service_account_json
Optional[Union[str, Dict]]
Service-account credentials as a JSON string or dict. Used for Vertex AI authentication when vertexai=True. Takes precedence over service_account_path.
​
service_account_path
Optional[str]
Path to a service-account JSON key file. Used when service_account_json is not provided and vertexai=True. Falls back to GOOGLE_APPLICATION_CREDENTIALS env var.

​
aclose

Release any resources held by the provider.

​
cancel_current_generation

Cancel the in-progress generation for the active session, if any.

​
chat

Generate a streamed chat completion.
​
messages
Any
required
The conversation history to send to the model.
​
tools
Any
Optional tool definitions the model may call.
​
conversational_graph
Any
Optional conversational graph guiding the exchange.
​
returns
AsyncIterator[LLMResponse]
LLMResponse: Response chunks produced as the model generates output.

​
emit

Emit an event, invoking all registered handlers with the given arguments. Handlers are called in registration order. Coroutine handlers are scheduled on the running event loop. If the emitter is closed, the call is a no-op.
​
event
T
required
The event to emit.

​
off

Remove a previously registered handler for an event. If the handler is not registered for the event, the call is a no-op.
​
event
T
required
The event the handler was registered for.
​
callback
Callable[..., Any]
required
The handler to remove.

​
on

Register a handler for an event. Can be used directly by passing a callback, or as a decorator when callback is omitted.
​
event
T
required
The event to listen for.
​
callback
Callable[..., Any] | None
The handler to invoke when the event is emitted. If None, a decorator is returned that registers the decorated function.
​
returns
Callable[..., Any]
The registered callback when callback is provided, otherwise a decorator that registers and returns the function it wraps.

​
GoogleLLMModel