> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Context Window
> Automatically manage conversation history with token and item budgets.
Automatically manage history for the `Pipeline`. Configure a `ContextWindow` with token and item budgets. Before each LLM call it summarizes older turns and truncates excess items so your agent preserves key memory without exceeding limits.
Context Window replaces manual context management. All token budgeting, history compression, and truncation is handled automatically through a single configuration object.
## How Context Window Works
Context Window is configured on a `Pipeline` instance via the `context_window` parameter. It runs a two-step management cycle before every LLM call:
| Step | Action | Purpose |
| :-------------- | :--------------------------------------- | :------------------------------------------------------ |
| **1. Compress** | Summarize old conversation turns via LLM | Preserve long-term memory without keeping every message |
| **2. Truncate** | Remove oldest non-protected items | Enforce hard token and item count limits |
Three item types are **always protected** and never removed:
| Protected Item | Reason |
| :-------------------- | :---------------------------------------------------- |
| **System message** | Agent instructions must persist |
| **Summary message** | Compressed history is the agent's long-term memory |
| **Last user message** | LLMs require the conversation to end with a user turn |
```python title="Python" Python theme={null}
from zeroruntime import Pipeline, ContextWindow
from zeroruntime.plugins import DeepgramSTT, OpenAILLM, CartesiaTTS, SileroVAD
from zeroruntime.inference import AICousticsDenoise, TurnDetector
pipeline = Pipeline(
stt=DeepgramSTT(),
llm=OpenAILLM(),
tts=CartesiaTTS(),
vad=SileroVAD(),
turn_detector=TurnDetector(model="echo-large"),
# Configure context window management
context_window=ContextWindow(
max_tokens=4000,
max_context_items=20,
keep_recent_turns=3,
max_tool_calls_per_turn=10,
),
denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
)
```
```typescript title="Node JS" Node JS theme={null}
import { Pipeline, ContextWindow } from '@zeroruntime/js-sdk';
import { DeepgramSTT, OpenAILLM, CartesiaTTS, SileroVAD } from '@zeroruntime/js-sdk/plugins';
import { AICousticsDenoise, TurnDetector } from '@zeroruntime/js-sdk/inference';
const pipeline = Pipeline({
stt: DeepgramSTT(),
llm: OpenAILLM(),
tts: CartesiaTTS(),
vad: SileroVAD(),
turn_detector: TurnDetector({ model: 'echo-large' }),
// Configure context window management
context_window: ContextWindow({
max_tokens: 4000,
max_context_items: 20,
keep_recent_turns: 3,
max_tool_calls_per_turn: 10,
}),
denoise: AICousticsDenoise({ model_id: 'quail-vf-2.2-l-16khz' }),
});
```
## Configuration Parameters
Adjust these parameters to match your application's token limits, cost targets, and expected tool-call patterns.
| Parameter | Type | Default | Description |
| :------------------------ | :------------ | :------ | :-------------------------------------------------------------------------------------------------------- |
| `max_tokens` | `int \| None` | `None` | • Token budget for the full history.
• On overflow, old turns compress, then truncate. |
| `max_context_items` | `int \| None` | `None` | • Max items (messages + tool calls + results).
• Either limit triggers compression/truncation. |
| `keep_recent_turns` | `int` | `3` | • Recent exchanges kept verbatim.
• Older ones get summarized. |
| `max_tool_calls_per_turn` | `int` | `10` | • Max tool calls per user turn.
• Prevents infinite tool-call loops. |
| `summary_llm` | `LLM \| None` | `None` | • Optional LLM for summaries.
• Falls back to the main LLM.
• Use a cheaper model to cut costs. |
## Processing Cycle
Before each LLM call, an internal process is automatically executed. This process performs two steps in sequence.
### Step 1: Compress
When the context exceeds the token or item budget **and** there are enough old turns to compress (more user turns than `keep_recent_turns`), compression kicks in:
1. **Split** - Separate items into old turns and recent turns (keeping the last N user exchanges).
2. **Render** - Convert old items into human-readable text for the summarization prompt.
3. **Summarize** - Call the LLM (or `summary_llm`) to generate a concise summary.
4. **Replace** - Remove all old items and insert the summary as a system message tagged with the internal Summary source (rendered as `[Conversation Summary]`).
The summary preserves:
* Key facts, names, and numbers
* Decisions made and their reasoning
* Tool/function call results and outcomes
* Commitments or promises the assistant made
* User objectives, preferences, and unresolved tasks
### Step 2: Truncate
After compression (or if compression wasn't needed), truncation enforces hard limits:
1. Remove the oldest non-protected items one at a time.
2. Function call/output pairs are removed together to avoid orphaned tool calls.
3. Continue until both `max_tokens` and `max_context_items` are satisfied.
4. If only protected items remain, stop even if still over budget.
## How Tool Chaining Works
Context Window works with tool chaining. Here's the lifecycle of a multi-tool turn:
User says "Plan for Dubai" → LLM returns `get_weather(Dubai)`.
Tool executes → result added to context → LLM called again.
LLM returns `get_clothing_advice(22°C)` → execute → call LLM again.
LLM returns `get_activity_suggestion(22°C, "jacket")` → execute → call LLM.
LLM returns text "Dubai is 22°C, wear a jacket, go hiking!" → spoken by TTS.
That's 3 tool calls + 1 text response = 4 rounds, well within `max_tool_calls_per_turn=10`.
Some LLMs (Anthropic Claude, OpenAI GPT-4o) can return multiple tool calls in a single response. These are collected and executed in parallel using `asyncio.gather`, then all results are added to context before the next LLM call. Google Gemini sends one tool call at a time (always sequential).
## Example
A full example combining Context Window with tool chaining for a production-ready travel assistant:
```python title="main.py" Python theme={null}
import aiohttp
import zeroruntime
from zeroruntime import Agent, Pipeline, function_tool, ContextWindow
from zeroruntime.plugins import DeepgramSTT, CartesiaTTS, OpenAILLM, SileroVAD
from zeroruntime.inference import AICousticsDenoise, TurnDetector
AGENT_ID = "travel-assistant"
@function_tool
async def get_weather(city: str) -> dict:
"""Get the current weather temperature for a given city."""
city_coords = {
"dubai": (25.2048, 55.2708),
"mumbai": (19.0760, 72.8777),
"new york": (40.7128, -74.0060),
}
coords = city_coords.get(city.lower(), (25.2048, 55.2708))
lat, lon = coords
url = f"https://api.open-meteo.com/v1/forecast?latitude={lat}&longitude={lon}¤t=temperature_2m"
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
if response.status == 200:
data = await response.json()
temp = data["current"]["temperature_2m"]
return {"city": city, "temperature": temp, "unit": "Celsius"}
else:
return {"city": city, "temperature": 25, "unit": "Celsius", "note": "fallback"}
@function_tool
async def get_clothing_advice(temperature: float) -> dict:
"""Get clothing recommendation based on temperature."""
if temperature > 35:
advice = "Very light breathable clothes, hat, and sunscreen."
elif temperature > 25:
advice = "Light clothes like t-shirt and shorts."
elif temperature > 15:
advice = "Light jacket or sweater with comfortable pants."
elif temperature > 5:
advice = "Warm coat, scarf, and layered clothing."
else:
advice = "Heavy winter coat, gloves, hat, and thermal layers."
return {"temperature": temperature, "clothing_advice": advice}
pipeline = Pipeline(
stt=DeepgramSTT(),
llm=OpenAILLM(),
tts=CartesiaTTS(),
vad=SileroVAD(),
turn_detector=TurnDetector(model="echo-large"),
# ── Context Window Configuration ───────────────────────────
# max_tokens: token budget for the conversation (~8 city plans + chat).
# max_context_items: max messages + tool calls before compression/truncation.
# keep_recent_turns: recent exchanges kept verbatim; older ones summarized.
# max_tool_calls_per_turn: safety limit to prevent infinite tool-call loops.
context_window=ContextWindow(
max_tokens=4000,
max_context_items=20,
keep_recent_turns=3,
max_tool_calls_per_turn=10,
),
denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
)
class TravelAgent(Agent):
def __init__(self):
super().__init__(
agent_id=AGENT_ID,
instructions=(
"You are a helpful travel assistant. When a user asks what to do in a city:\n"
"1. FIRST call get_weather to get the temperature\n"
"2. THEN call get_clothing_advice with that temperature\n"
"3. Combine results into a natural spoken response (2-3 sentences max)."
),
pipeline=pipeline,
tools=[get_weather, get_clothing_advice],
)
async def on_enter(self) -> None:
await self.session.say("Hi! I'm your travel assistant. Ask me about any city!")
async def on_exit(self) -> None:
pass
if __name__ == "__main__":
# Pass the class itself (not an instance): serve() builds a fresh TravelAgent +
# pipeline per call, which is required for correct per-call state under concurrent calls.
zeroruntime.serve(TravelAgent, on_ready=lambda: zeroruntime.invoke(AGENT_ID, room=zeroruntime.Room(playground=True)))
```
```typescript title="main.ts" Node JS theme={null}
import * as zeroruntime from '@zeroruntime/js-sdk';
import { Agent, ContextWindow, Pipeline, Room, function_tool } from '@zeroruntime/js-sdk';
import { AICousticsDenoise, TurnDetector } from '@zeroruntime/js-sdk/inference';
import { CartesiaTTS, DeepgramSTT, OpenAILLM, SileroVAD } from '@zeroruntime/js-sdk/plugins';
const AGENT_ID = 'travel-assistant';
const CITY_COORDS: Record = {
dubai: [25.2048, 55.2708],
mumbai: [19.076, 72.8777],
'new york': [40.7128, -74.006],
};
const get_weather = function_tool({
name: 'get_weather',
description: 'Get the current weather temperature for a given city.',
parameters: {
city: { type: 'string', description: 'The city to look up.' },
},
execute: async ({ city }) => {
const [lat, lon] = CITY_COORDS[city.toLowerCase()] ?? CITY_COORDS.dubai;
const url =
`https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${lon}` +
'¤t=temperature_2m';
const response = await fetch(url);
if (!response.ok) {
return { city, temperature: 25, unit: 'Celsius', note: 'fallback' };
}
const data = (await response.json()) as { current: { temperature_2m: number } };
return { city, temperature: data.current.temperature_2m, unit: 'Celsius' };
},
});
const get_clothing_advice = function_tool({
name: 'get_clothing_advice',
description: 'Get clothing recommendation based on temperature.',
parameters: {
temperature: { type: 'number', description: 'The temperature in Celsius.' },
},
execute: async ({ temperature }) => {
let advice: string;
if (temperature > 35) advice = 'Very light breathable clothes, hat, and sunscreen.';
else if (temperature > 25) advice = 'Light clothes like t-shirt and shorts.';
else if (temperature > 15) advice = 'Light jacket or sweater with comfortable pants.';
else if (temperature > 5) advice = 'Warm coat, scarf, and layered clothing.';
else advice = 'Heavy winter coat, gloves, hat, and thermal layers.';
return { temperature, clothing_advice: advice };
},
});
const pipeline = Pipeline({
stt: DeepgramSTT(),
llm: OpenAILLM(),
tts: CartesiaTTS(),
vad: SileroVAD(),
turn_detector: TurnDetector({ model: 'echo-large' }),
// Context Window Configuration
// max_tokens: token budget for the conversation (~8 city plans + chat).
// max_context_items: max messages + tool calls before compression/truncation.
// keep_recent_turns: recent exchanges kept verbatim; older ones summarized.
// max_tool_calls_per_turn: safety limit to prevent infinite tool-call loops.
context_window: ContextWindow({
max_tokens: 4000,
max_context_items: 20,
keep_recent_turns: 3,
max_tool_calls_per_turn: 10,
}),
denoise: AICousticsDenoise({ model_id: 'quail-vf-2.2-l-16khz' }),
});
class TravelAgent extends Agent {
constructor() {
super({
agent_id: AGENT_ID,
instructions:
'You are a helpful travel assistant. When a user asks what to do in a city:\n' +
'1. FIRST call get_weather to get the temperature\n' +
'2. THEN call get_clothing_advice with that temperature\n' +
'3. Combine results into a natural spoken response (2-3 sentences max).',
pipeline,
tools: [get_weather, get_clothing_advice],
});
}
async on_enter(): Promise {
await this.session!.say("Hi! I'm your travel assistant. Ask me about any city!");
}
}
// Pass the class itself (not an instance): serve() builds a fresh TravelAgent +
// pipeline per call, which is required for correct per-call state under concurrent calls.
await zeroruntime.serve(TravelAgent, {
on_ready: () => zeroruntime.invoke(AGENT_ID, { room: Room({ playground: true }) }),
});
```
## What's Next
How ChatContext records conversation memory.
Share context across specialized agents.
## References
#### Examples
Compress and truncate long conversations.
#### Examples
Compress and truncate long conversations.