> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Attach RAG to your Agent
> Ground agent answers in a knowledge base with custom retrieval.
RAG (Retrieval-Augmented Generation) helps your AI agent find relevant information from documents to give better answers. It searches a knowledge base and uses that context to respond more accurately.
## Architecture
The RAG pipeline flow:
STT converts speech to text.
The knowledge base fetches relevant documents based on the transcript.
Retrieved context is injected into the LLM prompt.
The LLM generates a grounded response using the context.
TTS converts the response to speech.
## Managed RAG
**Managed RAG** lets you upload knowledge bases from the [Zero runtime](https://in.app.zeroruntime.ai/knowledge-base) dashboard and connect them to your agent. During a conversation, the agent retrieves relevant information from these knowledge bases to provide more accurate, context-aware responses.
Managed RAG is supported in **Python SDK `0.1.5+`**. JavaScript SDK support is coming soon.
### Step 1: Upload a Knowledge Base
Upload your documents and create a knowledge base from the Zero runtime dashboard.
After the knowledge base is created, the dashboard provides a unique **Knowledge ID**.
### Step 2: Attach the Knowledge Base
Pass the Knowledge ID to `KnowledgeBase` and set it as the pipeline's `knowledge_base`. On every user turn the agent searches the attached knowledge bases and places the best-matching sections above the user's message in the LLM prompt. Retrieval runs inside the agent process, so no extra code or hooks are needed.
```python title="Python" Python theme={null}
from zeroruntime import KnowledgeBase, Pipeline
from zeroruntime.inference import CartesiaTTS, DeepgramSTT, GoogleLLM, TurnDetector
from zeroruntime.plugins import SileroVAD
pipeline = Pipeline(
stt=DeepgramSTT(model="nova-2"),
llm=GoogleLLM(model="gemini-3.5-flash"),
tts=CartesiaTTS(),
vad=SileroVAD(),
turn_detector=TurnDetector(),
knowledge_base=KnowledgeBase(
knowledge_ids=["kb_id_1", "kb_id_2"],
top_k=5,
),
)
```
You can attach multiple knowledge bases to the same agent. All of them are searched on every turn.
The `knowledge_base` pipeline parameter works with cascade pipelines (separate STT, LLM, and TTS) only. For a realtime pipeline, use the [knowledge base as a function tool](#knowledge-base-as-a-function-tool).
### Configuration
| Option | Default | Description |
| - | - | - |
| `knowledge_ids` | required | Dashboard IDs of the knowledge bases to search. Must list at least one non-empty ID. |
| `top_k` | `5` | Number of sections retrieved per turn. Must be at least `1`. |
| `context_prefix` | built-in line | Replaces the line placed above the retrieved sections in the prompt, e.g. to tell the LLM how to treat the context. |
| `api_base_url` | Zero Runtime | Host of the knowledge API. Leave unset unless you were given a different endpoint. |
### Customize the context prefix
Use `context_prefix` to control how the LLM is told about the retrieved sections.
```python title="Python" Python theme={null}
knowledge_base = KnowledgeBase(
knowledge_ids=["kb_id_1"],
context_prefix="Answer only from the product documentation below. If it is not covered, say you don't know.",
)
```
`report_unanswered` is deprecated and ignored. Unanswered questions are no longer filed to the dashboard, so remove it from your configuration.
### Knowledge Base as a Function Tool
`knowledge_base=` searches on every user turn. To let the model decide when to search instead, call `KnowledgeBase.retrieve()` from a [function tool](/runtime-sdk/build/tools-and-capabilities/function-tools). This works with any pipeline, cascade or [realtime](/runtime-sdk/build/configure-a-pipeline/modes), and skips the lookup on turns that don't need it (greetings, small talk).
```python title="Python" Python theme={null}
from zeroruntime import Agent, KnowledgeBase, function_tool
kb = KnowledgeBase(knowledge_ids=["kb_id_1"], top_k=5)
@function_tool
async def search_docs(query: str) -> str:
"""Search the product documentation and support policies."""
return await kb.retrieve(query)
class SupportAgent(Agent):
def __init__(self, pipeline):
super().__init__(
instructions=(
"You are a support assistant. Call search_docs before answering "
"product questions, and answer only from its results."
),
pipeline=pipeline,
tools=[search_docs],
)
```
`retrieve()` returns the best sections as text, under the knowledge base's `context_prefix`. Override settings for a single call with keyword arguments:
| Argument | Description |
| - | - |
| `knowledge_ids` | Knowledge bases to search instead of the configured ones. |
| `top_k` | Sections to return instead of the configured `top_k`. |
| `context_prefix` | Line above the sections instead of the configured one. `""` drops it. |
Use either the tool or `Pipeline(knowledge_base=...)`, not both, or each turn searches twice.
Say in the agent's instructions when to call the tool. Models skip tools they aren't told to use and answer from their own knowledge.
## Custom RAG
For full control, Build your own RAG pipeline using any vector database (ChromaDB, Pinecone, etc.) with the `user_turn_start` hook.This hook fires when the user's transcript is ready, before the LLM is called, giving you the perfect place to retrieve documents and inject context.
**1. Set up the vector store.** Create a collection to hold your document embeddings. ChromaDB ships a Python client. The snippet below keeps embeddings in a minimal in-memory index.
```python title="Python" Python theme={null}
self.chroma_client = chromadb.Client()
self.collection = self.chroma_client.create_collection(name="rag_docs")
```
```typescript title="Node JS" Node JS theme={null}
// A vector store of your choosing; the SDK does not ship one.
this.collection = await vector_store.create_collection('rag_docs');
```
**2. Implement `retrieve()`.** Generate a query embedding and search the vector store for the top matching documents.
```python title="Python" Python theme={null}
async def retrieve(self, query: str, k: int = 2) -> list[str]:
response = await self.openai_client.embeddings.create(
input=query, model="text-embedding-ada-002"
)
query_embedding = response.data[0].embedding
results = self.collection.query(
query_embeddings=[query_embedding], n_results=k
)
return results["documents"][0] if results["documents"] else []
```
```typescript title="Node JS" Node JS theme={null}
async retrieve(query: string, k = 2): Promise {
const response = await this.openai_client.embeddings.create({
input: query,
model: 'text-embedding-ada-002',
});
const query_embedding = response.data[0].embedding;
const results = await this.collection.query({
query_embeddings: [query_embedding],
n_results: k,
});
return results.documents?.[0] ?? [];
}
```
**3. Inject context with the `user_turn_start` hook.** Retrieve documents from the transcript and fold them into the system prompt with `session.change_component(instructions=...)` before the LLM is invoked. Rebuild from a base string each turn so the retrieved context doesn't accumulate.
```python title="Python" Python theme={null}
BASE_INSTRUCTIONS = "You are a helpful assistant. Ground your answers in the retrieved context."
@pipeline.on("user_turn_start")
async def on_user_turn_start(transcript: str):
context_docs = await agent.retrieve(transcript)
if context_docs:
context_str = "\n\n".join(
f"Document {i+1}: {doc}" for i, doc in enumerate(context_docs)
)
await agent.session.change_component(
instructions=(
f"{BASE_INSTRUCTIONS}\n\nRetrieved Context:\n{context_str}\n\n"
"Use this context to answer the user's question."
)
)
```
```typescript title="Node JS" Node JS theme={null}
const BASE_INSTRUCTIONS =
'You are a helpful assistant. Ground your answers in the retrieved context.';
pipeline.on('user_turn_start', async (transcript: string) => {
const context_docs = await agent.retrieve(transcript);
if (context_docs.length) {
const context_str = context_docs
.map((doc, i) => `Document ${i + 1}: ${doc}`)
.join('\n\n');
await agent.session.change_component({
instructions:
`${BASE_INSTRUCTIONS}\n\nRetrieved Context:\n${context_str}\n\n` +
"Use this context to answer the user's question.",
});
}
});
```
The LLM then sees the injected context and uses it to generate a grounded answer.
## Best Practices
* Start by retrieving `k=2-3` documents and adjust based on performance.
* Keep document chunk sizes between 300-800 words.
* Use persistent storage for your vector database in production.
* Cache embeddings for frequently asked questions to reduce latency.
* Handle retrieval failures gracefully so the agent can still respond.
## What's Next
Add custom actions to your agent.
Connect external MCP servers as tools.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.