> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Attach RAG to your Agent

> Ground agent answers in a knowledge base with custom retrieval.

RAG (Retrieval-Augmented Generation) helps your AI agent find relevant information from documents to give better answers. It searches a knowledge base and uses that context to respond more accurately.

## Architecture

The RAG pipeline flow:

<Steps>
  <Step title="Voice Input">
    STT converts speech to text.
  </Step>

  <Step title="Retrieval">
    The knowledge base fetches relevant documents based on the transcript.
  </Step>

  <Step title="Augmentation">
    Retrieved context is injected into the LLM prompt.
  </Step>

  <Step title="Generation">
    The LLM generates a grounded response using the context.
  </Step>

  <Step title="Voice Output">
    TTS converts the response to speech.
  </Step>
</Steps>

## Custom RAG

For full control, Build your own RAG pipeline using any vector database (ChromaDB, Pinecone, etc.) with the `user_turn_start` hook.This hook fires when the user's transcript is ready, before the LLM is called, giving you the perfect place to retrieve documents and inject context.

**1. Set up the vector store.** Create a collection to hold your document embeddings. ChromaDB ships Python and JavaScript clients; in Go, use any vector store client. The snippet below keeps embeddings in a minimal in-memory index.

<CodeGroup>
  ```python title="main.py" Python theme={null}
  self.chroma_client = chromadb.Client()
  self.collection = self.chroma_client.create_collection(name="rag_docs")
  ```
</CodeGroup>

**2. Implement `retrieve()`.** Generate a query embedding and search the vector store for the top matching documents.

<CodeGroup>
  ```python title="main.py" Python theme={null}
  async def retrieve(self, query: str, k: int = 2) -> list[str]:
      response = await self.openai_client.embeddings.create(
          input=query, model="text-embedding-ada-002"
      )
      query_embedding = response.data[0].embedding
      results = self.collection.query(
          query_embeddings=[query_embedding], n_results=k
      )
      return results["documents"][0] if results["documents"] else []
  ```
</CodeGroup>

**3. Inject context with the `user_turn_start` hook.** Retrieve documents from the transcript and fold them into the system prompt with `session.update_instructions(...)` (`updateInstructions` in JavaScript, `UpdateInstructions` in Go) before the LLM is invoked. Rebuild from a base string each turn so the retrieved context doesn't accumulate.

<CodeGroup>
  ```python title="main.py" Python theme={null}
  BASE_INSTRUCTIONS = "You are a helpful assistant. Ground your answers in the retrieved context."

  @pipeline.on("user_turn_start")
  async def on_user_turn_start(transcript: str):
      context_docs = await agent.retrieve(transcript)
      if context_docs:
          context_str = "\n\n".join(
              f"Document {i+1}: {doc}" for i, doc in enumerate(context_docs)
          )
          await agent.session.update_instructions(
              f"{BASE_INSTRUCTIONS}\n\nRetrieved Context:\n{context_str}\n\n"
              "Use this context to answer the user's question."
          )
  ```
</CodeGroup>

The LLM then sees the injected context and uses it to generate a grounded answer.

## Best Practices

* Start by retrieving `k=2-3` documents and adjust based on performance.
* Keep document chunk sizes between 300-800 words.
* Use persistent storage for your vector database in production.
* Cache embeddings for frequently asked questions to reduce latency.
* Handle retrieval failures gracefully so the agent can still respond.

## What's Next

<CardGroup cols={2}>
  <Card title="Function Tools" icon="wrench" href="/build/tools-and-capabilities/function-tools">
    Add custom actions to your agent.
  </Card>

  <Card title="MCP" icon="plug" href="/build/tools-and-capabilities/mcp">
    Connect external MCP servers as tools.
  </Card>
</CardGroup>