> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Attach RAG to your Agent

> Ground agent answers in a knowledge base with custom retrieval.

RAG (Retrieval-Augmented Generation) helps your AI agent find relevant information from documents to give better answers. It searches a knowledge base and uses that context to respond more accurately.

## Architecture

The RAG pipeline flow:

<Steps>
  <Step title="Voice Input">
    STT converts speech to text.
  </Step>

  <Step title="Retrieval">
    The knowledge base fetches relevant documents based on the transcript.
  </Step>

  <Step title="Augmentation">
    Retrieved context is injected into the LLM prompt.
  </Step>

  <Step title="Generation">
    The LLM generates a grounded response using the context.
  </Step>

  <Step title="Voice Output">
    TTS converts the response to speech.
  </Step>
</Steps>

## Custom RAG

For full control, Build your own RAG pipeline using any vector database (ChromaDB, Pinecone, etc.) with the `user_turn_start` hook.This hook fires when the user's transcript is ready, before the LLM is called, giving you the perfect place to retrieve documents and inject context.

**1. Set up the vector store.** Create a collection to hold your document embeddings. ChromaDB ships a Python client. The snippet below keeps embeddings in a minimal in-memory index.

<CodeGroup>
  ```python title="Python" Python theme={null}
  self.chroma_client = chromadb.Client()
  self.collection = self.chroma_client.create_collection(name="rag_docs")
  ```

  ```typescript title="Node JS" Node JS theme={null}
  // A vector store of your choosing; the SDK does not ship one.
  this.collection = await vector_store.create_collection('rag_docs');
  ```
</CodeGroup>

**2. Implement `retrieve()`.** Generate a query embedding and search the vector store for the top matching documents.

<CodeGroup>
  ```python title="Python" Python theme={null}
  async def retrieve(self, query: str, k: int = 2) -> list[str]:
      response = await self.openai_client.embeddings.create(
          input=query, model="text-embedding-ada-002"
      )
      query_embedding = response.data[0].embedding
      results = self.collection.query(
          query_embeddings=[query_embedding], n_results=k
      )
      return results["documents"][0] if results["documents"] else []
  ```

  ```typescript title="Node JS" Node JS theme={null}
  async retrieve(query: string, k = 2): Promise<string[]> {
    const response = await this.openai_client.embeddings.create({
      input: query,
      model: 'text-embedding-ada-002',
    });
    const query_embedding = response.data[0].embedding;
    const results = await this.collection.query({
      query_embeddings: [query_embedding],
      n_results: k,
    });
    return results.documents?.[0] ?? [];
  }
  ```
</CodeGroup>

**3. Inject context with the `user_turn_start` hook.** Retrieve documents from the transcript and fold them into the system prompt with `session.change_component(instructions=...)` before the LLM is invoked. Rebuild from a base string each turn so the retrieved context doesn't accumulate.

<CodeGroup>
  ```python title="Python" Python theme={null}
  BASE_INSTRUCTIONS = "You are a helpful assistant. Ground your answers in the retrieved context."

  @pipeline.on("user_turn_start")
  async def on_user_turn_start(transcript: str):
      context_docs = await agent.retrieve(transcript)
      if context_docs:
          context_str = "\n\n".join(
              f"Document {i+1}: {doc}" for i, doc in enumerate(context_docs)
          )
          await agent.session.change_component(
              instructions=(
                  f"{BASE_INSTRUCTIONS}\n\nRetrieved Context:\n{context_str}\n\n"
                  "Use this context to answer the user's question."
              )
          )
  ```

  ```typescript title="Node JS" Node JS theme={null}
  const BASE_INSTRUCTIONS =
    'You are a helpful assistant. Ground your answers in the retrieved context.';

  pipeline.on('user_turn_start', async (transcript: string) => {
    const context_docs = await agent.retrieve(transcript);
    if (context_docs.length) {
      const context_str = context_docs
        .map((doc, i) => `Document ${i + 1}: ${doc}`)
        .join('\n\n');
      await agent.session.change_component({
        instructions:
          `${BASE_INSTRUCTIONS}\n\nRetrieved Context:\n${context_str}\n\n` +
          "Use this context to answer the user's question.",
      });
    }
  });
  ```
</CodeGroup>

The LLM then sees the injected context and uses it to generate a grounded answer.

## Best Practices

* Start by retrieving `k=2-3` documents and adjust based on performance.
* Keep document chunk sizes between 300-800 words.
* Use persistent storage for your vector database in production.
* Cache embeddings for frequently asked questions to reduce latency.
* Handle retrieval failures gracefully so the agent can still respond.

## What's Next

<CardGroup cols={2}>
  <Card title="Function Tools" icon="wrench" href="/build/tools-and-capabilities/function-tools">
    Add custom actions to your agent.
  </Card>

  <Card title="MCP" icon="plug" href="/build/tools-and-capabilities/mcp">
    Connect external MCP servers as tools.
  </Card>
</CardGroup>