> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Zero Runtime Inference Gateway

> Route STT, LLM, TTS, turn detection, and denoise through the Zero Runtime Inference Gateway with a single auth token, no per-provider API keys.

The **Inference Gateway** is a unified entry point for the model stack. Instead of holding a
separate API key for each provider, you authenticate once with your Zero Runtime auth token
and route speech-to-text, LLM, text-to-speech, turn detection, and denoise through the
gateway. It manages the upstream provider connections, resampling, and streaming
server-side.

## Setup

Gateway providers connect using your Zero Runtime auth token, no provider API keys are
required. Set it in the worker environment:

```bash theme={null}
export ZRT_AUTH_TOKEN=<token>
```

## Usage

Import gateway providers from `zrt.inference` and drop them into the pipeline exactly like
normal plugins. Every gateway class is a drop-in replacement for its `zrt.plugins`
equivalent. Turn detection is the unified `TurnDetector` (imported from `zrt.plugins`) using
an Echo model, which also routes through the gateway:

```python theme={null}
from zrt import Pipeline
from zrt.inference import SarvamAISTT, GoogleLLM, CartesiaTTS, AICousticsDenoise
from zrt.plugins import SileroVAD, TurnDetector

pipeline = Pipeline(
    stt=SarvamAISTT(),        # via gateway
    llm=GoogleLLM(),          # via gateway
    tts=CartesiaTTS(),        # via gateway
    vad=SileroVAD(),          # local plugin
    turn_detector=TurnDetector(model="echo-large"),  # Echo, via gateway
    denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
)
```

You can mix gateway and direct-provider components freely in the same pipeline. For example,
run STT and TTS through the gateway while the LLM uses its own key.

## Supported providers

| Stage              | Gateway providers                                 |
| :----------------- | :------------------------------------------------ |
| **Speech-to-text** | Deepgram, Google, Sarvam AI, AssemblyAI, Cartesia |
| **LLM**            | Google, Sarvam AI                                 |
| **Text-to-speech** | Cartesia, Google, Deepgram, Sarvam AI             |
| **Turn detection** | Echo (echo-small, echo-large)                     |
| **Denoise**        | ai-coustics, Sanas                                |

## Import paths

Every gateway class is imported from the single `zrt.inference` module:

| Stage          | Import                                                                                      |
| :------------- | :------------------------------------------------------------------------------------------ |
| Speech-to-text | `from zrt.inference import DeepgramSTT, GoogleSTT, SarvamAISTT, AssemblyAISTT, CartesiaSTT` |
| LLM            | `from zrt.inference import GoogleLLM, SarvamAILLM`                                          |
| Text-to-speech | `from zrt.inference import CartesiaTTS, GoogleTTS, DeepgramTTS, SarvamAITTS`                |
| Turn detection | `from zrt.plugins import TurnDetector` (Echo models route through the gateway)              |
| Denoise        | `from zrt.inference import Denoise, AICousticsDenoise, SanasDenoise`                        |

Denoisers also expose factory methods, `Denoise.sanas()` and `Denoise.aicoustics()`, which
return the same configured component as the named aliases above.

## Why use it

* **One credential.** Authenticate with a single Zero Runtime token instead of provisioning
  and rotating a key for every provider.
* **Less client work.** Connection management, resampling, and streaming happen server-side.
* **Easy switching.** Swap a gateway provider without onboarding a new API key.

## What's next

<CardGroup cols={2}>
  <Card title="Configure a Pipeline" icon="diagram-project" href="/build/integrations/inference-gateway">
    How gateway providers slot into a pipeline, with cascading and realtime examples.
  </Card>

  <Card title="Plugins" icon="puzzle-piece" href="/plugins/overview">
    The full catalog of STT, LLM, TTS, and denoise providers.
  </Card>
</CardGroup>