> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Zero Runtime Inference Gateway

> Route STT, LLM, TTS, turn detection, and denoise through the Zero Runtime Inference Gateway with a single auth token, no per-provider API keys.

The **Inference Gateway** is a unified entry point for the model stack. Instead of holding a
separate API key for each provider, you authenticate once with your Zero Runtime auth token
and route speech-to-text, LLM, text-to-speech, turn detection, and denoise through the
gateway. It manages the upstream provider connections, resampling, and streaming
server-side.

## Setup

Gateway providers connect using your Zero Runtime auth token, no provider API keys are
required. Set it in the worker environment:

```bash theme={null}
export ZERORUNTIME_AUTH_TOKEN=<token>
```

## Usage

Import gateway providers from `zrt.inference` and drop them into the pipeline exactly like
normal plugins. Every gateway class is a drop-in replacement for its `zrt.plugins`
equivalent. Turn detection is the unified `TurnDetector` (imported from `zrt.plugins`) using
an Echo model, which also routes through the gateway:

<CodeGroup>
  ```python Python theme={null}
  from zeroruntime import Pipeline
  from zeroruntime.inference import AICousticsDenoise, CartesiaTTS, GoogleLLM, SarvamAISTT, TurnDetector
  from zeroruntime.plugins import SileroVAD

  pipeline = Pipeline(
      stt=SarvamAISTT(),        # via gateway
      llm=GoogleLLM(),          # via gateway
      tts=CartesiaTTS(),        # via gateway
      vad=SileroVAD(),          # local plugin
      turn_detector=TurnDetector(model="echo-large"),  # Echo, via gateway
      denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
  )
  ```

  ```typescript Node JS theme={null}
  import { Pipeline } from '@zeroruntime/js-sdk';
  import { AICousticsDenoise, CartesiaTTS, GoogleLLM, SarvamAISTT, TurnDetector } from '@zeroruntime/js-sdk/inference';
  import { SileroVAD } from '@zeroruntime/js-sdk/plugins';

  const pipeline = Pipeline({
    stt: SarvamAISTT(),  // via gateway
    llm: GoogleLLM(),  // via gateway
    tts: CartesiaTTS(),  // via gateway
    vad: SileroVAD(),  // local plugin
    turn_detector: TurnDetector({ model: 'echo-large' }),  // Echo, via gateway
    denoise: AICousticsDenoise({ model_id: 'quail-vf-2.2-l-16khz' }),
  });
  ```
</CodeGroup>

You can mix gateway and direct-provider components freely in the same pipeline. For example,
run STT and TTS through the gateway while the LLM uses its own key.

## Supported providers

| Stage              | Gateway providers                                 |
| :----------------- | :------------------------------------------------ |
| **Speech-to-text** | Deepgram, Google, Sarvam AI, AssemblyAI, Cartesia |
| **LLM**            | Google, Sarvam AI                                 |
| **Text-to-speech** | Cartesia, Google, Deepgram, Sarvam AI             |
| **Turn detection** | Echo (echo-small, echo-large)                     |
| **Denoise**        | ai-coustics, Sanas                                |

## Import paths

Every gateway class is imported from the single `zrt.inference` module:

| Stage          | Import                                                                                              |
| :------------- | :-------------------------------------------------------------------------------------------------- |
| Speech-to-text | `from zeroruntime.inference import DeepgramSTT, GoogleSTT, SarvamAISTT, AssemblyAISTT, CartesiaSTT` |
| LLM            | `from zeroruntime.inference import GoogleLLM, SarvamAILLM`                                          |
| Text-to-speech | `from zeroruntime.inference import CartesiaTTS, GoogleTTS, DeepgramTTS, SarvamAITTS`                |
| Turn detection | `from zeroruntime.plugins import TurnDetector` (Echo models route through the gateway)              |
| Denoise        | `from zeroruntime.inference import Denoise, AICousticsDenoise, SanasDenoise`                        |

Denoisers also expose factory methods, `Denoise.sanas()` and `Denoise.aicoustics()`, which
return the same configured component as the named aliases above.

## Why use it

* **One credential.** Authenticate with a single Zero Runtime token instead of provisioning
  and rotating a key for every provider.
* **Less client work.** Connection management, resampling, and streaming happen server-side.
* **Easy switching.** Swap a gateway provider without onboarding a new API key.

## What's next

<CardGroup cols={2}>
  <Card title="Configure a Pipeline" icon="diagram-project" href="/build/integrations/inference-gateway">
    How gateway providers slot into a pipeline, with cascading and realtime examples.
  </Card>

  <Card title="Plugins" icon="puzzle-piece" href="/plugins/overview">
    The full catalog of STT, LLM, TTS, and denoise providers.
  </Card>
</CardGroup>