> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Zero Runtime Inference Gateway

> Route STT, LLM, TTS, turn detection, and denoise through the Zero Runtime Inference Gateway with a single auth token, no per-provider API keys.

The **Inference Gateway** is a unified entry point for the model stack. Instead of holding a
separate API key for each provider, you authenticate once with your Zero Runtime auth token
and route speech-to-text, LLM, text-to-speech, turn detection, and denoise through the
gateway. It manages the upstream provider connections, resampling, and streaming
server-side.

## Setup

Gateway providers connect using your Zero Runtime auth token, no provider API keys are
required. Set it in the worker environment:

```bash theme={null}
export ZERORUNTIME_AUTH_TOKEN=<token>
```

## Usage

Import gateway providers from `zeroruntime.inference` (Python) or
`@zeroruntime/js-sdk/inference` (Node JS) and drop them into the pipeline exactly like normal
plugins. Every gateway class is a drop-in replacement for its `plugins` equivalent. Turn
detection is the unified `TurnDetector`, imported from the same `inference` module, using an
Echo model, which also routes through the gateway:

<CodeGroup>
  ```python Python theme={null}
  from zeroruntime import Pipeline
  from zeroruntime.inference import AICousticsDenoise, CartesiaTTS, GoogleLLM, SarvamAISTT, TurnDetector
  from zeroruntime.plugins import SileroVAD

  pipeline = Pipeline(
      stt=SarvamAISTT(),        # via gateway
      llm=GoogleLLM(),          # via gateway
      tts=CartesiaTTS(),        # via gateway
      vad=SileroVAD(),          # local plugin
      turn_detector=TurnDetector(model="echo-large"),  # Echo, via gateway
      denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
  )
  ```

  ```typescript Node JS theme={null}
  import { Pipeline } from '@zeroruntime/js-sdk';
  import { AICousticsDenoise, CartesiaTTS, GoogleLLM, SarvamAISTT, TurnDetector } from '@zeroruntime/js-sdk/inference';
  import { SileroVAD } from '@zeroruntime/js-sdk/plugins';

  const pipeline = Pipeline({
    stt: SarvamAISTT(),  // via gateway
    llm: GoogleLLM(),  // via gateway
    tts: CartesiaTTS(),  // via gateway
    vad: SileroVAD(),  // local plugin
    turn_detector: TurnDetector({ model: 'echo-large' }),  // Echo, via gateway
    denoise: AICousticsDenoise({ model_id: 'quail-vf-2.2-l-16khz' }),
  });
  ```
</CodeGroup>

You can mix gateway and direct-provider components freely in the same pipeline. For example,
run STT and TTS through the gateway while the LLM uses its own key.

## Supported providers

| Stage | Gateway providers |
| :- | :- |
| **Speech-to-text** | Deepgram, Google, Sarvam AI, AssemblyAI, Cartesia |
| **LLM** | Google, Sarvam AI |
| **Text-to-speech** | Cartesia, Google, Deepgram, Sarvam AI |
| **Turn detection** | Echo (echo-small, echo-large, echo-omni) |
| **Denoise** | ai-coustics, Sanas |

## Import paths

Every gateway class is imported from the single `inference` module of each SDK:

| Stage | Python | Node JS |
| :- | :- | :- |
| Speech-to-text | `from zeroruntime.inference import DeepgramSTT, GoogleSTT, SarvamAISTT, AssemblyAISTT, CartesiaSTT` | `import { DeepgramSTT, GoogleSTT, SarvamAISTT, AssemblyAISTT, CartesiaSTT } from '@zeroruntime/js-sdk/inference'` |
| LLM | `from zeroruntime.inference import GoogleLLM, SarvamAILLM` | `import { GoogleLLM, SarvamAILLM } from '@zeroruntime/js-sdk/inference'` |
| Text-to-speech | `from zeroruntime.inference import CartesiaTTS, GoogleTTS, DeepgramTTS, SarvamAITTS` | `import { CartesiaTTS, GoogleTTS, DeepgramTTS, SarvamAITTS } from '@zeroruntime/js-sdk/inference'` |
| Turn detection | `from zeroruntime.inference import TurnDetector` (Echo models route through the gateway) | `import { TurnDetector } from '@zeroruntime/js-sdk/inference'` |
| Denoise | `from zeroruntime.inference import AICousticsDenoise, SanasDenoise` | `import { AICousticsDenoise, SanasDenoise } from '@zeroruntime/js-sdk/inference'` |

## Why use it

* **One credential.** Authenticate with a single Zero Runtime token instead of provisioning
  and rotating a key for every provider.
* **Less client work.** Connection management, resampling, and streaming happen server-side.
* **Easy switching.** Swap a gateway provider without onboarding a new API key.

## What's next

<CardGroup cols={2}>
  <Card title="Configure a Pipeline" icon="diagram-project" href="/runtime-sdk/build/integrations/inference-gateway">
    How gateway providers slot into a pipeline, with cascading and realtime examples.
  </Card>

  <Card title="Plugins" icon="puzzle-piece" href="/runtime-sdk/plugins/overview">
    The full catalog of STT, LLM, TTS, and denoise providers.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.