Fallback Adapter

The Fallback Adapter provides automatic failover between multiple STT, LLM, or TTS providers. It switches providers on two conditions: first, on errors when a provider fails or becomes unavailable, and second, on latency when a provider stays slower than its configured budget. In both cases the system automatically switches to the next configured provider without interrupting the session.

Features

Automatic Fallback

Switches to lower-priority providers if the primary provider fails.

Latency-based Fallback

Optionally switches providers when a component stays above its latency budget for several consecutive turns.

Cooldown-based Retry

Implements a cooldown period before retrying a failed provider, preventing immediate repeated failures.

Auto-Recovery

Automatically switches back to a higher-priority provider once it becomes healthy again.

Permanent Disable

Permanently disables a provider after a configured number of failed recovery attempts.

Error-based Fallback

Here is how you can implement error-based fallback providers for STT, LLM, and TTS in your agent’s Pipeline. When a provider fails or becomes unavailable, the system switches to the next configured provider.

from zeroruntime import FallbackSTT, FallbackLLM, FallbackTTS, Pipeline
from zeroruntime.plugins import SarvamAISTT, DeepgramSTT, OpenAILLM, GoogleLLM, CartesiaTTS, DeepgramTTS

# The head serves, the tail stands by.
pipeline = Pipeline(
    stt=FallbackSTT(
        [SarvamAISTT(model="saarika:v2"), DeepgramSTT(model="nova-2")],
        temporary_disable_sec=30.0,
        permanent_disable_after_attempts=3,
    ),
    llm=FallbackLLM(
        [OpenAILLM(model="gpt-4o-mini"), GoogleLLM(model="gemini-2.5-flash")],
        temporary_disable_sec=30.0,
        permanent_disable_after_attempts=3,
    ),
    tts=FallbackTTS(
        [CartesiaTTS(model="sonic-2"), DeepgramTTS(model="aura-2-thalia-en")],
        temporary_disable_sec=30.0,
        permanent_disable_after_attempts=3,
    ),
)

import { FallbackLLM, FallbackSTT, FallbackTTS, Pipeline } from '@zeroruntime/js-sdk';
import {
  CartesiaTTS, DeepgramSTT, DeepgramTTS, GoogleLLM, OpenAILLM, SarvamAISTT,
} from '@zeroruntime/js-sdk/plugins';

// The head serves, the tail stands by.
const pipeline = Pipeline({
  stt: FallbackSTT([SarvamAISTT({ model: 'saarika:v2' }), DeepgramSTT({ model: 'nova-2' })], {
    temporary_disable_sec: 30.0,
    permanent_disable_after_attempts: 3,
  }),
  llm: FallbackLLM([OpenAILLM({ model: 'gpt-4o-mini' }), GoogleLLM({ model: 'gemini-2.5-flash' })], {
    temporary_disable_sec: 30.0,
    permanent_disable_after_attempts: 3,
  }),
  tts: FallbackTTS([CartesiaTTS({ model: 'sonic-2' }), DeepgramTTS({ model: 'aura-2-thalia-en' })], {
    temporary_disable_sec: 30.0,
    permanent_disable_after_attempts: 3,
  }),
});

Each wrapper goes straight into its normal slot on Pipeline. The rest of your agent setup (Agent, Room, on_enter/on_exit, etc.) stays unchanged. A bare list — stt=[primary, standby] — is the same thing with every option left at its default.

Configuration Options

Set these on the wrapper for the slot they apply to. Each slot carries its own policy.

float

default:"60"

The duration (in seconds) to wait before retrying a failed provider.

int

default:"3"

The maximum number of recovery attempts allowed before a provider is permanently disabled.

Latency-based Fallback

Beyond hard failures, the Fallback Adapter can switch providers when a healthy provider becomes too slow. This is useful for keeping conversations responsive when a provider degrades without erroring out.

Latency-based fallback is off by default. Set latency_threshold_ms on a component to enable it.

Each component measures a relevant latency metric: STT uses stt_latency, LLM uses llm_ttft (time to first token), and TTS uses ttfb (time to first byte). The budget on a wrapper is checked against its own slot’s metric.
A provider is only switched after it stays above the threshold for consecutive_latency_hits turns in a row, avoiding switches caused by a single slow turn.
Recovery and cooldown for a latency-disabled provider use the same temporary_disable_sec and permanent_disable_after_attempts settings as the error path.

To enable latency-based fallback, add latency_threshold_ms (and optionally consecutive_latency_hits) to the wrapper. Budget each slot on its own: a few hundred milliseconds is generous for STT and TTS and punishing for an LLM’s first token.

pipeline = Pipeline(
    stt=FallbackSTT(
        [SarvamAISTT(model="saarika:v2"), DeepgramSTT(model="nova-2")],
        temporary_disable_sec=30.0,
        permanent_disable_after_attempts=3,
        latency_threshold_ms=350,       # enable latency-based fallback
        consecutive_latency_hits=3,
    ),
    llm=FallbackLLM(
        [OpenAILLM(model="gpt-4o-mini"), GoogleLLM(model="gemini-2.5-flash")],
        temporary_disable_sec=30.0,
        permanent_disable_after_attempts=3,
        latency_threshold_ms=800,
        consecutive_latency_hits=3,
    ),
    tts=FallbackTTS(
        [CartesiaTTS(model="sonic-2"), DeepgramTTS(model="aura-2-thalia-en")],
        temporary_disable_sec=30.0,
        permanent_disable_after_attempts=3,
        latency_threshold_ms=250,
        consecutive_latency_hits=3,
    ),
)

const pipeline = Pipeline({
  stt: FallbackSTT([SarvamAISTT({ model: 'saarika:v2' }), DeepgramSTT({ model: 'nova-2' })], {
    temporary_disable_sec: 30.0,
    permanent_disable_after_attempts: 3,
    latency_threshold_ms: 350,  // enable latency-based fallback
    consecutive_latency_hits: 3,
  }),
  llm: FallbackLLM([OpenAILLM({ model: 'gpt-4o-mini' }), GoogleLLM({ model: 'gemini-2.5-flash' })], {
    temporary_disable_sec: 30.0,
    permanent_disable_after_attempts: 3,
    latency_threshold_ms: 800,
    consecutive_latency_hits: 3,
  }),
  tts: FallbackTTS([CartesiaTTS({ model: 'sonic-2' }), DeepgramTTS({ model: 'aura-2-thalia-en' })], {
    temporary_disable_sec: 30.0,
    permanent_disable_after_attempts: 3,
    latency_threshold_ms: 250,
    consecutive_latency_hits: 3,
  }),
});

Configuration Options

You can configure the latency-based fallback behavior using the following parameters:

int

This slot’s latency budget in milliseconds, checked against its own metric (STT stt_latency, LLM llm_ttft, TTS ttfb). Off by default. Pass a value to enable latency-based fallback.

int

default:"3"

The number of consecutive turns that must exceed latency_threshold_ms before switching providers.

Get started

Modalities

Configure Pipeline

Tools and Capabilities

Turn Detection and Interruptions

Telephony Agents

Context Management

Observability and Monitoring

Features

Automatic Fallback

Latency-based Fallback

Cooldown-based Retry

Auto-Recovery

Permanent Disable

Error-based Fallback

Configuration Options

Latency-based Fallback

Configuration Options

References

Examples