> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Modalities

> The input and output channels your agent can work with: speech and audio, text, vision, and avatars.

A **modality** is a channel your agent uses to perceive or respond. Zero Runtime agents are
voice-first, but the same pipeline can take text in, see video, and render a visual avatar,
often at the same time. Pick the modalities your experience needs; the pipeline wires them in.

<CardGroup cols={2}>
  <Card title="Speech & Audio" icon="waveform" href="/build/modalities/speech-and-audio/overview">
    Voice in, voice out, plus background audio and TTS caching.
  </Card>

  <Card title="Text" icon="keyboard" href="/build/modalities/text/overview">
    Text input and output for chatbots, omnichannel, and debugging.
  </Card>

  <Card title="Vision" icon="eye" href="/build/modalities/vision/overview">
    Let the agent see: send camera or screen frames to the model.
  </Card>

  <Card title="Avatars" icon="user" href="/build/modalities/avatars/overview">
    Give the agent a real-time visual face.
  </Card>
</CardGroup>

## How modalities map to the pipeline

Modalities are selected by the components you pass to the [`Pipeline`](/build/configure-a-pipeline/overview)
and by per-session flags on `zeroruntime.Room(...)`:

| Modality         | Turned on by                                                                  |
| :--------------- | :---------------------------------------------------------------------------- |
| Speech & audio   | `stt` + `tts` (or a realtime model), the default voice loop                   |
| Text             | Sending text with `pipeline.process_text(...)`; LLM-only or LLM+TTS pipelines |
| Vision           | `zeroruntime.Room(vision=True)` + `session.reply(..., frames=N)`              |
| Avatars          | `Pipeline(avatar=...)`                                                        |
| Background audio | `zeroruntime.Room(background_audio=True)`                                     |

Modalities compose: a pipeline can run voice, vision, and an avatar together.

## References

<Tabs>
  <Tab title="Python">
    #### Examples

    <CardGroup cols={2}>
      <Card title="Multimodal Agent" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-python-examples/blob/main/vision/vision_cascade.py">
        Combine speech, text and vision in one agent.
      </Card>

      <Card title="Cascade Basic" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-python-examples/blob/main/getting_started/cascade_basic.py">
        Voice agent built from a cascade pipeline.
      </Card>

      <Card title="Realtime Basic" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-python-examples/blob/main/getting_started/realtime_basic.py">
        Speech-to-speech realtime agent.
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="Node JS">
    #### Examples

    <CardGroup cols={2}>
      <Card title="Multimodal Agent" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-js-examples/blob/main/vision/vision_cascade.ts">
        Combine speech, text and vision in one agent.
      </Card>

      <Card title="Cascade Basic" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-js-examples/blob/main/getting_started/cascade_basic.ts">
        Voice agent built from a cascade pipeline.
      </Card>

      <Card title="Realtime Basic" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-js-examples/blob/main/getting_started/realtime_basic.ts">
        Speech-to-speech realtime agent.
      </Card>
    </CardGroup>
  </Tab>
</Tabs>