> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Video Input

> Stream live video to a realtime model for continuous vision, from the participant's camera or a shared screen.

For continuous "watch what I'm doing" interactions, pair vision with a
[realtime](/build/configure-a-pipeline/modes) model. The model receives video frames as the
conversation flows, so the agent can react to what it sees in real time (no explicit capture
step needed).

<Note>
  Video input rides on whatever video track the participant publishes. That can be their
  **camera** or a **screen share**; the agent sees either the same way.
</Note>

## Realtime vision

Use a realtime model in the pipeline and turn on `vision=True` for the session, on the
`zeroruntime.Room` you invoke (pass the same `Room` to `zeroruntime.serve()` to apply it to
every caller). Frames from the
participant's video are forwarded to the model alongside audio:

<CodeGroup>
  ```python title="main.py" Python theme={null}
  import zeroruntime
  from zeroruntime import Agent, Pipeline
  from zeroruntime.plugins import OpenAIRealtime
  from zeroruntime.inference import AICousticsDenoise

  AGENT_ID = "vision-agent"

  pipeline = Pipeline(
      llm=OpenAIRealtime(
          model="gpt-4o-realtime-preview", config={"voice": "alloy"}
      ),
      denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
  )

  class VisionAgent(Agent):
      def __init__(self):
          super().__init__(
              agent_id=AGENT_ID,
              instructions="You can see the participant's video. Describe what you observe.",
              pipeline=pipeline,
          )

  if __name__ == "__main__":
      # Pass the class itself (not an instance): serve() builds a fresh VisionAgent +
      # pipeline per call, which is required for correct per-call state under concurrent calls.
      zeroruntime.serve(
          VisionAgent,
          on_ready=lambda: zeroruntime.invoke(
              AGENT_ID, room=zeroruntime.Room(playground=True, vision=True)
          ),
      )
  ```

  ```typescript title="Node JS" Node JS theme={null}
  import * as zeroruntime from '@zeroruntime/js-sdk';
  import { Agent, Pipeline } from '@zeroruntime/js-sdk';
  import { OpenAIRealtime } from '@zeroruntime/js-sdk/plugins';
  import { AICousticsDenoise } from '@zeroruntime/js-sdk/inference';

  const AGENT_ID = 'vision-agent';

  const pipeline = Pipeline({
    llm: OpenAIRealtime({
      model: 'gpt-4o-realtime-preview', config: {voice: 'alloy'},
    }),
    denoise: AICousticsDenoise({ model_id: 'quail-vf-2.2-l-16khz' }),
  });

  class VisionAgent extends Agent {
    constructor() {
      super({
        agent_id: AGENT_ID,
        instructions: "You can see the participant's video. Describe what you observe.",
        pipeline,
      });
    }
  }

  // Pass the class itself (not an instance): serve() builds a fresh VisionAgent +
  // pipeline per call, which is required for correct per-call state under concurrent calls.
  zeroruntime.serve(VisionAgent, {
    on_ready: () => zeroruntime.invoke(AGENT_ID, { room: zeroruntime.Room({ playground: true, vision: true }) }),
  });
  ```
</CodeGroup>

For on-demand snapshots in a cascade pipeline instead, see [Image Input](/build/modalities/vision/image-input).

## References

<Tabs>
  <Tab title="Python">
    #### Examples

    <CardGroup cols={2}>
      <Card title="Vision (Realtime)" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-python-examples/blob/main/vision/vision_realtime.py">
        Stream video frames to a realtime agent.
      </Card>

      <Card title="Vision Hook" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-python-examples">
        Hook into frames before they reach the model.
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="Node JS">
    #### Examples

    <CardGroup cols={2}>
      <Card title="Vision (Realtime)" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-js-examples/blob/main/vision/vision_cascade.ts">
        Stream video frames to a realtime agent.
      </Card>

      <Card title="Vision Hook" icon="github" href="https://github.com/ZeroRuntimeAI/zeroruntime-js-examples">
        Hook into frames before they reach the model.
      </Card>
    </CardGroup>
  </Tab>
</Tabs>