> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Observation

> What the follower saw and where it was at one instant, and the version the leader receives.

**Everything on this page travels follower to leader.** The follower reports
where it is and what its cameras see, and the leader receives that as a
`SyncedObservation`.

```mermaid theme={null}
sequenceDiagram
    participant L as Leader
    participant F as Follower
    L->>F: Command, where to move
    F->>F: checks your limits, writes AppliedAction
    F->>L: Observation, which frames go together
    F->>L: RobotState, joints and safe state
    F-->>L: camera video
    L->>L: pairs them into SyncedObservation
```

What goes the other way is an [Action](/robotics/api-reference/python/action).

| Type | You get it from |
| - | - |
| `Observation` | `follower.last_observation`, `on_observation` on either side |
| `SyncedObservation` | `leader.observation()` |

## Observation

What the follower saw and where it was at one instant: one camera frame per
label, plus the joints read on the same tick.

<ParamField path="observation_id" type="int">
  Identifies this instant. Counts up and never repeats, and a command carries it
  back so you can tell which observation the leader was acting on.
</ParamField>

<ParamField path="frames" type="dict">
  One entry per camera label, each carrying that frame's id and capture time.
  Identity only, no pixels: the images travel on the video track.
</ParamField>

<ParamField path="capture_ts" type="float">
  When it was captured, in milliseconds.
</ParamField>

<ParamField path="skew_ms" type="float">
  How far apart the cameras were, oldest to newest. `0.0` with fewer than two
  cameras.
</ParamField>

<Note>
  Camera skew is recorded rather than smoothed away, so you can tell whether two
  cameras really described the same instant.
</Note>

`on_observation` gives you this type on both machines.

```python theme={null}
@leader.on_observation
def _(obs, staleness_ms):
    print(obs.observation_id, list(obs.frames), obs.skew_ms)
```

## SyncedObservation

What the leader receives: images and joints describing the same instant, with
the joints worked out for the video frame's own capture time.

<ParamField path="joints" type="dict[str, float]">
  Where the follower was when that frame was taken.
</ParamField>

<ParamField path="images" type="dict[str, ndarray]">
  One decoded image per camera label. `{}` until video is up, which is how you
  know it is not ready yet.
</ParamField>

<ParamField path="observation_id" type="int | None">
  Which observation the joints came from.
</ParamField>

<ParamField path="capture_ts" type="float | None">
  When the follower captured that frame, in milliseconds.
</ParamField>

<ParamField path="interpolated_ms" type="float">
  The gap between the frame and the nearest joint reading, in milliseconds. `0`
  means one landed on it, and anything larger is how far the joints had to be
  worked out.
</ParamField>

<ParamField path="exact" type="bool">
  `True` when `interpolated_ms` is `0`. The joints were measured, not
  calculated.
</ParamField>

<ParamField path="bracketed" type="bool">
  `True` when the frame sat between two joint readings, so the joints were
  worked out from both sides of it. `False` means the nearest reading was
  carried forward.
</ParamField>

<ParamField path="skew_ms" type="float">
  How far apart the cameras in `images` were, oldest to newest. `0.0` with
  fewer than two cameras.
</ParamField>

```python theme={null}
obs = leader.observation()
print(obs.observation_id, list(obs.images), obs.interpolated_ms)
```

<Warning>
  Several frames can share one `observation_id` here, so it identifies where the
  joints came from, not the row. Do not join a leader recording to a follower
  recording row by row. See
  [Recording](/robotics/concepts/recording#how-to-record).
</Warning>

## RobotState

The follower's condition on one tick, delivered with every observation.

<ParamField path="joints" type="dict[str, float]">
  Read fresh every observation.
</ParamField>

<ParamField path="safe_state" type="str">
  `"disconnected"`, `"holding"`, `"active"`, `"estopped"` or `"torque_off"`.
  A plain string, so compare it against the
  [`SafeState`](/robotics/api-reference/python/safety-config#values) value rather than
  the enum member.
</ParamField>

<ParamField path="reason" type="str | None">
  Why it is in that state, when there is one: `"starvation"`, `"estop"`,
  `"lease_lost"`, `"transport_lost"`, `"robot_fault"`, `"manual"` or
  `"stalled"`. `None` while nothing has tripped.
</ParamField>

<ParamField path="load, temp_c" type="dict[str, float]">
  Per joint, and only when your `telemetry()` provides them.
</ParamField>

```python theme={null}
@follower.on_state
def _(state):
    print(state.safe_state, state.joints)
```

<Note>
  `load` and `temp_c` are read once a second by default, so the same value
  repeats until the next read. They are there to spot a motor straining or
  overheating, not to track it tick by tick.
</Note>

## Related

<CardGroup cols={2}>
  <Card title="Action" icon="bolt" href="/robotics/api-reference/python/action">
    What the leader asked for, and what the follower actually wrote.
  </Card>

  <Card title="Sync" icon="film" href="/robotics/concepts/sync">
    How joints and frames are matched to the same instant.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.