Skip to main content
Simulation tests your agent with realistic callers. A simulated caller brings a situation to your agent, talks to it in text or audio, and a judge scores your agent’s side of the conversation against criteria you write. You see which conversations passed, which turn failed and why, and how to fix it. Open Simulation in the sidebar. It has three tabs: Runs, Scenarios, and Personas.
Simulation page with the Runs, Scenarios, and Personas tabs, the Run simulation and Write a Scenario buttons, and a list of runs with status, mode, pass rate, cost, and start time

​
Core concepts

​
How a run is built

Every selected scenario is paired with every selected persona. In audio mode, each pair is also played once from every selected environment.
For example, 3 scenarios, 2 personas, and 2 environments make 12 audio conversations. A run can have up to 500 conversations.

​
Modes

The modes you can pick depend on the agent’s pipeline. Text only isn’t available for an agent that uses a realtime model, because a realtime model has no separate text step. Run those agents in audio mode. See Run a simulation.

​
Before you start

  • Deploy the branch you want to test. A simulation tests the version that’s deployed on a branch, not your draft. See Deploy an agent.
  • Have an active API key on your account.
  • Have enough credits. You see the estimated cost before anything runs.

​
How it works

1

Set up scenarios and personas

Start with the default scenarios and personas, write your own, or generate scenarios from your agent’s prompt. See Scenarios and Personas.
2

Run a simulation

Pick the agent, branch, scenarios, personas, and mode, review the cost, and start the run. See Run a simulation.
3

Review the results

Follow the run live, open failed conversations to see the failing turn and the judge’s reasoning, and copy its suggested fixes. See Review results.
4

Fix and re-run

Update the agent, commit and deploy the change, then re-run the simulation to compare pass rates.

Scenarios

Write the situations and criteria to test.

Personas

Define who calls and how they behave.

Run a simulation

Configure text or audio runs and see the cost.

Review results

Find failing turns and how to fix them.