Review Results

After a run finishes, start on the Overview tab to see how the agent scored, then open a run to find which judge failed on which call and why.

​

Overview

Once a run has finished, the Overview tab shows the scores from the newest finished run:

• Scorecard: A · Accuracy and X · Experience as the share of scenarios that passed, D · Diagnostic as the number of flags raised, and the run’s score with how many calls landed in each band. When there’s an earlier run that’s fair to compare with, each card shows the change since it.
• What failed most: the three scored judges that failed the most tries. Select one to open the run filtered to that judge.
• Recent runs: the last three runs. Select All runs to open the Runs tab.

If a newer run is still going, failed, or was stopped, a banner says so, and the scores stay on the last finished run.

​

Runs

The Runs tab lists every run of this agent under Run history, with its status, when and how it started, and its accuracy, experience, and number of diagnostic flags. Filter by status, or search. How a run started is shown next to its start time: Manual, API, Auto run · live calls, or Auto run · simulation.

​

A run

Select a run to open it. The header shows the run’s number and status, and its actions:

• Stop run while it’s running. See Stop a run.
• Compare with run #N when there’s an earlier run that’s fair to compare with. See Compare runs.
• Re-score on a finished run. See Re-score a run.
• Run again on a finished run, or Try again on a failed or stopped one. See Run again.

Below the header you see how and when the run started, how long it took, the version (branch) it tested, the scorecard, the number of scenarios, tries per scenario, total tries, and the judges.

​

Scorecard

The same score cards as on the Overview tab, for this run. When each scenario was played more than once, switch between Best of N tries and All N tries. See How a run is scored.

​

Every try, every judge

A grid with one row per scenario and caller, and one column per judge, grouped by category. Each cell shows one try’s verdict: Each row also shows the try’s score and band. Select a cell or a scenario name to open the try. When you open the run from What failed most or from a judge, the grid shows only that judge and the scenarios where it failed. Select Show every judge to see the full grid again.

​

A try

A try opens in a side panel. Use Prev and Next to move through every try in the run. The top shows the try’s status, its score and band, how many scored judges passed, and the caller persona. A try passes a category only when every judge in that category passes.

• Verdicts: every judge’s verdict, failures first, with its reason. A judge that quoted a turn shows the quote and the turn number; select See turn N in transcript to jump to it. A code check shows the value it measured for each rule.
• Transcript: the full conversation.

Select Open judge to open the first scored judge that failed, or Open in Simulation to open a simulated call in Simulation, with its recording.

​

Fix and re-run

​

Run again

Run again opens the Run evaluation panel limited to what the original run used: its branch, its scenarios and callers or its calls, its tries, its judges, and its scorecard. You can untick some, but not add new ones, so the two runs stay comparable. For a run that graded simulated calls, choose:

• Simulate again: plays the same scenarios again on the branch as it’s deployed now. Use this to test a change. Only offered when the original run graded Scenarios.
• Same calls: grades the same simulated calls again. Nothing new is simulated, so this doesn’t test a change to the agent.

For a run that graded live calls, Same calls grades the same calls again.

​

Compare runs

On the Runs tab, tick two or three finished runs and select Compare. The oldest run is the baseline. You can also select Compare with run #N on a run. Runs can be compared only when they’re fair to compare: they graded the same kind of calls (simulated or live), with the same judges at the same versions, the same scorecard, and the same scenarios × tries or the same calls. The agent’s version can differ, since that’s what you’re testing. A run that isn’t fair to compare is disabled, and hovering it says why. The compare page shows:

• Scorecard change: how accuracy, experience, and the score changed from the baseline.
• By judge: how often each judge passed in each run.
• By scenario: whether each scenario passed accuracy (A) and experience (X) in each run.

Switch between Best of N and All N to change how a scenario passes.

​

Re-score a run

Select Re-score on a finished run and pick a scorecard. Each call’s verdicts are scored again with it, so the scores and bands change, but the verdicts don’t. Judges don’t run again, so re-scoring costs nothing. Re-score with the same scorecard after you edit it to apply your edits to the run.

​

Stop a run

Select Stop run on the run page, or in the run’s ⋮ menu on the Runs tab. The confirmation shows how many tries are done.

• Tries already graded keep their results.
• Tries not graded yet are discarded and not charged.