> ## Documentation Index > Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt > Use this file to discover all available pages before exploring further. # Evaluations > Score your agent on accuracy and experience, and see why calls fail. Evaluations grade your agent's calls. Judges check each call against one question, such as *"Did the agent confirm the booking?"*, and a scorecard turns their verdicts into a score. You can grade scenarios played by a simulated caller, a simulation run that already finished, or real calls your agent took, and see which judge failed and why. Open **Evals** in the sidebar. Evaluations score one agent at a time: pick the agent with the agent picker in the page header. The page has three tabs: **Overview**, **Judges**, and **Runs**. ![Evals page with the agent picker, New judge and Run evaluation buttons, the Overview, Judges, and Runs tabs, the How evaluations work cards for Accuracy, Experience, and Diagnostic, and the Set up evaluations checklist with two of three steps done](https://cdn.zeroruntime.ai/zrt/docs-resource/agent-builder/evals-overview-setup.png) ## Core concepts | Concept | What it is | | - | - | | **Judge** | Answers one question about every call and gives a verdict: pass, fail, or can't tell, with its reason. See [Judges](https://docs.zeroruntime.ai/agent-builder/evaluations/judges.md). | | **Scorecard** | Turns a call's verdicts into a score, such as 82 out of 100, and a band, such as *Good*. See [Scorecards](https://docs.zeroruntime.ai/agent-builder/evaluations/scorecards.md). | | **Run** | One evaluation: a set of calls graded by the judges you picked. Each run has a number, for example `Run #4`. | | **Try** | One graded call in a run. When you grade scenarios, each scenario can be played several times, and each play is a try. | | **Auto run** | Grading that happens without you starting a run, on the agent's live calls, its simulation runs, or both. See [Grade calls automatically](https://docs.zeroruntime.ai/agent-builder/evaluations/run-an-evaluation.md#grade-calls-automatically). | ## Judge categories Every judge has a category. The category decides whether the judge counts toward the score. | Category | Question it answers | Scored | | - | - | - | | **A · Accuracy** | Did it do the right thing? The goal was reached, the facts were right, tools were used correctly, and the agent stayed within policy. | Yes | | **X · Experience** | Did it feel right on the phone? Short, natural replies, good turn-taking, and a clear close. | Yes | | **D · Diagnostic** | Why did it miss? Misheard words, tool errors, and slow replies. | No. A failed diagnostic judge shows up as a flag that points to the fix. | ## Verdicts | Verdict | Meaning | | - | - | | **Pass** | The call meets the judge's pass criteria. | | **Fail** | The call meets the judge's fail criteria. | | **Can't tell** | The judge's topic never came up in the call. It counts as not passed in a run's pass rates. | | **Judge error** | The judge couldn't grade the call, for example because the recording is missing. It isn't counted. | Each try also gets an overall status: | Status | When | | - | - | | **Passed** | No accuracy or experience judge failed. | | **Needs review** | An accuracy or experience judge failed, or a judge errored. | | **Failed** | **Goal Completion**, **Response Accuracy**, or **Safety & Compliance** failed. | | **Didn't count** | The call was skipped, for example because it was too short. Skipped tries are left out of the scores, never counted as failed. | ## How a run is scored A try passes a category when every judge in that category passes. A run's **Accuracy** is the share of scenarios whose tries passed every accuracy judge, and **Experience** is the same for experience judges. When you grade live calls, each call counts as its own scenario. When each scenario is played more than once, use the toggle above the scores to choose how a scenario passes: | Mode | A scenario passes when | | - | - | | **Best of N tries** | Any of its tries passed. Shows whether the agent *can* handle the scenario. | | **All N tries** | Every one of its tries passed. Shows whether it handles the scenario *reliably*. | Separately, the scorecard gives every graded call a score and a band. The run shows the average score, the median, and how many calls landed in each band. See [Scorecards](https://docs.zeroruntime.ai/agent-builder/evaluations/scorecards.md). ## Before you start * **Create an agent.** Evaluations need at least one agent on your account. * **Have something to grade**: [scenarios](https://docs.zeroruntime.ai/agent-builder/simulation/scenarios.md) in Simulation, a finished [simulation run](https://docs.zeroruntime.ai/agent-builder/simulation/run-a-simulation.md), or live calls to the agent. * **Have enough credits.** AI and audio judges are billed as model usage for each call they read. ## Set up evaluations Until your first run finishes, the **Overview** tab shows a short checklist: **1. Add judges** Use the default judges, or write your own for what a good call looks like. See [Judges](https://docs.zeroruntime.ai/agent-builder/evaluations/judges.md). **2. Add scenarios** Scenarios come from Simulation. Write a few callers your agent must handle. See [Scenarios](https://docs.zeroruntime.ai/agent-builder/simulation/scenarios.md). You can skip this step if you only want to grade a past simulation run or live calls. **3. Run your first evaluation** Select **Run evaluation**, pick what to grade and the judges, and start the run. See [Run an evaluation](https://docs.zeroruntime.ai/agent-builder/evaluations/run-an-evaluation.md). After your first run, the **Overview** tab shows the latest scores, the judges that failed most, and your recent runs. See [Review results](https://docs.zeroruntime.ai/agent-builder/evaluations/review-results.md). > [!TIP] > > Prefer the terminal? `zeroruntime evals` uses the same judges, scorecards, and runs. See [Evaluations in the CLI](https://docs.zeroruntime.ai/runtime-sdk/deployments/evaluations.md). > **[Judges](https://docs.zeroruntime.ai/agent-builder/evaluations/judges.md)** > > Write the questions every call is graded on. > **[Scorecards](https://docs.zeroruntime.ai/agent-builder/evaluations/scorecards.md)** > > Weigh judges and set score bands. > **[Run an evaluation](https://docs.zeroruntime.ai/agent-builder/evaluations/run-an-evaluation.md)** > > Grade scenarios, a simulation run, or live calls. > **[Review results](https://docs.zeroruntime.ai/agent-builder/evaluations/review-results.md)** > > See which judge failed, on which call, and why. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.