# Rehearsals

> Quality · Callable Docs · https://docs.callableio.com/rehearsals

Rehearsals let a simulated customer talk to your real agent from start to finish, then grade how it went. Use them to test whole conversations — a booking, a cancellation, an angry caller — before your customers have them.

Open **Quality → Rehearsals**. A rehearsal is a set of tests for one agent.

## Spot Checks or Rehearsals?

- **Spot Checks** script the caller's exact lines and check each answer. They're fast, and best for "does it still say the right thing?". See [Spot Checks](https://docs.callableio.com/evals.md).
- **Rehearsals** give a simulated customer a goal and a personality, and let it talk to the agent until the goal is met. It answers whatever your agent actually says — it doesn't replay a script. You can run each test many times, in chat or voice, and compare runs.

Rehearsals run on the server, so a run keeps going if you leave the page or close the tab.

## Create a rehearsal

Click **+ Create Rehearsal** and fill in:

- **Name** and an optional **Description** — what this rehearsal protects.
- **Target agent** — the real agent under test. Runs use its live configuration: prompt, tools, knowledge and workflow.
- **Import this agent's existing spot checks** — shown when the agent already has spot checks. Copies them in as tests.
- **Write and run tests automatically** — on by default: Callable writes tests from your agent and runs them for you. Turn it off to add and run tests yourself.
- **Testing mode** — **Chat** (text only: fast and cheap, the best way to iterate on prompts, tools and knowledge) or **Voice** (the full phone pipeline: speech-to-text, text-to-speech, turn-taking, interruptions, silence and voice latency, with a recording).

New rehearsals run in **Sandbox**: tools that change something outside Callable (bookings, texts, webhooks) aren't executed unless you mock them or choose **Live** for a run.

## The tester

The tester is the simulated customer. Each test gives it a scenario (what it wants and when to stop) and a personality (how it behaves).

- **Basic tester** — sensible defaults: English, natural replies.
- **Custom tester** — set the model, system instructions, language, first message, max turns and max duration yourself, plus voice and transcriber settings for voice mode.

Pick a **Default personality** — Friendly, Impatient, Confused, Angry, Skeptical, Talkative, Quiet, Interrupting, Elderly or Professional — or **Add personality** to write your own. The default is used when a test doesn't pick one.

## Tests

Click **✨ Generate Tests** and Callable reads the agent's real configuration — prompt, first message, tools, knowledge, workflow, guardrails and transfer rules — and proposes tests across happy paths, variations, missing information, ambiguity, error recovery, edge cases, guardrails and transfers. Add an optional focus, such as *cancellations and rescheduling*. You review every test before anything is saved.

Or click **+ Add Test** and write one:

- **Full conversation** — a simulated customer talks to the agent until the goal is met.
- **Single decision** — you write the conversation so far, ending with a customer message, and only the agent's reply to it is graded.

Each test needs at least one success criterion — without one it reports an error, never a pass. Add an **AI judge**, **Text check**, **Flow check** or **Structured output** criterion, and mark each one required (a failure fails the test) or optional (reported separately). You can also mock a tool's response, so the test never touches your real systems.

Click **Regression** on a test (**Mark as regression test**) to add it to your regression set.

## Run tests

Click **▶ Run Tests** and choose:

- **Tests** — **All** or **Selected**.
- **Mode** — Chat or Voice.
- **Iterations** — run each test 1, 5, 10 or 25 times (or a custom number up to 50) to see how consistent the agent is.
- **Environment** — **Sandbox** (side-effect tools aren't executed unless mocked; lookups and transfers still run) or **Live** (every unmocked tool runs for real against your connected systems). Live asks you to confirm that the run can perform real actions: payments, bookings, messages and record changes.
- **Concurrency** — how many tests run at the same time: 1, 5, 10, or a custom number up to 10.

Each voice test is a real call to your agent over a private test line: the tester speaks, the agent answers in its own voice, and the call is recorded. It takes as long as the call does and uses voice and speech-recognition credits like a phone call.

To run only your regression set, open the **⋯** menu and choose **Run regression**. Results are compared with the previous completed run.

## Runs

**▶ All runs** lists every run across your rehearsals. Open one for the live view: each test's transcript and tool calls appear as they happen, with the pass rate, average latency, cost and errors.

- **Compare with previous** — new failures, and any test that changed result or behaviour since the previous run.
- **Consistency across iterations** — when tests ran more than once, how consistently each one passed.
- **Cancel run** — stops a run that's still going.

Expectations are never adjusted to make a run pass — a changed result means the agent changed.

## Results

Open a test's result to read the transcript, the timeline, every tool call, each criterion's verdict and, for a failure, the failure analysis. Voice tests include the recording.

- **Re-grade with current criteria** — grades the same conversation against the test's current criteria, without re-running the agent. It's a preview and isn't saved.
- **Rerun test** — runs this test again.
- **Create fix task** — files the failure as an open issue in [Call Watch](https://docs.callableio.com/monitoring.md), with the analysis and a link back to the result.

The agent is never changed automatically.

## Who can do what

Admins and developers create, edit and run rehearsals. Members can view them.

## Costs

Rehearsals are included on every plan. A run uses AI for your agent and for the tester, and voice tests use voice minutes; all of it is billed like any other usage.

## Next steps

- [Spot Checks](https://docs.callableio.com/evals.md) — scripted checks of single answers
- [How Call Watch works](https://docs.callableio.com/monitoring.md) — watch real calls once you ship
- [Callable Intelligence](https://docs.callableio.com/callable-intelligence.md) — tests suggested fixes before they go live
