Quality

How Call Watch works

Call Watch checks your agents' calls and chats for problems, so you're not refreshing dashboards hoping to catch one. This page explains exactly how it works: what's checked, how often, where each number comes from, and what happens when a monitor trips.

The flow

Your customers' calls, chats & messages
            │
            ▼
   Call Watch checker ── every 2 minutes, every account
            │
            ▼
   Each monitor reads real data ── not enough data? → "No data"
            │
      threshold crossed?
       │          │
      no         yes
       │          │
   Healthy     Issue opened (or kept open — never duplicated)
                  │
                  ├─ shows on the Issues tab and in the bell at once
                  └─ goes into your nightly report (9 PM, your time)

The checker works through accounts a few at a time. If one run can't reach every account in its time, the next run starts with the ones it missed, so every account is checked within a few minutes however many there are.

Separately, every 10 minutes a second sweep analyses finished calls to produce the quality signals some monitors depend on (see Post-call analysis).

What counts

Monitors look at customer traffic only: phone calls, website chat, SMS, email, campaigns and Community conversations. Your own use is left out — dashboard tests, Co-pilot, evals, test suites, the landing-page demo and Analytics questions — so testing an agent can never raise an issue about your customers.

What gets monitored

Your account

  • Failed calls & errors — failed requests (AI replies, transcription, voice, tools) during customer conversations.
  • High error rate — the share of requests failing, not the raw count. Twenty failures out of twenty thousand is noise; out of forty it's an outage. Needs at least 20 requests.
  • Slow AI responses — reply time on phone calls, measured by the voice bridge from the caller's last word to the agent's first audio. The monitor takes the median across calls, so one slow call can't trigger a false alarm. Needs at least 5 calls that recorded it.
  • Unusually short calls — 30% or more of calls ending under your threshold. This is the signature of a greeting that didn't play, one-way audio, or callers hanging up immediately. Each call counts once, however it was billed.
  • Negative caller sentiment — how many analysed calls ended with an unhappy caller.
  • Poor call audio quality — audio frames arriving late or not at all (network jitter or packet loss). This is what a caller hears as choppy or robotic audio, and it's usually what's behind low transcription confidence.
  • Low speech recognition confidence — the transcriber was unsure what it heard. Usually poor line quality, background noise, or a strong accent — and it's the upstream cause when an agent starts mishearing callers.
  • Agent not understanding callers — the agent repeated itself or kept mishearing.
  • Booking / payment / CRM failures — the agent's tools failing: a booking that didn't save, a payment that didn't take, a CRM that rejected the write. The call still completes and the agent apologises, so nothing else reports these.
  • Monthly API budget — how much of your plan's included usage is spent. Pay-as-you-go has no allowance to measure; low-balance alerts cover it instead.
  • No agent activity — no customer calls, chats or messages for a while, often a misrouted number.
  • Provider rate limits — requests a provider refused because too many were sent at once (HTTP 429). Callers hear silence or a fallback while it lasts.
  • Timeouts — requests to the AI, voice, transcription or a tool that took too long and were cut off.
  • Provider errors — one AI, voice or transcription provider failing a share of your requests, named in the issue. Needs at least 20 requests to that provider.
  • Calls failing to connect — not measured yet. Callable doesn't record a call that fails to connect, so this monitor shows No data instead of a healthy 0%.
  • Low tool success rate — the share of tool calls failing, which a count alone can miss on a busy line. Needs at least 10 tool calls.
  • Callers hanging up early — callers who hung up before hearing the agent, measured by the voice bridge: usually a slow or missing greeting.
  • Transcription failing on calls — calls where speech-to-text errored or dropped mid-call, counted by the voice bridge; the agent may have missed what the caller said.
  • Agent voice failing on calls — calls where the voice provider failed and another had to take over. Critical when callers heard silence because none could.

Compliance (from the post-call analysis — each call's transcript is read once, for quality and compliance together):

  • Required disclosure missing — calls where the agent didn't give the disclosure you write on the Monitors tab (for example that the call is recorded), judged by meaning rather than exact words. Nothing is checked until you write one.
  • Opt-out not honoured — the caller asked not to be contacted again or to stop, and the agent kept going.
  • Claimed actions that didn't happen — the agent told the caller something was done (booked, paid, sent) while the call's own tool records show it failed and nothing succeeded.
  • Sensitive data asked for on a call — the agent asked the caller to say a full card number, security code, password, PIN or full ID number, or read one back.

These read conservatively: when the analysis is unsure, nothing is flagged. Calls that weren't analysed — very short calls, calls without a transcript, calls from before this check existed, and workspaces whose calls Callable Intelligence analyses instead — show as not measured, never as compliant. Call Watch flags possible problems for you to review; it doesn't certify that calls meet any law or regulation.

Platform health (platform admins only): server load, database latency, SSL expiry, tenant isolation and reachability of the providers Callable depends on.

Sentiment and confusion are the ones aggregates can't see. A call where the caller ends up furious, or where the agent asked the same question four times, is a perfectly "successful" call by every other measure. Only the transcript shows the problem.

Monitor status

Every monitor shows one of six states, the same on every screen:

  • Healthy — it had enough data and the reading is comfortably inside the threshold.
  • Close to threshold — enough data, and the reading is within 20% of the threshold.
  • Firing — the threshold is crossed and an issue is open.
  • No data — not enough data to judge: no customer traffic, fewer readings than the monitor needs, or calls that don't record that signal. A monitor with no data is never shown as healthy.
  • Check failed — the data couldn't be read on the last check. It tries again on the next one.
  • Off — switched off; it isn't checked and raises nothing.

Per-monitor settings

On the Monitors tab, each monitor can also set, where it makes sense:

  • Looks at — how far back it judges: the last 15 minutes, hour, 24 hours or 7 days. Rate limits, timeouts and provider errors look at the last hour by default; the rest at the last 24 hours.
  • Critical at — your own level for Critical. Leave it blank for the built-in rule; set it and an issue is Critical only at or past it, and never more than High below it.
  • Only these agents — narrow a monitor to some agents; it then sees only their calls and chats.

The background check and the Issues tab's default view (Each monitor's window) judge every monitor by its own settings. The Issues tab can also look at every monitor over one range — the last hour, 24 hours or 7 days — to explore.

Severity

Every issue carries one of four levels: Critical, High, Medium, Low. Severity comes from how far past your threshold the reading is — an error rate at double your threshold is Critical where the threshold itself is High.

Issues: created, deduplicated, auto-resolved

When a monitor trips, Call Watch opens an Issue recording what happened, how severe it is, what was measured against which threshold, which agents were involved and when it started.

  • Deduplicated. One issue per monitor. A problem that persists keeps its issue open and up to date; it is never opened twice.
  • Started and last seen. An issue records when the current occurrence started, when it last happened, and how many times it has come back. A reopened issue's age starts again from the reopen.
  • Auto-resolved. When the condition clears, the next check marks the issue resolved and stamps the time.
  • Reopened. If it happens again, the same issue reopens and its count goes up — the history stays in one place.

Working an issue

Open any issue for its full picture: what was measured against which threshold, the agents involved, the affected calls (each opens in the call inspector), the errors behind it, what to check next, and a timeline of everything that happened to it.

  • Acknowledge — you've seen it. It leaves the bell (which shows only what nobody has picked up) but stays on the Issues tab until it clears.
  • Investigating — someone is working on it.
  • Resolve — you've fixed it. It stays resolved even while the old failures are still inside the 24-hour window, and reopens only if it happens again after you resolved it. For a standing condition like a spent budget, it reopens if the condition clears and then comes back.
  • Reopen — bring a resolved issue back.

Likely cause (Callable Intelligence)

Where Callable Intelligence is available to your workspace, an issue's details offer Analyze. It gathers the issue's verified facts from your records — the measurement against its threshold, when it happened, the agents, the errors behind it, what else was open at the same time and the affected calls — and asks Callable Intelligence for the most likely cause.

  • Every claim shows the facts it rests on (F1, F2, …), and you can open the full list of facts it was given.
  • Anything not backed by those facts is shown as a hypothesis — not verified, with how to check it. If the facts don't point to a cause, it says so instead of guessing.
  • You see the cost before it runs and nothing happens until you say go ahead. The result is kept with the issue, so opening it again costs nothing; when new evidence arrives you can analyse again.
  • Call summaries are only included for roles allowed to see conversation content. Workspaces whose data must stay in Saudi Arabia can't use it.
  • When the cause is in the agent's own configuration, Create improvement proposal drafts a change for review in Callable Intelligence. Nothing changes on the agent until you approve it there.

The owner, admins and developers can change an issue's status; other members can see it. Filter the Issues tab by status to see what needs attention, what's acknowledged and what's being investigated.

The timeline and these actions need a one-time Call Watch database update. Until it's applied, issues still open and resolve on their own.

Notifications

Issues show on the Issues tab and in the dashboard bell as soon as a check finds them. Where else they go is up to you, under Call Watch → Notifiers. Add up to ten destinations — Email, SMS, Slack (an Incoming Webhook URL from Slack) or Webhook (JSON POSTed to your https endpoint, with optional Bearer token, OAuth 2.0 client-credentials and custom headers; a Discord channel webhook URL works here too).

Apps you connected under Integrations — Slack, Discord, Zapier, Make, n8n or a custom webhook — appear as extra choices under From your integrations when you add a destination. Pick one and it uses the address saved there, following it if you change it; nothing to paste. An app you haven't connected doesn't appear. Zapier, Make and n8n receive the same JSON as a webhook; a custom webhook with a signing secret gets an x-callable-signature header (HMAC-SHA256 of the body), as its call summaries do. Discord messages never mention anyone.

Each destination chooses:

  • Alert as soon as an issue opens — one alert per issue, and one more if it reopens; never repeated while it stays open. Pick the level: critical only (the default), high and critical, medium and above, or every issue. At most 20 an hour per destination; anything beyond that waits for the nightly report.
  • Nightly report — one report at 9 PM in your time zone: what's open, what came up and what cleared.
  • Only some issues — narrow it to categories, monitors or agents. Narrowed to agents, it receives only issues tied to those agents.

Webhook alerts carry the issue as JSON: event (call_watch.issue_opened, call_watch.issue_reopened, call_watch.test), and issue with its title, severity, category, monitor, measured value and threshold, agents, affected conversations and a link. An Idempotency-Key header lets your endpoint ignore a repeat.

Delivered means delivered

Delivery history lists every alert, report and test with its real status:

  • Delivered — the destination confirmed it: Slack or Discord answered that it posted, your webhook answered 2xx, or the email or SMS provider confirmed it reached the inbox or phone.
  • Sent, not yet confirmed — the email or SMS provider accepted it and hasn't confirmed delivery yet. Call Watch checks again on its next runs.
  • Failed — with the reason. Failures that might pass on a second try (a timeout, a 5xx, a rate limit) are retried, up to three times.
  • Held back — over the hourly limit; it's in the nightly report.

Send test sends through the same path as a real alert and waits a few seconds for the provider's confirmation before telling you anything.

Secrets stay on the server: a saved token, client secret, header value or Slack/Discord/webhook URL shows as "saved" and is kept unless you type a new one. Every address is checked, including where it resolves to, so a URL can't be used to reach internal infrastructure.

Several destinations, immediate alerts and delivery history need a one-time Call Watch database update. Until it's applied there is one destination, and it gets the nightly report.

Post-call analysis

Sentiment and agent-confusion can't be measured from usage counters — they only exist in what was said. Every 10 minutes Callable analyses finished calls that have a transcript and haven't been analysed yet, recording the caller's sentiment, whether the agent repeated itself, and whether it kept failing to understand the caller.

Deliberate limits:

  • Calls shorter than 15 seconds are skipped — a wrong number isn't a quality signal, and counting it would distort the percentages.
  • A bounded number of calls are analysed per sweep, so a backlog is worked through steadily rather than all at once.
  • Only analysed calls count toward a percentage, so an unanalysed backlog can never make a struggling agent look healthy.
  • When the analysis is unsure, it records neutral. Call Watch should under-report rather than cry wolf.

Overview and agent health

Call Watch → Overview is where Call Watch opens: how your agents are doing over the last hour, 24 hours, 7 days, 30 days or a custom period of up to 90 days. It reads the same call records as Analytics, so the numbers match.

  • Health — Critical when a critical issue is open, Needs attention when any other issue is open, Healthy when calls came in and nothing is open, and No data when there were no calls and no issues.
  • Calls (and all customer conversations, chats and messages included), failed calls, successful calls (analysed calls only), reply time (median, average and p95, measured by the voice bridge), tool failures, transcription confidence, audio quality and spend (with how much of this month's included usage is used).
  • Incidents in the period, split into those affecting conversations and account-wide ones (a budget, a quiet account) that can't be tied to specific calls.
  • Agents — each agent's health, calls, failed calls, reply time, tool failures and open issues. Open one for its own figures, its issues and its recent calls; each call opens in the call inspector.
  • Monitors, providers (how many of your customers' requests each provider handled without an error) and alerts sent.

Trends, under the figures, show how they moved over the period — hour by hour up to two days, day by day beyond: calls, failed calls, reply time, tool failures, provider errors, spend, incidents (filter them by category) and alerts sent. Hover or use the arrow keys for the exact value, or switch to a table. Next to them:

  • Compared with the period before — the same figures for the period of the same length just before, marked better, worse or about the same.
  • Unusual — anything in the last 24 hours far outside your last 30 days (failed calls, reply time, tool failures, provider errors, spend). It needs at least 7 days that each had enough calls, enough data today, and a large change (three standard deviations and at least 50%) — small or new accounts are never flagged on a hunch.
  • Predicted — from the third day of the month, if your usage is on course to pass this month's included amount. It's a projection, clearly labelled, not an issue.

A period with nothing measured shows as a gap, never as zero.

Every figure says how many calls it's based on and, behind its ⓘ, what it measures. Your own dashboard test calls are left out. When nothing was measured a figure says No data — never zero.

The Call Watch page

Open Quality → Call Watch. The header shows a live status for the last 24 hours — All clear, or how many issues are open and how bad the worst is — and each tab carries a count.

Issues lists open issues worst first, each with its severity, category, what was measured against which threshold, the agents involved, the conversations it affected and a link to the monitor that raised it. Filter by severity, category or agent, and switch between the last hour, 24 hours or 7 days. The page checks again every minute while it's open.

Conversations affected counts each customer conversation once, however many issues touched it. Some issues can't be tied to specific conversations — a budget nearly spent, a quiet account, a platform problem — and those say so instead of claiming zero.

If some monitors have no data in the window you're looking at, the page says how many: "no issues" means less when a monitor couldn't look.

Tuning it

Monitors groups every monitor by category, with its status, what it measured just now against its threshold, how many readings that's based on, where its data comes from and exactly what raises an issue. Switch each on or off and set its threshold within the allowed range shown under the box; a value outside it can't be saved. Reset to default puts a threshold back.

Monitor settings belong to the workspace: the owner, admins and developers can change them, and everyone in the workspace sees the same monitors and issues. The next check uses new settings.

Notifiers lists your destinations with what each receives, a Send test button and the delivery history. Slack, Discord and webhook addresses show only their host or "Connected …", since the full address contains a secret.

Next steps