Almost everyone has been stuck in a phone system that wouldn't let them talk to a person. You say "agent," it asks you to describe your problem. You describe it, it offers a menu. You start pressing zero.
That experience is why I think the handoff is one of the most important parts of any voice agent. At WiZR I led the work on real-time conversational voice agents, and we spent a lot of time balancing prompt logic with how the conversation actually felt. The point where the agent stops and a person takes over was one of the moments that shaped how people felt about the whole product.
I've written about turn-taking and latency separately. This post is about knowing when to step aside.
Decide when to escalate
Escalation triggers usually fall into a few groups:
| Trigger | Example | Why it matters |
|---|---|---|
| User asks for a person | "Can I talk to someone?" | Ignoring this destroys trust fast |
| Repeated failure | Agent misunderstands twice in a row | The conversation is going nowhere |
| Out of scope | Request the agent isn't designed for | Guessing makes it worse |
| Sensitive topic | Complaint, distress, legal or safety issue | Needs human judgement |
| High-value or high-risk action | Cancelling a contract, large changes | Mistakes are expensive |
| Low confidence | Agent isn't sure what was meant | Better to hand over than act wrongly |
The first one deserves special care. If someone asks for a person, the agent can offer to help briefly if it really can, but it shouldn't block them. Making people fight for a human is the fastest way to make them hate the agent.
Pass the context along
The worst handoff is one where the person who picks up asks the caller to start from the beginning. Everything the agent learned should be available to them:
- Who the caller is, if identified.
- What they asked for.
- What the agent already tried or answered.
- Why the handoff happened.
A short summary is better than a full transcript. The person taking over needs to understand the situation in a few seconds, not read through a long conversation.
Tell the caller what's happening
The caller should know they're being transferred, roughly how long it might take, and that their information is being passed on. Something like: "I'll connect you to someone who can help with this. I've passed on what you've told me so you won't need to repeat it."
If there's a wait, say so honestly. If no one is available, offer a callback or another channel instead of leaving them on hold indefinitely.
Handle "no one is available"
Escalation assumes there's someone to escalate to. Outside working hours, or during busy periods, that may not be true. Decide in advance what happens:
- Offer a callback with a realistic time.
- Take a message and confirm it was recorded.
- Point to a different channel if it's genuinely faster.
Each of these needs to actually work. A callback promise nobody follows up on is worse than no promise.
Measure the handoff
Useful metrics:
- Escalation rate, broken down by reason. Too high might mean the agent isn't capable enough. Too low, combined with poor satisfaction, might mean it's holding on too long.
- Repeat information rate. How often the human has to ask for things the agent already collected.
- Resolution after handoff. Whether escalated calls actually get resolved.
- Time to human for callers who ask for one.
Listen to a sample of escalated calls regularly, with appropriate consent and data handling. You'll learn more from twenty real handoffs than from any dashboard.
The agent doesn't need to win every call
There's a temptation to measure voice agents by how many calls they handle without a person. That number matters, but pushing it too hard leads to agents that won't let go.
I'd rather the agent handled the calls it's good at very well and passed on the rest quickly and gracefully. Callers remember being stuck far longer than they remember being helped.