Failure 03: The System Disagrees With Itself
Prerequisite: Lesson 04 ("Failure 02: Tools That Look Like Backchannels") and its two-tier interrupt classifier — the word-count fast path plus small-LLM check that decides, in production at roughly 95% accuracy, whether a user's speech is a real interrupt or a backchannel like "mm-hmm." After this lesson, you can: design a cancellation path that stops every in-flight operation (LLM stream, TTS stream, tool call) the instant an interrupt is confirmed, instead of leaving stale work to finish and speak itself into a live call.
Lesson 04 answered one question: was that "yes" a real interrupt, or just the user keeping the bot talking? Assume the classifier did its job and said interrupt, confirmed. This lesson starts one step later, with a harder question underneath it: confirming an interrupt and acting on it are not the same event, and the gap between them is where failure 03 lives.
A voice agent is not one system. It's at least three.
The voice channel, the dialogue loop, and whatever tool call is currently in flight are three separate pieces of running code, and nothing forces them to agree about whether the conversation is still active. Most engineers build "handle interruptions" as if it were a single feature, a callback to wire up, an event to catch. It isn't. It's the same class of problem as cancelling a job in any distributed system, and it shows up here specifically because frameworks built for turn-based chat have no primitive for it at all. In chat, nothing is "running" between turns. In voice, something is almost always running.
Here is what that gap looks like when nobody has closed it.
The symptom isn't the diagnosis
The user experiences a broken bot talking over them. But that's the symptom, not the diagnosis. What's actually wrong is that the architecture is confused, not the model: the voice channel knew about the "stop" at 0.8 seconds; the dialogue loop didn't find out until the tool call it was waiting on returned, 800 milliseconds later, by which point it dutifully generated and spoke a reply to a question the user had already retracted. The LLM did nothing wrong. It answered the question it was given. Nobody told it the question was withdrawn.
The fix is not a feature. It's cooperative cancellation.
Borrow the pattern distributed systems already use for cancelling work that's in flight: cooperative cancellation. Every long-running operation (the LLM stream, the TTS stream, the tool call) checks a shared cancellation token at its own next safe checkpoint. Barge-in flips the token. Nothing gets forcibly killed mid-operation; each component notices the flip on its own schedule and bails cleanly at the next point where bailing is safe. That's the whole mechanism: one shared signal, checked cooperatively, everywhere that matters.
Treating interruption handling as something you bolt on to the dialogue loop alone. The tool call and the TTS stream need to observe the same cancellation signal, or you've only closed part of the race: the part furthest from the user's ear is usually the part that gets forgotten.
If your framework was built for turn-based chat, this is not a setting you flip. It's a primitive you retrofit, because chat frameworks were never asked to reason about "what's still running" — there was never anything running between turns to ask about.
A reference demo built for this course runs the same agent twice, same model, same tool call, same "stop":
Say "stop" and the stale reply still plays, roughly a second and a half after you said it. You can hear the bot talk over you.
The exit is clean, in roughly 100 milliseconds, and the user gets the floor back.
The only difference between the two passes is whether the three systems were ever told to agree.
What to build
A cancellation token lives somewhere all three components can read it. None of them needs to know about the others — each just needs to ask "has this been cancelled?" often enough that the answer arrives before it does something the user didn't ask for anymore:
| Component | Checks the token |
|---|---|
| Voice channel | Sets the token the instant barge-in is confirmed |
| LLM stream | Between tokens |
| Tool-call executor | Between steps |
| TTS stream | Between chunks |
Cooperative cancellation stops the agent from talking past a "stop" it already heard. The next failure is the mirror case: the agent deciding, on its own, that the conversation is over.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
