The Thesis, the Landscape, and the Six Failures
Prerequisite: Lesson 01 (the two numbers). This lesson assumes you already feel why a serial cascaded pipeline structurally can't hit human turn-taking timing, and builds the vocabulary on top of that feeling. After this lesson, you can: state the one-sentence reason voice is a different engineering problem than text, place cascaded and speech-to-speech architectures correctly for a regulated-industry decision, and name all six failures this course diagnoses, in order.
Lesson 01 gave you two numbers and a feeling: something about the rhythm of a cascaded voice pipeline is wrong, even when every component is fast. This lesson names what's actually going on, draws the map of where the industry stands in 2026, and hands you the six-failure list the rest of the course walks one at a time.
The single most useful idea in this course
Voice is not text with audio glued on top. It's a different engineering problem, and the reason is structural:
| Text | Voice | |
|---|---|---|
| Turn boundary | Explicit: the user hits send | Inferred, from silence, prosody, and context |
| Certainty | The system knows exactly when the turn ended | The inference can be wrong in ways a keystroke never is |
| Reversibility | A draft can be silently edited or deleted | Once spoken, irreversible; the caller already heard it |
Silence is a signal your system has to interpret, not a boundary it can trust.
Every failure this course teaches is a consequence of that one missing send button.
What shipped, and what didn't
By 2026, it's easy to assume voice-agent quality is bottlenecked by component quality: that ASR still mishears people, that TTS still sounds robotic, that the model isn't smart enough yet. That assumption is out of date.
| Layer | Status by 2026 |
|---|---|
| ASR (speech-to-text) | Production-grade |
| TTS (text-to-speech) | Human-indistinguishable on short prompts |
| Component latency | Sub-second, table stakes, not a differentiator |
| Orchestrating all three under real conversational pressure (turn-taking, cancellation, shared state, compliance gates) | Not shipped. The components work; the seams between them don't, reliably, by default |
That reframe matters because it tells you where your leverage actually is. If the bottleneck were component quality, the fix would be waiting for a better model or a better ASR vendor, something outside your control. Since the bottleneck is the orchestration layer, the fix is something you build, own, and can test. The six failures in this course are not model failures. They're orchestration-layer gaps that sit squarely in the engineer's seat, not the vendor's.
Cascaded vs. speech-to-speech: already covered, credited here
There are two architectural camps for building a voice agent, and you may already know the comparison: cascaded (STT → LLM → TTS, chained) against speech-to-speech (audio in, audio out, one model). This course's sibling course, tools-memory-and-multi-agent-systems, walks this comparison in full in its lesson Multi-Model and Voice Agent Architecture, including the same core argument this lesson would otherwise have to re-derive: a cascaded pipeline gives you a full text transcript at every stage, so you can block, rewrite, route, or log at each hop, at the cost of paying the round-trip three times; speech-to-speech is lower-latency and preserves prosody and native interruption, but there's no intermediate text to inspect, which is exactly why compliance-bound systems can't use it as a black box. If you haven't read that lesson, it's the fuller treatment; this lesson doesn't repeat it, only orients against it.
What's worth adding here, rather than re-arguing, is where the industry stands as of 2026.
Coval's own figure: speech-to-speech enterprise adoption is still under 15% through the first half of 2026. For regulated industries like banking and healthcare, cascaded remains the answer, for the same auditability reason named above: compliance requires a text layer you can gate, and speech-to-speech doesn't have one to offer yet.
What has changed since the comparison above was first made is the component layer, not the architecture choice. Two named 2026 advances are narrowing the latency gap that used to be cascaded's main weakness: Deepgram Flux, a conversational STT model with turn detection around 260ms, and Cartesia Sonic 3, a TTS model synthesizing in roughly 40-90ms. Neither of those changes the auditability argument. They just mean cascaded's latency tax is shrinking, which makes "cascaded because compliance" a decision with a smaller cost attached than it had a year earlier.
The patterns this course teaches are cascade-oriented, for the same reason cascaded is still the regulated-industry default. Most of what you'll learn translates conceptually to speech-to-speech, but the gating story for speech-to-speech systems is still being written industry-wide. That's a different course.
Six failures, five patterns
Here's the map for the rest of this course. Each failure below is a specific, named way a cascaded voice pipeline breaks under real conversational pressure: not a model limitation, an orchestration gap.
Turn-taking: the agent commits to a transcript before the caller has actually finished. Backchannels: the same short word ("yes," "mm-hmm") means opposite things depending on what the agent was doing when it heard it. Split state: the three components lose sync on what's actually happened in the call. Premature goodbye: the agent ends the call before the caller's actual need is resolved. Spoken hallucination: the agent commits to saying a wrong number or fact out loud, and unlike a chat UI, there's no way to quietly edit it after the fact. Handoff amnesia: a transfer to another system or human loses the context the caller already gave.
None of it is theoretical
Each of these maps to an orchestration pattern you control, not a smarter model, not a different vendor. And none of it is theoretical: everything in this course runs against a public reference stack (Deepgram for STT, Rime for TTS, a small LLM in the middle), so every failure you'll see over the next several lessons, you can also reproduce and fix yourself.
Lesson 03 starts with the first one: turn-taking, and why "just tune the silence threshold" doesn't fix it.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
