Why Agents Fail: Harness Engineering for AI Projects
The demo worked in the meeting. Three months later the project is dead, and it wasn't the model's fault. This course is about the gap in between: if you fund or greenlight AI projects, five plain-language questions that expose a weak one before you commit budget, and what a project actually worth backing looks like. If you build them, the shared vocabulary and failure taxonomy behind harness engineering, worked through against four real architectures — research agents, workflow agents, decision-support agents, and a named case study on Rasa's own policy-enforced CALM system. The capstone is a nineteen-item failure catalogue and a five-hour exercise: name your own project's archetype and build a working harness skeleton for it, a manifest schema, a deterministic verifier, a human-in-the-loop gate, a real defense against one catalogued failure. Finishing it in five hours is the proof the discipline transferred, not just that the lesson got read. Migrated from a funder-facing keynote and a three-hour practitioner workshop.
MethodThis course teaches a discipline. The examples use today's tools; the method is meant to outlast them.
Rod Rivera
Professor
Desk
First lesson is free to preview.
What You'll Learn
- Why the demo-to-production gap in AI agent projects is operational, not a model-capability problem
- Five plain-language questions that expose a weak AI project before you approve the budget
- What a well-run AI agent project's staged-autonomy rollout actually looks like
- The shared vocabulary and six-category failure taxonomy behind harness engineering
- The five-component architecture behind research and synthesis agents, with a real manifest schema and verifier
- The two-halves split and session-isolation discipline behind process and workflow agents
- Why a confidence score in a decision-support agent is a legal promise, and how calibration drift is caught before a regulator finds it
- The architecture behind policy-enforced conversational agents, credited to Rasa's own CALM system by name
- How to name your own project's archetype and sketch a harness skeleton in five hours
Prerequisites
- No programming experience required for lessons 01-03 (the executive's diagnostic)
- Comfort reading YAML and simple Python for lessons 04-09 (the engineer's path)
- No prior AI agent building experience required — every architecture is worked from first principles
Syllabus
Ready to keep going?
The lessons in Why Agents Fail: Harness Engineering for AI Projects are open. The Prof Rod weekly lesson is the next thing, not a door.
