Architectural Hindsight for Production
Sometimes an agent finishes a task, reports its work in plain text instead of calling the framework's formal "I'm done" signal, and the framework never marks the session complete — even though the task genuinely succeeded. That's a real bug sovereign-agent v0.1.0, the teaching framework this course builds, shipped with. It wasn't missed: the debugging in the previous two lessons surfaced this gap and two others, in detail, with working task-prompt workarounds already in hand. None of the three got fixed for this version. Shipping the workarounds first, and generalizing later, was the correct sequencing — here's why.
Prerequisite: Lesson 31 — Live demo and dataflow integrity, which established the structural-versus-semantic distinction this lesson's three fixes build directly on. After this lesson, you can: explain why shipping a framework with named, documented gaps can be the correct engineering call rather than a shortcut, and list the concrete additions — isolation, credential scoping, a persistent queue, observability, cost control — a teaching framework needs before it's a production one.
Three gaps, left in on purpose
Yesterday's debugging surfaced three legitimate framework limitations, each now first on the v0.2 roadmap with a name and a sketch.
Gap 1 — the planner assigns to halves that don't exist. The default planner prompt lists "loop" and "structured" as options regardless of what's actually wired into the scenario. Models take it at face value.
Gap 2 — no per-tool argument validation. A scenario has no way to say "reject a call to analyze_workspace_file where the path doesn't exist" — the tool has to catch that itself, after the call has already consumed a turn.
Gap 3 — complete_task sometimes gets skipped. This is the gap from the top of this lesson: models occasionally emit a final text response summarizing what they did instead of calling the formal completion sentinel, and there's no auto-close mechanism to catch it.
class LoopHalf:
def __init__(self, planner, executor, has_structured=False):
self.planner = planner
self.has_structured = has_structured
async def run(self, session, input):
halves = ["loop", "structured"] if self.has_structured else ["loop"]
subgoals = await self.planner.plan(
task=input["task"],
available_halves=halves,
)Only include "structured" as a valid assigned_half in the planner's prompt if StructuredHalf is actually wired into the scenario — makes the planner physically unable to assign to a half that doesn't exist, the same move as Decision 8's explicit tool registry, applied to halves instead of tools.
Why these three stayed unfixed for v0.1.0
Because the task-prompt workarounds already solved the immediate problem, and v0.1.0's actual job was getting three scenarios working end-to-end — not building the most general possible framework.
The bar for promoting a workaround to a framework feature is that multiple scenarios independently need the fix. Each of A, B, and C clears that bar — at least two of the three shipped scenarios hit each gap. So they go on the roadmap. But fixing them before shipping would have been premature, because the workarounds are exactly what taught the team what the eventual framework API should look like.
Extract a framework abstraction after you've written it three times, not before.
All three fixes share a shape worth naming: they each make the framework more aware of a scenario's semantics. Fix A gives the planner awareness of which halves are wired in. Fix B gives tools awareness of what valid arguments look like. Fix C gives the executor awareness of when a goal is already satisfied. That's the direction mature agent frameworks move in — v0.1.0 guarantees structural properties; v0.2 starts enforcing semantic properties, but only the ones a scenario explicitly declares. The dataflow integrity checks from the previous lesson are the working prototype for that generalization.
What a teaching framework skips, and production can't
Sovereign-agent v0.1.0 is honest about being a teaching framework. Putting it in production means adding, at minimum, three things deliberately left out.
Every addition in that chain reuses a decision the baseline already made rather than bolting on something new — container isolation reuses the session directory's filesystem IPC (Decision 5), the persistent queue reuses the session directory again, credential scoping reuses the same per-tool registry pattern Decision 8 introduced for tools themselves. None of this is a rewrite. It's the same architecture, hardened.
Container isolation. If an LLM-driven tool call runs shutil.rmtree("/"), an unsandboxed host disappears. The fix: run tools in a container with only the session directory bind-mounted, network disabled:
services:
executor:
image: sovereign-agent/executor:latest
volumes:
- ./sessions/sess_xxx:/workspace/session:rw
read_only: true
tmpfs: /tmp
network_mode: noneA tool can trash its own workspace. It can't touch the host. Filesystem IPC — Decision 5 from the architecture lesson — is already there, which means half the work for this is already done.
Credential scoping. If every tool shares access to every credential, one compromised tool leaks everything. The fix is per-tool scoping: send_email gets the SMTP credentials, web_search gets the search API key, and neither sees the other's:
reg.register(_RegisteredTool(
name="send_email",
credentials=["SMTP_HOST", "SMTP_USER", "SMTP_PASS"],
fn=send_email,
...
))The stub for this lives in orchestrator/credentials.py, not yet wired up — a named stretch goal for v0.2, not a claim that it's done.
Persistent queue. If the orchestrator crashes, an in-memory queue of pending sessions is gone. The fix leans on Decision 1 again: the session directories are the queue. On startup, scan for any session in a non-terminal state and re-enqueue it — the filesystem already has everything needed to recover, because nothing about a session's state lived anywhere else.
Two things round this out without needing their own sketch: observability — logs/trace.jsonl is the raw material; production wants LLM-as-judge scoring of trajectory quality, cost tracking per call, latency percentiles, and a failure taxonomy on top of it, consumed by a stack like Evidently, Langfuse, or LangSmith. And prompt injection, which is a real, not hypothetical, attack surface once tool outputs can contain adversarial text an LLM interprets as an instruction. Per-tool credential scoping bounds the blast radius; sovereign-agent's structured half is a genuine defense here too, because a deterministic rule can't be talked into anything.
The debugging day behind this whole arc ran roughly 200 real LLM calls across six hours on Qwen3-235B and MiniMax-M2.5 via Nebius — about $4. That's nothing for development. But each scenario averages 4-6 LLM calls, and a production system running 1,000 scenarios a day at that rate is roughly 5,000 LLM calls a day — about $20/day, close to $7k/year at that volume. Caching planner outputs for identical tasks, matching model size to role instead of using the largest model everywhere, and failing fast instead of retrying three times at 4x cost all cut into that number directly. The offline scripted mode this framework ships with is a cost-saving measure as much as a testing convenience.
What to carry forward
You now have the full picture: the architecture, the failures that forced it, a clean live demo, and an honest account of what's still missing before this could run in production. The last lesson in this arc closes the loop — the seven rules worth carrying to your next job, and what to actually do with the five weeks ahead of you.
The seven rules, restated once, and the homework that asks you to build on sovereign-agent rather than rebuild it from scratch.
Reply here and it goes straight to Rod. Same as replying to one of his emails.
