Choosing the Right Memory Structure

The last lesson ended mid-sentence on vector memory's landscape table. This lesson finishes it, adds the type that actually won by default in 2026, and closes with the type that turns a security problem from Week 2 into something structurally worse: memory shared across more than one agent.

Picking a vector store: five combinations, and why benchmark alone is the wrong axis

Vector databases do one thing well — nearest-neighbor search — and one thing very well: scale. That is a commodity, and the 2026 catalog reflects it:

For this course's labs, that means Chroma in-process — it installs in one line, runs anywhere, and the API matches what you'd use switching to a PersistentClient for production. For PyNanoClaw's production path, it means pgvector, because once an agent has multiple data needs (memory, audit logs, a user table) one database is simpler than three.

The reason none of this is decided by a leaderboard: in Rod Rivera's own production experience, retrieval-quality differences between the top five vector stores are smaller than the differences between two versions of the same prompt. That is a practitioner's call, not a published benchmark result — treat it as operating guidance, not a citable statistic. Don't pick by benchmark. Pick by operational fit and metadata-filter capability — the thing that actually determines whether your hard constraints (capacity, price, date) get enforced or silently ignored.

col.add(
    ids=["v1", "v2", "v3"],
    documents=[
        "The Bow Bar — capacity 80, too small for big groups.",
        "The Albanach — capacity 180, full vegan menu.",
        "Hemma in Holyrood — capacity 150, vegan, books out fast.",
    ],
    metadatas=[
        {"capacity": 80,  "vegan": False},
        {"capacity": 180, "vegan": True},
        {"capacity": 150, "vegan": True},
    ],
)

# Vanilla similarity search
col.query(query_texts=["vegan venue for a big group"], n_results=2)

# Same query with a metadata filter — embeddings can't enforce constraints
col.query(
    query_texts=["vegan venue for a big group"], n_results=2,
    where={"$and": [{"capacity": {"$gte": 150}}, {"vegan": True}]},
)

The metadata filter is doing 90% of the work in that second call. Embeddings handle the fuzzy semantic part; filters handle everything embeddings are structurally bad at.

File-based memory: the type that won by default

Type 9 is memory stored as markdown files on a filesystem, loaded into context by convention — no database, no embedding model, no server. CLAUDE.md in Claude Code, .claude/skills/*.md in OpenClaw and NanoClaw, /memories/*.md in Anthropic's own memory tool, .cursorrules, .windsurfrules — five different teams reached the same answer independently, because markdown is what LLMs read most fluently, files are what humans already understand, and the filesystem metaphor matches how these models have been trained to reason about tools.

The killer features are not subtle: inspectable in any editor, version-controllable with git, debuggable with grep and diff, cheap (no embedding cost, no hosting), deterministic (no false positives from cosine similarity), and portable — move the directory, the memory moves with it. Files start to break down past roughly a thousand atomic facts, under high-frequency concurrent writes, or where strict ACID guarantees are required — at which point you add a vector index next to the files, not instead of them.

The strongest evidence for this bet is Anthropic's own memory tool, shipped September 29, 2025: beta header context-management-2025-06-27, tool type memory_20250818, and exactly six operations — view, create, str_replace, insert, delete, rename — on a client-side filesystem, defaulting to a /memories directory. That minimalism is deliberate. Vendor lock-in: zero. Inspectability: total.

class FileMemory:
    """A single CLAUDE.md file with named sections you can append to."""
    def __init__(self, path: str, title: str):
        self.path = Path(path)
        if not self.path.exists():
            self.path.write_text(
                f"# {title}\n\n## User profile\n\n(empty)\n\n"
                "## Known venues\n\n(empty)\n"
            )

    def load(self) -> str:
        return self.path.read_text()

    def append(self, section: str, line: str):
        text = self.path.read_text()
        marker = f"## {section}"
        i = text.find(marker)
        if i == -1:
            text += f"\n## {section}\n\n- {line}\n"
        else:
            j = text.find("\n## ", i + len(marker))
            j = j if j != -1 else len(text)
            body = text[i + len(marker):j].replace("(empty)", "").rstrip()
            text = text[:i + len(marker)] + "\n" + body + f"\n- {line}\n\n" + text[j:]
        self.path.write_text(text)

No database, no server, no embeddings, no network call — this is persistent agent memory in about thirty lines, and for most use cases it is enough. That is not a beginner's simplification; it is where the field landed after roughly two years of building more elaborate alternatives.

Reflexive memory: the agent critiquing its own past

Type 10 is memory the agent generates about itself: it looks back at what it did, produces a critique in natural language, and stores that critique for retrieval the next time a similar task comes up. Shinn et al.'s Reflexion paper (NeurIPS 2023, arXiv:2303.11366) framed this precisely as "verbal reinforcement learning" — instead of updating weights, the agent updates its prompt with verbal lessons from past failures. The lesson is the gradient.

Why it works: failures are usually more informative than successes, and "what went wrong and how to avoid it" is exactly the kind of compact, transferable knowledge that fits comfortably in a prompt. Why it usually gets built badly: the reflection prompt is the single most important prompt in the system, and a weak one produces platitudes ("I should be more careful next time") instead of actionable rules ("for 'tonight' bookings, check lead time first — most failures came from venues needing advance notice").

Treat reflections as code, not data: version-control the prompt, test it against golden trajectories, review it when behavior drifts. This lesson introduces the pattern; the applied build — reflection actually changing a planner's output on the identical task — is lesson 26's Lab 4.

Shared memory: where the security story changes shape

Type 11 is memory accessible by more than one agent, usually with namespaces and access controls. The use cases are real: customer support spanning email, chat, and voice for one customer; parallel research subagents pooling findings; a primary agent plus a background sleep-time agent sharing state. Shared memory is what turns a swarm of independent processes into an actual team.

It is also where concurrency gets genuinely hard — last-write-wins is the default and is usually wrong for partial updates, two agents can race on a read-decide-write sequence, and stale reads happen when Agent A acts on state Agent B already invalidated.

The sharper problem is security. Men et al. (2025) documented a contagious jailbreak: in a multi-agent system with shared memory, an attack on one agent can spread to others through the shared memory layer itself. Agent A gets jailbroken, writes poisoned memories, Agent B reads them, and Agent B is compromised — without ever being directly attacked. This is a structural consequence of shared memory, not an edge case: any agent with write access can shape what every other agent believes.

class SharedMemory:
    """File-backed shared memory with write locking and source attribution."""
    def __init__(self, path: str):
        self.path = Path(path)
        self.lock = FileLock(str(self.path) + ".lock")
        if not self.path.exists():
            self.path.write_text("[]")

    def write(self, agent_id: str, content: str, trust: float = 0.5):
        with self.lock:
            data = json.loads(self.path.read_text())
            data.append({"agent_id": agent_id, "content": content,
                         "trust": trust, "ts": datetime.now(timezone.utc).isoformat()})
            self.path.write_text(json.dumps(data, indent=2))

    def read(self, min_trust: float = 0.0) -> list[dict]:
        with self.lock:
            data = json.loads(self.path.read_text())
        return [m for m in data if m["trust"] >= min_trust]

FileLock buys mutual exclusion across processes for free. Combined with source attribution and trust-level filtering — a high-stakes agent only reading from high-trust sources — that is the bare minimum for shared memory that isn't either a race condition or an open attack vector.

Context engineering and memory engineering are different questions

This is Rod Rivera's own distinction, and it's worth stating precisely because the two questions get blurred constantly. Week 1's context engineering asks: given this task right now, what goes into the model's context window, in what order, at what token cost? Memory engineering asks a different question: what persists across calls, sessions, users, and time — and how is it written, updated, deleted, and made retrievable?

Every memory, to matter at all, eventually has to become tokens in a context window. That is the seam between this week and Week 1, and it is why a context budget matters even after memory is solved. The worked allocation below is set against a 200K-token context window — the same order of magnitude as a current-generation frontier model's working window:

Prompt caching, where cached input runs at roughly a tenth the cost of fresh input, is why the ordering here is not arbitrary: stable prefixes (system prompt, tool schemas, core memory) belong at the top where they get cached; volatile content (retrieved memories, recent messages, tool results) belongs after, where it can't be cached and has to stay fresh. Memory lives in the volatile middle. Putting retrieved memories at the top breaks the cache and burns money for no quality gain — and a condensed, consolidated memory block keeps that cached prefix small, which is the other reason consolidation matters, beyond just tidiness.

What to carry into the next lesson

The taxonomy is complete: eleven types, largely orthogonal rather than mutually exclusive, each answering a different piece of "what does the agent need to remember and in what shape." What comes next is the harder half — what you actually do to what's stored. The next lesson opens the CRUD++ operations table and Lab 1's hybrid-retrieval build, where naive top-k vector search meets its actual limits against a real query set.