Architecture
A memory architecture for AI agents
The core principle
Not all memory should load the same way. Stable rules load every session. A lean index loads every session but only points at detail. The detail itself loads only when relevant. Separating these keeps the always-on cost low while keeping deep knowledge reachable.
The layers
| Layer | Loads | Use for |
|---|---|---|
| Rules / conventions | Fully, every session, under a measured ceiling | Stable "always / never" instructions |
| Index | Every session, capped | One-line pointers to topic files |
| Topic files | On demand | Detailed knowledge, one topic each |
| Project context | Per project | Who is involved, what was decided, how the pieces fit |
| Task files | On demand; written through a tool | What is open, one file per task |
The rules that keep it working
The index is an index, not content
The always-loaded index should hold one-line pointers to topic files, not the knowledge itself. Multi-line blocks belong in topic files. Every line of content you jam into the index spends the budget that should point at ten other things.
Cap the index and leave headroom
There's a practical ceiling on how much loads every session. Target well under it. Many systems truncate silently past the cap. You won't get a warning; the agent just stops seeing the newest entries. Put high-value pointers near the top.
Descriptions drive retrieval
Each topic file needs a clear description in its header. Agents typically scan metadata first, then load a handful of relevant files. Vague descriptions mean the right file never gets pulled. Write the description for the future search, not for yourself today.
Merge before you add, and keep a file budget
Keep topic files focused, but prefer adding to an existing file over starting a new one. Hold the total to a budget; I use about 45 topic files and check the count by hand. Over budget means merge or delete before you add. Every new file is one more description the agent has to scan and one more place a fact can hide.
Derive from source, don't duplicate it
Don't store what the code, the file tree, or version history already records. Stale memory is worse than no memory: the agent trusts a remembered path or value that has since changed. If it's derivable from live state, let it be derived.
Put a ceiling on the always-loaded layer
The rules layer loads in full every session, and it grows a few hundred bytes at a time. Nobody notices that by eye. Give the layer a byte ceiling and let a script measure it. Count everything that rides along in every request, including the descriptions of every skill and any tool definitions that load up front.
Open work lives in task files
A project's context file holds context: who's involved, what was decided, how the pieces fit together. What's open lives in task files, one per task, written through a small tool instead of by hand so dates and logs stay honest.
The list of open items is generated from those files and never typed. A typed list is out of date the moment something closes, and it keeps claiming the work is open long after it shipped.
Fix the index line in the same edit
When you correct a topic file, correct its line in the index at the same time. The index loads every session and the topic file loads only when something pulls it in, so the stale one-liner is what the agent actually sees. It reads as settled fact and wins over the corrected file nobody opened.
Common mistakes
Don't: Content in the index instead of topic files
Do: Index holds one-line pointers only
Don't: Rules living in the memory index
Do: Instructions go in the always-loaded rules layer
Don't: No rotation plan (silent truncation)
Do: Cap the index, keep headroom, prune regularly
Don't: Trusting stale memory over live state
Do: Verify paths/values against the source before acting
Don't: Orphaned topic files with no pointer
Do: Every topic file gets an index entry, or it's never found
Don't: Correcting a topic file and leaving its index line stale
Do: Fix both in the same edit
Don't: A new topic file for every new fact
Do: Merge into an existing file and stay under the file budget
Don't: An action-item checklist in the project file
Do: Task files hold what is open; the project file holds context
Don't: Same fact duplicated across layers
Do: One home per fact
Updated October 2026
← Back to Tools & Resources