saved
Agent memory as a file format
gist
Cal Paterson argues agent memory should be a portable file format, not a harness-tied pipeline. A memoryfield is a zip of Markdown pages with optional YAML frontmatter and an optional SQLite vector index. Agents write short prose pages themselves, then jump to relevant pages with semantic search instead of walking a knowledge graph. Low mechanism lets models invent access patterns in bash, Markdown, and SQLite, so the format scales with the frontier and stays interchangeable across harnesses.
ideas
- Memory is data, not a process. Lab-tied mining, graph-plus-vector stacks, and distilled-fact stores all put machinery between the agent and what it needs to remember.
- Write prose pages, not chunks. The agent authors Markdown memories itself, so they do not need RAG-style chunking; keep each page under about 8kb so it still embeds.
- Jump by content, do not walk the graph. Semantic search plus parallel reads find relevant pages in two tool calls without serial graph traversal or SEO-style titles.
- Low mechanism scales with the model. A zip of files plus a deletable SQLite index lets agents invent bash, perl, and SQL access patterns as models improve.
- Keep the format open and pinned. Portable zip archives, a spec, and sha256 review avoid harness lock-in and let untrusted shared memories be inspected.
quotes
“But memory - especially to a model - is much better represented as data.”
“just have the agent write the memory directly in its favourite format (which is Markdown).”
“So in a memoryfield, at most 2 tool calls are required (#1 to search, #2 to read in parallel).”
“Memory is data! The less fixed machinery we put between the agent and that data, the better the agent can be.”