saved
Agent Swarms are a Distributed Systems Problem
gist
Chroma's Foundation memory layer has an agent swarm ingest traces and company data into a shared wiki. Textbook database transactions abort and redo work that is cheap for a database and ruinously expensive for an agent. Robert Escriva's Fission protocol keeps exclusive page locks and wound-wait, but treats abort as early commit so already-written pages survive. Atomicity across the wiki is an explicit non-goal; per-page writes stay atomic on Chroma Cloud OCC.
ideas
- Agent work is too expensive to abort. A database retry re-runs the same code in milliseconds. An agent re-discovers its read set through search and reasoning, which costs minutes and tokens.
- Atomicity across the wiki is a non-goal. A valid reasoned edit is cheaper to leave than revert. Individual page writes stay atomic through Chroma Cloud optimistic concurrency control.
- Fission is two-phase locking plus early commit. Exclusive locks at first page read, not over search results; wound-wait on batch arrival timestamps; no rollback. Abort leaves committed work.
- Git merge throws away finished work. An early Foundation used git; three of eight calls gave up after conflicts and four after stale reads. Agents skip the contended page instead of resolving it.
- Prefix-safe writes make anomalies benign. A TODO tool keeps every write prefix valid. Haiku 4.5 plus Context-1 was enough to retry from observed state; 39.7% of wounded read-waits revisited the wounding page.
quotes
“the swarm is editing one shared state, just with the collisions postponed.”
“when a database transaction aborts, retrying re-runs the same code in milliseconds.”
“atomicity across the wiki as a whole is an explicit non-goal of Foundation.”
“Aborting is treated as early commit, not roll back.”