saved
Scaling Discovery through Test-Time Communication
Hraness cites a source capture. The source author remains the source.
gist
Identical agents sharing a workspace and verifier scores outperform matched independent best@$k$ on open-ended discovery. On ARC-AGI-3, team@$k$ matches roughly $4k$–$6k$ solo attempts and solves games no single agent clears. With enough compute, the same protocol sets new highs on polyomino packing and a 1,957-byte MNIST classifier. Gains need clear progress feedback; without it, communication can lose to independent sampling.
ideas
- Verified progress sharing beats parallel luck. Peers adopt measured breakthroughs while still searching, turning parallel exploration into cumulative progress rather than best-of-$k$ selection alone.
- Communication compounds with team size. team@$k$ matches far more independent agents as $k$ grows, and unlocks tasks solitary agents never solve.
- Research-horizon tasks transfer the pattern. Shared-workspace teams beat solo plateaus on packing scores and beat the prior human MNIST compression baseline under an accuracy floor.
- Coordination has conditions. Limited compute or weak intermediate verifiers can make independent agents win; clear feedback plus enough budget is what makes communication pay.
- Roles are optional. Homogeneous agents with slot ownership and an append-only shared log suffice when adoption is gated on measured improvement.
quotes
“test-time communication can substantially outperform independent parallel attempts on challenging tasks”
“a team of $k$ communicating agents, team@$k$, matches the success rate of $4k$ independent agents”
“A team of four agents produced a 1,957-byte classifier submission achieving 99.4% test accuracy”
“Independent agents may outperform communication when compute is limited or when a clear measure of progress is absent.”