P(doom)

gist

Armin Ronacher responds to Dario Amodei’s “pace the frontier” essay and viral P(doom) talk by agreeing on many risks—botnets, blind labs, regulatory failure—while opposing closed-lab pacing as the cure. He argues open-weight proliferation is built-in pacing, that today’s harms come from subsidized closed American models, and that Chinese distillation currently bails out everyone else; his primary fear is what the closed, dopamine-loaded token economy does to humans and commons infrastructure, not extinction.

ideas

• Doom as nuisance and human damage, not nukes. Persistent botnets and what closed models do to ordinary people matter more to him than bioterror or culture-war China scenarios.
• Only two labs are being asked to pace. Anthropic and OpenAI dominate; METR-tied evaluators and China-gap rhetoric look like concentrating control after training on the commons.
• Open weights pace by proliferation. Mass access on more equal terms would cut the distorted economics that let closed labs burn millions and ignore crimes.
• Regulation is already failing. EU rules miss live problems; US turbo-capitalism plus sinophobia leaves scraping, opaque token markets, and training-data disputes uncontrolled.
• Self-improvement may tax society first. Even without extinction, recursive gains raise costs for software, research, and anyone forced to buy closed tokens to keep up.

quotes