saved
EA Safety
Hraness cites a source capture. The source author remains the source.
gist
Venkatesh Rao argues Effective Altruism has become an "EA formation"—a philanthropy-to-AI-safety complex whose abstract, low-empiricism moral machinery plus urge to act now threatens to be a second-order problem of its own. He writes as a friend-with-footnotes: critical of the formation's totalizing style and eschatological AI story, not of personal integrity, and calls for checks analogous to post-Reformation limits on rival doctrines.
ideas
- Name the formation, not the membership card. EA now spans GiveWell-style philanthropy, Singerite life-optimization, rationalism, longtermism, x-risk, and frontier-lab governance; porous borders make card-checks scholastic.
- Friends with footnotes. Rao keeps personal friendship while refusing public fealty or obligatory debate—until the formation's power over AI moral framing forces a harder look.
- Uncertainty inside the model, not about it. The style numbers doubt within a frame while insulating premises about value, futures, evidence, and who prioritizes.
- AI as eschatology. Extinction, transcendence, and a narrow window turn careers into vocation and alignment into a salvation genre dressed as secular decision theory.
- Now there are two problems. EA grew by idiosyncratically posing AI safety; success creates "EA safety"—how to constrain a formation sure it alone should govern the first.
quotes
“Friends with footnotes are great to have.”
“My hypothesis is that these developments have a common cause.”
“AI supplies the formation not merely with a problem but with an eschatology”
“As the old joke goes, now we have two problems.”