saved
Where’s the “intelligence explosion”?
Hraness wrote this summary from a saved copy of the source. Quotations are taken word for word from the source.
gist
Ramez Naam argues in Noahpinion that current AI progress does not show a runaway intelligence explosion. He defines recursive self-improvement (RSI) as a range from AI helping researchers to a self-sustaining loop, and estimates today’s loop produces about 2–3% more research productivity per Epoch Composite Intelligence (ECI) point against a roughly 15–19% threshold. Capability gains remain rapid, but research tasks, model scaling, test-time compute, and agent swarms show diminishing returns. Naam expects narrow superintelligence in verifiable domains while calling for better measurements that could change this assessment.
ideas
- RSI spans several levels of automation. Naam distinguishes AI-assisted human work, autonomous improvement with diminishing returns, and a Type 5 runaway loop; current evidence supports Types 1 and 2.
- Real research lags benchmark horizons. OpenAI’s 80% success horizon for internal research tasks was roughly 15 minutes in July, versus four hours from ECI extrapolation and an 11-hour AI 2027 forecast; Naam notes that the tasks differ.
- More activity yields smaller gains. OpenAI researchers used 124× more tokens per person and ran 1.6× as many experiments, while Anthropic estimates that doubling overall progress would require productivity uplift roughly an order of magnitude larger than observed.
- The feedback loop remains below its self-sustaining threshold. Naam estimates 2–3% productivity gain per ECI point against a 15–19% threshold, a five- to tenfold gap, while noting that the data is noisy.
- Narrow superintelligence is arriving first. Highly verifiable domains allow synthetic data and machine-speed checking, while open-ended tasks still need human judgment and better ideas.
quotes
“Given our best current data, the AI self-improvement loop would need to be roughly 5–10× stronger to sustain itself, let alone run away.”
“An enormous increase in AI output has accompanied a much smaller increase in experiments run.”
“AI is helping build better AI. Under this estimate, though, each turn of the loop adds less than the last.”
“That doesn’t mean we’re close to super-intelligence for less verifiable, messier, open-ended work - or to a general ASI.”