saved
The plunging price of thought
Hraness cites a source capture. The source author remains the source.
gist
Luke Emberson and David Roodman measure the frontier cost of fixed AI performance across five math, science, and skill-game benchmarks since 2023. Using CAISI-style expense-performance curves and a preferred Box-Tidwell fit to the empirical cost frontier, they find about a 47% quarterly drop, or roughly 13× per year—faster than historical declines for electricity, compute, batteries, or DNA sequencing. Declines start near 66% per quarter when a capability is new SOTA and slow to about 32% two years later. Caveats include benchmaxxing, imperfect real-task transfer, and users who do not stay on the cost frontier.
ideas
- Price the task, not the token. Reasoning models burn more tokens on cheaper bases, so cost-per-performance replaces price-per-token as the right metric.
- About 47% cheaper each quarter at fixed accuracy. Model-free and preferred frontier fits agree on ~13× per year across five primary benchmarks, with math faster and games slower.
- New SOTA is briefly expensive, then crashes. Averaged across primary benchmarks, decline rates fall from ~66% per quarter at debut to ~32% two years later.
- Map full expense-performance curves. CAISI transcript truncation plus low-effort runs chart Pareto frontiers instead of unlimited-budget scores alone.
- Treat the headline as rough, not exact. Benchmaxxing, short samples, averaging choices, and real users off the frontier all soften the measured savings.
quotes
“Across that time, we find that the price for a given level of performance has fallen about 47% per quarter, or 13× per year.”
“No other general-purpose technology in history appears to have gotten so cheap so fast.”
“It is like the sticker price on a new car falling from $50,000 to $69.”
“On average for the primary benchmarks, the decline rate drops from 66% per quarter on average at SOTA to 32% two years later.”