saved
Fractal basins trap latent reasoning
Hraness cites a source capture. The source author remains the source.
gist
Lai, Bao, Quinn, and Gilpin treat latent reasoning as a dynamical system and show that hard tasks produce fractal convergence-time basins. Across Sudoku, mazes, visual puzzles, and logic, leading recurrent reasoners exhibit transient chaos: trajectories linger near saddle points that decode as nearly-correct wrong answers, so tiny initialization changes stretch solve time by orders of magnitude. Reasoning slowdowns are cast as a physical consequence of problem hardness, not merely a training quirk.
ideas
- Hard tasks make fractal basins. Convergence-time maps over latent initializations become self-similar as Sudoku, maze, visual-puzzle, and logic difficulty rise.
- Slowdowns are transient chaos. Trajectories eventually reach the correct fixed point, but saddle scattering creates long, divergent routes that look like Plinko.
- Saddles are almost-right answers. Decoded latent states near traps are dead ends in mazes and illegal digit grids in Sudoku, not random noise.
- Basin entropy tracks difficulty. Slice-averaged loops-to-converge scale with basin entropy across architectures and tasks.
- Training bifurcates into fractality. When a looped transformer learns multi-step linear solving, incorrect attractors become saddles and fractal basins appear with solvability.
quotes
“reasoning models exhibit transient chaos, a physical consequence of the computational complexity of difficult tasks.”
“even an infinitesimal change to the model’s initialization increases convergence time by orders of magnitude.”
“reasoning becomes trapped near answers that are nearly, but not quite, correct.”
“reasoning slowdowns are an inevitable consequence of problem hardness in modern artificial intelligence models”