hraness

saved

Dream-RSI: Recursive Self-Improvement through Evolving Worlds

by Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng and Yunsong GuoarXivpublished

read the original source

Hraness cites a source capture. The source author remains the source.

OompaSponge

gist

Tong Zheng and coauthors introduce Dream-RSI, an orchestration layer that makes exploration for coding agents programmable without changing the underlying agent. Fixed exploration policies fail as search scales, and online policy optimization is expensive under delayed long-horizon feedback. Dream-RSI treats accumulated discovery trees as a replay simulator, dreams off-policy to refine exploration cheaply, then redeploys the improved policy online so new discoveries expand the simulator. Across algorithm engineering, mathematical optimization, and GPU kernel engineering, it matches or improves discovery quality while cutting cost in several settings.

ideas

  • Exploration is the RSI bottleneck. Autonomous agents need high-value discovery, but fixed strategies do not adapt as search spaces grow and online meta-optimization faces delayed, expensive rollouts.
  • Keep the coding agent unchanged. A lightweight orchestration layer makes exploration explicit and programmable around an unmodified underlying agent.
  • History becomes a replay simulator. Accumulated discovery trees stand in for the realized search space so the system can evaluate exploration policies without new online runs.
  • Dream offline, redeploy online. Off-policy dreaming yields immediate low-cost feedback; the refined policy then drives further discovery and expands the simulator pool.
  • Quality holds while cost falls. On algorithm engineering, math optimization, and GPU kernel engineering, discovery quality is competitive or better with substantially lower cost in several settings.

quotes

“A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying coding agent unchanged.”

Tong Zheng et al., describing the orchestration boundary.

“accumulated discovery history can serve as a replay simulator over the realized search space.”

Tong Zheng et al., stating the key insight.

“achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings.”

Tong Zheng et al., summarizing empirical outcomes.