Back to Home
arXiv AI··Papers & Tech

Self-discovering RL in the Era of Experience: Is Learning History an Asset or a Burden?

中文摘要

该研究探讨强化学习的递归自我改进,分析在交互驱动的经验时代中,学习历史对算法累积适应性的影响。

English Summary

This paper investigates recursive self-improvement in reinforcement learning, analyzing whether learning history benefits or burdens cumulative adaptation within interaction-driven environments.

Original Excerpt

arXiv:2609.35897v1 Announce Type: new Abstract: The pursuit of recursive self-improvement (RSI) toward general intelligence is divided between macro-level language model scaling and the interaction-driven principles of "Era of Experience". Yet, any self-improving architecture ultimately rests upon its underlying optimization engine: if general intelligence requires learning from grounded interaction, the reinforcement learning (RL) update rule itself must be capable of cumulative adaptation. While algorithm self-discovery has produced Disco103 that surpassed PPO to achieve SOTA benchmark performance -- its internal update machinery remains an uninspected black box. We present the first causal mechanistic audit of a self-discovered RL rule, structured directly around the five pillars of the Era of Experience: extended horizon, grounded reward scales, continuing streams, within-lifetime change, and exploration depth. By surgically pinning, freezing, and transplanting recurrent states while holding meta-parameters fixed, we test when learning history acts as an asset or a burden. Three findings organize the audit: (1) Recurrent history actively expands usable reward scales, sustaining…