RSI-Anything: Unlocking Recursive Self-Improvement with World Models

Paper: 2610.13912
Authors: Taoyong Cui, Pinxian Zeng, Wanghan Xu, Hao Xiao, Shuhan Xue
Categories: cs.CL

The Gap

Autonomous Recursive Self-Improvement (RSI)—the theoretical engine where an agent continually harvests its own operational rollouts to enhance its capabilities without human intervention—remains one of the Holy Grails of artificial intelligence. In real-world environments, however, RSI architectures frequently succumb to distributional collapse:

1. The Cost of External Feedback: Physical ground-truth verification (such as robotic execution, wet-lab experiments, or full test-suite runs) is prohibitively slow and expensive.
2. Simulation Drift & Echo Chambers: When agents rely on internal “world models” to simulate environment feedback and synthesize training trajectories, the world model rapidly diverges as the agent policy explores novel behaviors. Training the policy on a drifted world model, and then re-training the world model on distorted policy rollouts, produces catastrophic hallucination loops.

   THE VICIOUS CYCLE OF UNGATED RECURSIVE SELF-IMPROVEMENT

   Policy pi_0 generates novel exploratory action
            |
            v
   World Model M_0 predicts imaginary reward (uncalibrated)
            |
            v
   Policy updates to pi_1 based on simulated hallucination
            |
            v
   World Model updates to M_1 on distorted policy traces
            |
            v
   ===> RAPID DELUSION & CATASTROPHIC DEGRADATION (ECHO CHAMBER)
            |
            v
   RSI-Anything SOLUTION:
   - Predictive World Model guides action & data curation
   - Sparse Real Feedback conditionally adapts the World Model
   - Strict Compatibility & Component Checks gate every permanent checkpoint update!

The Increment

One sentence: Built on top of the JEPA-Anything foundation, RSI-Anything introduces a domain-agnostic co-evolution framework that coordinates agent policy optimization and predictive world model adaptation through strict compatibility gates, unlocking stable, multi-round self-improvement across software engineering, GUI automation, and scientific discovery.

Core Mechanism

RSI-Anything constructs a disciplined, dual-track co-evolution pipeline governed by formal compatibility gating:

1. Predictive Action & Data Selection:
   • The world model acts as an in-situ predictive judge. During agent rollout generation, the world model predicts the downstream success probability and information gain of candidate steps, pruning unpromising execution branches before external execution.
2. Conditional World-Model Adaptation:
   • Whenever sparse real-world feedback is obtained (e.g., a test suite result or a reaction yield measurement), the delta between prediction and reality is logged.
   • The world model undergoes targeted adaptation, continuously refreshing its latent transition dynamics to match the shifting support distribution of the active policy.
3. Component & Compatibility Gating:
   • Crucially, neither the updated policy nor the updated world model is committed automatically.
   • A bi-directional validation suite verifies mutual compatibility: the new policy must not induce catastrophic forgetting on established tasks, and the updated world model must maintain predictive calibration across historical exploration traces. Only paired updates that pass the compatibility threshold are permanently committed.

   RSI-Anything DUAL-TRACK CO-EVOLUTION PIPELINE

          +-----------------------------------------------+
          |          Current Policy & World Model         |
          +-----------------------------------------------+
                       |                     |
           [Rollout Selection]       [World Model Filter]
                       v                     v
   Candidate Actions -----------------> Predicts Trajectory Success
                       |
                       v (Selects Top Prediction)
          Sparse Real-World Environment Execution
                       |
                       +-----------------------------+
                       |                             |
                       v (Policy Loss)               v (Prediction Error)
            Policy Candidate (pi_{t+1})    World Model Candidate (M_{t+1})
                       |                             |
                       +--------------+--------------+
                                      |
                                      v
                      [Bi-Directional Compatibility Gate]
                         Did validation accuracy drop?
                         Is predictive calibration sound?
                                      |
                         +------------+------------+
                         |                         |
                     Pass [Commit]             Fail [Revert]
                         |                         |
                         v                         v
               Next Iteration State        Keep Current Stable Checkpoint

Key Concepts

• Recursive Self-Improvement (RSI): An algorithmic loop wherein an AI system refines its own algorithms, model weights, or operational workflows using internally generated and validated experience.
• Predictive Selection: Using an internal predictive world model to score candidate rollout paths, bypassing the need to query costly real-world environments for every micro-action.
• Compatibility Gating: A strict regression test that ensures newly adapted policy-model pairs do not degrade baseline performance or lose physical calibration.

Framework Shift

Before (Ungated Policy-World Model Self-Play):
  Update Rule: Continuously train policy on world model rollouts, and world model on policy rollouts.
  Failure Mode: Rapid hallucination drift; policy exploits simulation quirks; runaway collapse.

After (RSI-Anything Gated Co-Evolution):
  Update Rule: Predictive filtering for exploration, conditional world-model updates on sparse ground truth.
  Gatekeeper: Bi-directional compatibility check protects system integrity.
  Cross-Domain Impact:
    - Code Repair: 26.3% -> 31.2% success.
    - GUI Agents: 39.4% fewer action steps to achieve goal.
    - Wet-Lab Planning: 21.8% -> 46.0% accuracy across 5 successive iterations.
    - Chemical Optimization: 96.15% yield (vs 82.16% Bayesian optimization).

Expert Assessment

Problem choice: Foundational milestone. Moving recursive self-improvement from theoretical science fiction and narrow games (like chess/Go) into messy open-domain engineering and scientific discovery.

Method maturity: Grounded in Joint Embedding Predictive Architectures (JEPA) rather than pure autoregressive generative noise; the addition of compatibility gates prevents model rot.

Experimental integrity: Unusually comprehensive experimental validation spanning 4 distinct domains: software debugging, computer GUI navigation, biological wet-lab planning, and physical chemical reaction yields.

Writing quality: Clear, well-structured, and realistic about the compute and validation requirements for multi-round RSI.

Verdict: strong accept — A pioneering framework that proves recursive self-improvement can be stabilized and scaled in diverse complex domains.

Takeaways

• Do not let world models and policies co-train unconstrained; mandatory compatibility checks are essential to prevent delusion cascades.
• Use predictive world models primarily for action-candidate filtering to maximize the sample efficiency of rare, expensive ground-truth calls.
• Track calibration error across successive policy generations; a world model that loses calibration must be re-anchored immediately.