What Should World Models Forget? Stratified Retention for Continual Adaptation

Paper: 2610.03713 Authors: Nishit Anand, Ramani Duraiswami, Dinesh Manocha Categories: cs.AI, cs.CV, cs.LG, eess.IV, eess.SP

The Gap

Continual learning in deep neural networks has long operated under a foundational assumption inherited from static supervised classification: any degradation in performance on previously encountered data represents catastrophic forgetting. Under this paradigm, a benchmark scores a model higher the more strictly it preserves its past predictions.

However, world models predicting physical environments violate this stationary assumption. The physical world undergoes continuous state transitions:

1. When a robot picks up a cup from a table and places it on a shelf, the prior ground truth (“the cup is on the table”) is no longer valid.
2. A world model that stubbornly insists the cup is still on the table is malfunctioning, yet standard continual learning metrics count the update as “forgetting.”
3. Consequently, existing evaluation metrics paradoxically rank a completely frozen model as having zero catastrophic forgetting, penalizing the adaptive model that correctly revised its beliefs.

   PROBLEM: STATIONARY FORGETTING METRICS BREAK IN DYNAMIC WORLDS

   Past State: Cup on Table            Action: Robot moves Cup to Shelf
               |                                      |
               +------------------+-------------------+
                                  |
                                  v
   Observation: World Model Updates Belief (Cup is on Shelf)
                                  |
       +--------------------------+--------------------------+
       |                                                     |
       v                                                     v
   Standard Continual Learning Metric            Actual Physical Reality
   "Prediction on t_0 data degraded!"            "Outdated state was revised!"
   -> Penalized as Catastrophic Forgetting       -> Required adaptive behavior!
   (Frozen model achieves 100% score)            (Frozen model is hallucinating)
                                  |
                                  v
   METHOD: STRATIFIED RETENTION ACROSS INVARIANCE TIMESCALES
   - Physical Invariants (Gravity, Permanence): Zero Tolerance for Forgetting
   - Transient Instance States (Object Positions): Low Revision Latency
                                  |
                                  v
   EVIDENCE: Differential retention curves isolate physics regression from state update
                                  |
                                  v
   CONCLUSION: Discarding outdated world state is correct adaptation, not a bug

The Increment

One sentence: By categorizing knowledge along an invariance timescale, this paper distinguishes between physical invariants (which must never be unlearned) and non-stationary environment states (which must be revised rapidly), proposing a differential retention metric that evaluates continuous adaptation without rewarding frozen models.

Core Mechanism

The authors formulate continual world modeling as a multi-timescale adaptation problem. Rather than treating all past transitions uniformly, the framework splits learned knowledge into two orthogonal strata:

1. Physical Invariants ($\tau = \infty$):
   • Universal laws such as conservation of momentum, gravity, collision rigidity, and object permanence.
   • These rules do not change when the agent moves from room to room or day to day. Degradation here represents true catastrophic forgetting.
2. Transient Instance States ($\tau < \infty$):
   • Ephemeral facts: where an object was placed five minutes ago, room illumination, or drawer open/closed status.
   • When the environment undergoes a transition, retaining the old state creates cognitive dissonance. The model’s objective is to minimize revision latency—the number of interaction steps required to overwrite obsolete beliefs.

To measure this without conflating the two, the authors introduce Differential Retention:

• Continuously runs regression tests on synthetic invariant physics benchmarks (evaluating gravity and collision predictions) while simultaneously feeding a stream of environment changes.
• Reports a 2D diagnostic vector: Invariant Retention Rate vs. State Revision Latency, explicitly forbidding naive scalar aggregation.

   STRATIFIED ARCHITECTURE AND RETENTION METRIC

   Environment Stream
         |
         +----------------------------------+
         |                                  |
         v                                  v
   +---------------------------+      +---------------------------+
   | Permanent Invariant Layer |      | Transient State Layer     |
   | - Gravity / Collisions    |      | - Object Locations        |
   | - Geometry Permanence     |      | - Open/Closed Attributes  |
   +---------------------------+      +---------------------------+
         |                                  |
         | Target: Zero Drift               | Target: Low Revision Lag
         v                                  v
   Invariant Regression Check         State Overwrite Measurement
         |                                  |
         +-----------------+----------------+
                           |
                           v
               Differential Retention Plot
               (Separates True Failure from Correct Revision)

The load-bearing structural metaphor is a ship’s navigation deck.

• The laws of hydrodynamics and celestial mechanics are cast in solid bronze and bolted to the bulkhead: gravity pulls down, compass points magnetic north, hulls displace water. If a navigator wakes up one morning and forgets that icebergs float (loss of invariant), the ship sinks.
• The tactical chart on the navigation table is made of slate and chalk: the current wind direction, the distance to the port, and the depth of the local shoal. When the ship sails 50 miles east, the navigator must wipe away yesterday’s pencil marks and draw the new coordinates. A navigator who refuses to erase yesterday’s shoal positions because “preserving old memory is a virtue” will run the ship directly onto the rocks.

Key Concepts

• Invariance Timescale ($\tau$): A parameter defining the expected lifespan of a physical belief. Universal physics has $\tau \to \infty$, while object coordinates have finite $\tau$ tied to agent manipulation.
• Revision Latency: The delay in observation steps before a world model updates its internal simulation to reflect a modified environmental state. Lower is better.
• Differential Retention: An evaluation protocol that tracks invariant stability alongside state flexibility, dismantling the perverse incentive where unyielding frozen checkpoints score highest.

Framework Shift

Before (Standard Continual Learning Metric):
  Metric = Accuracy(Test on Task 1) + Accuracy(Test on Task 2)
  Flaw: Punishes the model when "Task 1: Keys are on Desk" becomes false in Task 2.
  Result: A completely dead, frozen model achieves top marks.

After (Stratified Differential Retention):
  Evaluate: [Invariant Physics Retention Rate] vs. [State Revision Latency]
  Clarity: Demands zero tolerance for gravity loss, while rewarding rapid erasure of obsolete state.

From “never forget anything seen in previous trajectories,” the core shift is that a world model must actively prune and overwrite transient historical states to stay physically accurate in a dynamic reality.

Expert Assessment

Problem choice: Exceptional conceptual clarity. The continual learning literature has suffered from metric saturation and artificial setups for years. Applying stationary metrics to embodied world models was an obvious category error that someone had to call out.

Method maturity: The theoretical separation of timescales is intuitive and mathematically clean. By refusing to collapse invariant preservation and revision latency into a single scalar score, the authors avoid the standard pitfall of arbitrary weight tuning.

Experimental integrity: Validated across simulated robotic environments with controlled object relocations and physical stress tests. The paper exposes how previous state-of-the-art continual learning algorithms (EWC, replay buffers) inadvertently lock models into hallucinations by penalizing proper state deletion.

Writing quality: Razor-sharp argumentation. The critique of existing evaluation protocols is rigorous and constructive.

Verdict: strong accept — A foundational reframing that will recalibrate how the embodied AI community benchmarks continuous learning in dynamic physical systems.

Takeaways

• When evaluating world models, do not use standard continual learning metrics that treat historical state changes as forgetting.
• Architect world models with stratified representation layers: lock down invariant mechanics (gravity, collisions) while providing high-plasticity memory slots for volatile environment states.
• Track revision latency explicitly: measure how quickly an agent can overcome prior expectations once an object has been moved.