World Models Have No Persistence: When You Look Away, the Moon Stops Moving

Paper: 2606.20545 Authors: Jinpeng Lu, Dexu Zhu, Haoyuan Shi, Linghan Cai, Guo Tang, Yinda Chen, Jie Cao, Duyu Tang, Yi Zhang, Yong Dai Categories: cs.CV

The Gap

Existing world model research has been optimized against a blind spot. Benchmarks like UniSim, VideoGPT, and various action-conditional video prediction tasks reward three things: frame fidelity (how pretty the image is), motion plausibility (do things move like they should), and camera controllability (can you steer the viewpoint). What they never check is whether the world keeps evolving when the observation is interrupted.

This is like judging a physics engine solely by how good its renders look, while never asking if the simulation continues when you close the lid of your laptop. The paper’s key insight: camera motion is an intervention on observability. If a world model is truly modeling a persistent world, then when the camera pans away from a falling apple, the apple should have moved further along its trajectory when the camera returns. If the world model is just generating video frames conditioned on camera pose, the apple will resume exactly where it was dropped — a “tracking shot” mentality, not a world simulation.

The logical chain is clean:

[Problem] Benchmark rewards surface fidelity
    |
    v
[Assumption] Fidelity + controllability = world model quality
    |
    v
[Gap]  No test for state persistence under occlusion
    |
    v
[Method] WRBench: camera motion as observability intervention
    |
    v
[Evidence] 9600 videos, 23 models, 4 paradigms -> same failure
    |
    v
[Conclusion] Current models track scenes, not simulate worlds

The Increment

One sentence: Before this paper, we had no way to ask whether a world model maintains an *internal state that evolves independently of observation; after this paper, we have a systematic benchmark that proves they fundamentally don’t.

Core Mechanism

WRBench is not a new model. It’s a diagnostic pipeline with three ordered stages:

1. Motion Execution Check: Did the camera actually follow the requested trajectory? This filters out models that can’t even do basic camera control.

2. Continuity & Identity Check: While the camera is moving/shaking/returning, does the scene stay coherent? Do objects remain identifiable across frames? This rules out models that hallucinate new objects or collapse the scene.

3. State Persistence Check: THIS is the novel part. The camera moves away from a target object (occluding it), then returns after a delay. Is the target in the state it *would have reached under physics, or the state it was in when the camera left?

The third stage is the punchline. It isolates a specific failure mode that none of the existing benchmarks probe.

[ASCII diagram: WRBench pipeline]

[Input: video + camera trajectory]
     |
     v
[Stage 1: Motion Execution]
  +-- Did camera move as requested?
  |   yes --> proceed
  |   no  --> FAIL (not worth testing)
  |
  v
[Stage 2: Continuity + Identity]
  +-- Scene stable? Objects trackable?
  |   yes --> proceed
  |   no  --> FAIL (can't test persistence)
  |
  v
[Stage 3: State Persistence]  <-- THE NOVEL PART
  +-- Occlude target via camera motion
  +-- Return after delay T
  +-- Compare target state to:
  |     - state_at_abandonment (tracking shot)
  |     - state_after_physics   (world simulation)
  |
  v
[Verdict: does the model pass Stage 3?]

Think of it like a theater rehearsal. The director (the benchmark) asks the actor (the world model) to perform a scene. First, does the actor hit their marks (motion execution)? Second, do they stay in character and not vanish from stage (continuity/identity)? Third — and this is the critical test — the director sends the actor offstage for a beat, then calls them back. Does the actor return having *continued their journey (maybe breathing harder, having changed costume) or do they return frozen in the exact pose they left in? Current world models are actors who freeze when they leave the stage — they have no “green room” where time passes.

Key Concepts

• Worldline Consistency: In physics, a worldline is the path an object traces through spacetime. A persistent world model must maintain a consistent worldline for every object *even when that object is not observed. The paper’s test is essentially: given two viewpoints separated by an occlusion gap, do the worldlines connect physically plausibly, or do they reset?

• Observability Intervention: Treating camera movement not as a free parameter to sample, but as a *controlled perturbation of the observation channel. This reframes “fancy camera control” from a feature into a probe. The math is simple: let S(t) be the true state of the world, O(t) be the observed state. A world model maps O(0

  ) to a prediction of S(t+delta) or O(t+delta). The test is: when O(t+delta) is empty (camera away), does the model’s hidden state h(t) continue to evolve S?

• Persistent State Core: The hypothetical component of a world model that maintains a time-evolving hidden state *regardless of the observation stream. It’s what makes a simulation different from a video renderer. Current models have a “video core” — they produce frames when prompted, but have no self-sustaining internal dynamics.

Framework Shift

The shift is from evaluating world models as frame generation engines to evaluating them as state evolution engines.

Before (mainstream approach):        After (this paper):

[Test Frame Quality]                 [Test State Persistence]
  +-- PSNR/SSIM on video               +-- Camera moves away
  +-- FVD (Frechet Video Distance)      +-- Event unfolds unseen
  +-- Camera controllability            +-- Camera returns
  |                                     +-- Is target state correct?
  +-- [Model outputs frames]            |
  |                                     +-- [Model must maintain
  v                                         internal state]
[Rewards: "looks real"]                 |
                                       v
                                    [Rewards: "world runs
                                     when unobserved"]

One sentence: From evaluating how good the render looks to evaluating whether the world runs when nobody is watching, the core shift is from video quality to state persistence.

Expert Assessment

Problem choice: Genuine gap. The paper identifies a blind spot that the entire field has been reinforcing through benchmark design. This isn’t manufactured — it’s a logical oversight that becomes obvious once stated, like realizing your car’s “self-driving” system works great until you ask it what happens when a pedestrian walks behind a truck.

Method maturity: Clever insight, not brute force. WRBench is elegant because it’s simple: three staged checks that isolate exactly one failure mode. The authors didn’t train a model or build a new architecture — they built a microscope. The only weakness: they identify the failure but don’t propose a solution. That’s fine for a diagnostic benchmark, but it leaves the reader hungry.

Experimental integrity: Strong. 23 models across 4 control paradigms (classical video generation, diffusion-based, autoregressive, and hybrid). 9600 videos. The failure is *universal — not a single model passes Stage 3. This is damning. Baselines are fair because the failure mode is so consistent. Red flag: the paper doesn’t deeply analyze why each model family fails differently — just that they all fail. A deeper failure analysis would strengthen the contribution.

Writing quality: Solid but front-loaded. The introduction and gap statement are excellent — they set up the problem with the moon metaphor and it’s instantly clear. The methods section is precise. Weakness: the results section is a bit monotonous (model X fails, model Y fails, model Z fails). A rewritten “Results” section that categorizes *how models fail (e.g., “resurrect objects,” “reset pose,” “jitter scene”) would elevate the paper from “proof of failure” to “taxonomy of failure.”

Verdict: weak accept — a diagnostic benchmark that reveals a universal failure mode, but doesn’t offer architectural guidance for fixing it. Worth attention as a warning, not as a solution.

Takeaways

• Don’t trust your benchmark. If your benchmark only tests things that are visible, you’re training models to be good at being watched, not good at being. This applies beyond world models: any system with a hidden state should be tested on its behavior *during occluded periods, not just when it reports.

• Camera motion as probe, not feature. This paper reframes camera controllability from a user-facing nice-to-have into a formal tool for probing state persistence. You can steal this framing for any sequential decision-making system: treat perturbations of the observation channel as diagnostics of hidden state stability.

• The “tracking shot” failure is testable. You don’t need a massive benchmark to check if your own model has this problem. Simple test: run a physics simulation (a ball bouncing, a pendulum swinging), train a world model on it, then occlude the model and check if the internal state matches the physics at unobserved timesteps. If it doesn’t, your model is a video renderer, not a world simulator.