SolarWM: Making Video World Models Reproducible Across Backbones

Paper: 2609.02886 Authors: Junchao Huang, Guian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao, Wei Huang, Yihua Du, Zixin Zhang, Justin Cui, Yuchao Gu, Yukang Chen, Xinting Hu, Tianyu He, Shaoshuai Shi, Zhuotao Tian, Xin Wang, Mike Zheng Shou, Li Jiang Categories: cs.CV

The Gap

Interactive video world models are trained on aggregated data from many sources and built on many different video backbones, and that combination is harder than it looks. Datasets differ in temporal scale, camera geometry, visual quality, motion, and captioning styles. Video generators use distinct representations and architectures. So two failures compound: naive data mixing produces inconsistent supervision — the same field means different things in different sources — and model-specific implementations make results difficult to reproduce and compare.

This is a reproducibility problem with a specific cause. It is not that the individual contributions are wrong; it is that the coupling between data processing and model architecture means every new combination re-derives its own conventions. The field ends up with results that cannot be compared because the pipelines that produced them are not the same pipeline.

   AGGREGATED DATA  x  MANY BACKBONES
   domain: interactive VIDEO WORLD MODELS

   DATASETS differ in:
     temporal scale | camera geometry | visual quality
     motion | captioning styles

   VIDEO GENERATORS use:
     distinct representations and architectures
        |
        v
   TWO FAILURES COMPOUND
     [1] NAIVE DATA MIXING -> INCONSISTENT SUPERVISION
           the same field means different things across sources
     [2] MODEL-SPECIFIC IMPLEMENTATIONS -> results hard to
           REPRODUCE and hard to COMPARE
        |
        v
   [GAP] a REPRODUCIBILITY problem with a specific cause:
         data processing and model architecture are COUPLED,
         so every new combination re-derives its own conventions
         -> results cannot be compared, because the pipelines
            that produced them are not the same pipeline

The Increment

One sentence: Before this paper, video world-model results could not be compared across backbones; after it, a reconfigurable data engine and a backbone-native adaptation framework instantiate four 5B-33B models under shared interfaces and release the whole pipeline.

Core Mechanism

The contribution is infrastructure, and it is deliberately split along the axis the gap identified — data processing on one side, model adaptation on the other.

A reconfigurable multi-source data engine. It converts 1.43 million canonical clips from 10 datasets into a unified, frame-aligned contract covering visual observations, metric camera geometry, captions, quality metadata, selection decisions, and provenance. Two words carry the design. Frame-aligned means the temporal convention is fixed across sources, which is what makes mixing consistent. And the engine decouples source processing from mixture construction — so a new dataset is added by processing it into the contract, not by rewriting the mixer, and a new mixture is a configuration rather than a code change.

A backbone-native adaptation framework. Under shared camera-conditioning, training, and inference interfaces, the work instantiates four models of 5B-33B based on Wan2.2, LTX-2.5 and MiniMax-H3, while preserving their native representations and objectives. That last clause is the important design choice: rather than forcing all backbones into one representation — which is what makes comparison impossible in the first place — the framework keeps each backbone native and unifies the interfaces instead. Comparison then happens at the interface, where it is meaningful.

A unified three-stage recipe: bidirectional adaptation, teacher-forced autoregressive initialization, and distribution matching distillation. The stages correspond to a recognisable progression — adapt the model to the data, give it a usable autoregressive starting point, then distil toward efficient sampling.

Then the result that explains why the infrastructure matters. The resulting causal models enable real-time interaction over rollouts ranging from minutes to hours, having been trained on only 5-second sequences. So the horizon capability is not bought with long training clips; it comes from the recipe and the causal formulation. That is the kind of finding a shared pipeline makes checkable, since the same recipe was applied across four backbones of different sizes and origins.

And the paper releases the data, pipeline, recipes, weights and framework — which is the necessary final step for a contribution whose argument is that reproducibility was the problem.

   DESIGN: SPLIT ALONG THE AXIS THE GAP IDENTIFIED

   [DATA SIDE] RECONFIGURABLE MULTI-SOURCE DATA ENGINE
     1.43M canonical clips from 10 datasets
       -> a UNIFIED, FRAME-ALIGNED CONTRACT covering:
            visual observations
            metric camera geometry
            captions
            quality metadata
            selection decisions
            provenance
     KEY: DECOUPLES source processing from mixture construction
       -> adding a dataset = processing it into the contract
       -> a new mixture = a CONFIGURATION, not a code change

   [MODEL SIDE] BACKBONE-NATIVE ADAPTATION FRAMEWORK
     under SHARED camera-conditioning / training / inference
     interfaces, instantiate FOUR models, 5B-33B, based on
       Wan2.2 | LTX-2.5 | MiniMax-H3
     while PRESERVING their NATIVE representations and objectives
       <- crucially, backbones are NOT forced into one representation
          (that is exactly what makes comparison impossible)
       -> unify the INTERFACES, compare at the interface

   UNIFIED THREE-STAGE RECIPE
     bidirectional adaptation
     teacher-forced autoregressive initialization
     distribution matching distillation

   RESULT
     causal models enable REAL-TIME INTERACTION over rollouts
       from MINUTES to HOURS
     after training on ONLY 5-SECOND sequences
       -> horizon capability comes from the recipe and the causal
          formulation, not from long training clips

   RELEASED: data, pipeline, recipes, weights, framework

Think of it as a standards body for shipping containers rather than a better ship. Individual shipping lines were already good at their own routes. The problem was that a container packed for one line could not be loaded onto another, so every combination needed its own packing rules and nothing could be compared across carriers. The fix is a contract — one frame, one set of fields, decided once — plus keeping each carrier’s own vessel rather than forcing everyone onto the same hull. The payoff is that a route’s performance becomes measurable against another’s, because the cargo and the interfaces are the same.

Key Concepts

• The frame-aligned contract: a single specification covering observations, metric camera geometry, captions, quality metadata, selection decisions and provenance. Fixing the field meanings across sources is what makes mixture construction consistent, and provenance makes the result auditable.
• Decoupling source processing from mixture construction: adding data means processing into the contract, and changing a mixture means changing a configuration. It is what stops each new dataset from requiring a pipeline rewrite.
• Backbone-native adaptation: preserving each backbone’s representations and objectives while unifying interfaces. Forcing a single representation would have reproduced the comparability failure, so the design choice is the opposite one.
• Shared interfaces as the comparison surface: camera conditioning, training and inference specified once. It relocates comparison from “our pipeline versus yours” to a shared protocol, which is the precondition for reproducible numbers.
• Horizon from recipe, not from data: minute-to-hour rollouts after training on 5-second clips. It is a capability finding that the shared pipeline makes checkable across four backbones of different sizes.

Framework Shift

Before (per-combination pipelines):
  mix heterogeneous datasets naively -> inconsistent supervision
  implement per backbone -> results not comparable
  -> each new combination re-derives its own conventions
  -> the field cannot compare numbers

After (contract plus shared interfaces):
  1.43M clips -> frame-aligned contract, source processing
    decoupled from mixture construction
  four 5B-33B backbones kept NATIVE, adapted to SHARED interfaces
  three-stage recipe applied uniformly
  -> minutes-to-hours rollouts after 5-second training clips
  -> data, pipeline, recipes, weights, framework released

From results that could not be compared because every combination had its own pipeline, to a shared contract and interface set under which four different backbones are measured alike, the core shift is that reproducibility in this area is an engineering artifact built from data contracts and interface discipline rather than a property that follows from good models.

Expert Assessment

Problem choice: Excellent, and the framing is unusually clear-eyed. Treating the coupling between data processing and model architecture as the root cause — rather than as an inconvenience — is what makes the fix structural instead of per-project, and the release of the whole stack is the right conclusion for a reproducibility argument.

Method maturity: The split of the contribution along the gap’s own axis is what gives the design its logic: a data engine on one side to make mixing consistent, an adaptation framework on the other to make comparison possible. The decision to keep backbones native and unify interfaces rather than force a common representation is the substantive engineering judgement, and it is exactly the choice that distinguishes this from a framework that quietly standardises away the differences it was meant to study.

Experimental integrity: The breadth is the evidence — ten datasets, 1.43M clips, and four backbones spanning 5B to 33B from three different families, so the recipe’s uniformity is demonstrated rather than asserted. The 5-second-training to multi-hour-rollout result is striking and is the kind of claim a shared pipeline makes checkable, since the same recipe produced it across four backbones. The limitation is that all four backbones are video-generation architectures, so the contract’s generality for other model classes is untested, and the paper is careful to scope its claims to this domain.

Writing quality: The abstract lists the sources of heterogeneity concretely — temporal scale, camera geometry, visual quality, motion, captioning styles — which makes the problem tangible rather than abstract, and the contract’s fields are enumerated so a reader can see what unification required. Because the practical value is in adoption, a short section on what it costs to bring a new dataset or a new backbone into the contract would help others judge whether they can use it.

Verdict: strong accept — it identifies a structural reproducibility failure in video world-model research, fixes it with a data contract and interface discipline, and demonstrates the result across four backbones from three families.

Takeaways

• When results cannot be compared, look for coupling between data processing and model code. Unifying interfaces while keeping models native is usually the fix that preserves comparability.
• Define the data contract explicitly, including provenance. Frame alignment makes mixing consistent; provenance makes the result auditable.
• Decouple ingestion from mixture construction. Adding a dataset should mean conforming it to the contract, not rewriting the pipeline.
• Check whether a capability comes from data or from the recipe. Multi-hour rollouts from 5-second training clips is a recipe result, and it only becomes visible when one recipe is applied across many backbones.