MMDiff: Diffing SAEs to Find (and Flip) the Features Multimodal Training Added

Paper: 2608.09928 Authors: Hunar Batra, Lachin Naghashyar, Ashkan Khakzar, Philip Torr, Christian Schroeder de Witt, Constantin Venhoff, Ronald Clark Categories: cs.AI, cs.CL, cs.CV, cs.LG

The Gap

Sparse autoencoders have become the default tool for pulling interpretable directions out of a language model’s residual stream. Train an SAE on activations, get a dictionary of sparse features, browse the max-activating examples, write a label. That pipeline works, and people have applied it to multimodal models too — train an SAE on a VLM, find “features that fire on text in images” or “features about left/right.”

Two things it does not give you. First, it does not tell you which of those features are new. An MLLM is almost always a language model plus a vision encoder plus adaptation training. When you find a spatial feature in LLaVA, you do not know whether the base LM already had it and the projector merely learned to address it, or whether multimodal training built it. That distinction matters if your question is “what did this fine-tune actually change, and what did it break?” Model diffing work in the text-only setting (crosscoders between base and chat models) asks exactly this question, but the multimodal version has been mostly unexamined.

Second, browsing max-activating examples is descriptive, not causal. A feature that lights up on OCR-ish tokens might be a bystander. And the control literature has largely gone a different route: activation steering with a single contrast-derived vector injected at one layer (CAA, ITI, refusal directions). That is coarse — one direction, one layer, tuned by hand — and it lives disconnected from the interpretability pipeline that produced the feature labels in the first place.

MMDiff tries to close both gaps with one pipeline: diff to localize, contrast to establish causality, then intervene at the feature level.

[Problem] MLLM visual behaviors are opaque.
   |      SAEs give features but do not say which ones
   v      multimodal training created and are not control knobs.
[Assumption] Multimodal adaptation writes its changes into a
   |         small number of feature directions that survive
   v         being read out by a sparse dictionary.
[Method] MMDiff = SAE(base LM) vs SAE(MLLM) diff
   |            + per token contrastive firing analysis
   v            + ablate or steer the surviving features
[Evidence] ablation: -12% spatial . -17% OCR . -24% attack success
   |        VQA control flat . steering +3.6% / +1.8% over
   v        single layer steering baseline . 3 model families
[Conclusion] The features multimodal training adds are sparse
             causally specific and directly steerable.

The Increment

One sentence: Before, an SAE on an MLLM was a browsing interface over an undifferentiated feature soup; after, it is a diff plus a switchboard — you can say *this feature came from multimodal training, this one causes the OCR behavior, and here is what happens when I turn it off.

Core Mechanism

Three stages, and the ordering is the whole idea. Stage one is isolation by subtraction. Train an SAE on residual-stream activations of the base language model, train one on the multimodal-adapted model, and compare the two dictionaries. Features present in both are inherited language-model machinery; features that appear only in the multimodal dictionary (or whose behavior shifts substantially) are the candidates attributable to multimodal training. This cuts the search space from tens of thousands of features to a manageable candidate pool without running a single intervention.

Stage two is causal narrowing by contrast. For a target behavior — say, spatial relations — build two prompt sets that differ in whether the behavior is required, then measure per-token firing rates across the candidate pool. Per-token, not per-sequence, matters here: the firing that carries OCR or spatial content is concentrated on a few token positions (the image patch tokens, the relation word), and sequence-averaged statistics wash it out. What survives is a small set that fires differentially exactly where the behavior is being computed.

Stage three is intervention, in both directions. Clamp the surviving features to zero and measure whether the target task degrades while a control task (general VQA) holds. Or add the feature directions back with positive coefficients — across layers rather than at one hand-picked layer — and measure whether the target task improves. The bidirectional test is what upgrades the claim from correlation to causal specificity: removal must hurt the target and spare the control, addition must help.

 image + text
      |
      v
 +--------------------------+
 |  residual stream acts    |  layer L
 +--------------------------+
      |                        \
      | (A) SAE on MLLM acts     (B) SAE on base LM acts
      v                            v
   [ D_mm ]                     [ D_base ]
      \                            /
       \_________  diff  _________/
                   |
                   v
      candidate multimodal only features
                   |
        per token contrastive firing
        prompts_need_task  vs  prompts_no_task
                   |
                   v
           causal feature set F*
             /              \
       ablate F*            steer +F*
     (clamp to zero)      (add across layers)
            |                    |
            v                    v
   target task drops       target task improves
   VQA control flat        over single layer baseline

Think of an electrician handed a house that has been renovated. The SAE is the breaker panel: the wall wiring is a hopeless tangle of overlapping currents, and the panel’s job is to expose it as a set of labeled, individually switchable circuits. Now the renovation question. The electrician has the pre-renovation panel photo (base-LM SAE) and the current panel (multimodal SAE). Comparing them tells you which breakers the contractors added — that is the diff, and it is why you do not have to test all eighty breakers. Next, causality: labels on breaker panels lie. So you run one appliance at a time and watch the ammeter on each new circuit — the dishwasher draws on breaker 14, not on 12, even though 12 is labeled “kitchen.” That is per-token contrastive firing: run spatial prompts, see which candidate circuits actually draw current, and at which moment. Finally, verification and use. Flip breaker 14 off: the dishwasher dies and the lights stay on — that is selective ablation with the VQA control. Or upgrade the service on that circuit at several points in the run rather than only at the panel — that is multi-layer steering, and it is why it beats injecting one vector at one layer, the electrical equivalent of only ever touching the main breaker.

Key Concepts

• Sparse autoencoder (SAE): A model’s internal state at a given layer is one long vector of numbers, and each number is a mush of many unrelated things at once — this is superposition. An SAE is trained to re-express that vector as a sum of a *few items drawn from a large catalogue: instead of “3000 numbers all slightly on,” you get “features 41, 902, and 7715 are on, everything else is off.” The catalogue entries tend to be human-nameable (“this is a street sign,” “this is a comparison of two positions”). The catch: nothing forces two independently trained SAEs to use the same catalogue ordering or the same decomposition, which is exactly the difficulty that makes diffing two of them nontrivial.
• Model diffing: Rather than asking “what does this model know,” ask “what did this training stage change.” You have a before-checkpoint and an after-checkpoint; you want the delta expressed in interpretable units rather than in raw weight differences. It is a change-of-state audit. Applied here: multimodal adaptation is the training stage, and the delta is the set of features that vision brought into existence.
• Feature-level control vs activation steering: Classic steering computes one direction (average activation on “safe” prompts minus “unsafe” prompts), then adds it at one layer with a scaling knob. Feature-level control instead uses directions that came out of the dictionary — units the pipeline has already labeled and causally tested — and can act on several of them at multiple layers. The difference is between nudging the whole vehicle sideways and adjusting the specific control you identified as responsible.

Framework Shift

Before (mainstream approach):          After (this paper):

  MLLM acts                             base LM acts    MLLM acts
     |                                      |               |
     v                                      v               v
  [ one SAE ]                           [ SAE_base ]   [ SAE_mm ]
     |                                      \_____ diff ____/
     v                                              |
  browse max activating                             v
  examples . write labels                 candidates from
     |                                    multimodal training
     v                                              |
  (separate track)                        per token contrast
  one contrast vector                               |
  injected at one layer                             v
     |                                      causal feature set
     v                                          /        \
  behavior shifts . unclear                 ablate      steer
  which unit did it                        (control    (multi
                                            task flat)  layer)

From “read the features, then separately steer the model” to “subtract to localize, contrast to prove, then intervene on the same units you localized” — the core shift is making interpretation and control share one object.

Expert Assessment

Problem choice: Real gap, and well-placed in the trajectory. Text-only model diffing became a live subfield once people wanted to audit what RLHF changed; nobody had run the equivalent for multimodal adaptation, which is arguably a more dramatic intervention on a base model than chat tuning is. The safety angle is the sharpest instance: multimodal jailbreaks work because vision-side pathways bypass text-side refusal machinery, and “which features did vision training add that the safety training never saw” is close to the right question. This is not a manufactured gap.

Method maturity: Compositional rather than inventive. Every component exists — SAEs, checkpoint diffing, contrastive activation analysis, ablation, steering. The contribution is the ordering: use the cheap diff as a filter so the expensive causal testing has a small candidate set. That is a good instinct and it generalizes. But it also means the paper’s validity rests on the shakiest joint: comparing two independently trained sparse dictionaries. SAE dictionaries are not identifiable up to a fixed basis; two SAEs on the *same activations will produce different, partially split features. Crosscoders exist precisely to sidestep this. If MMDiff initializes the multimodal SAE from the base SAE, or matches features by decoder cosine similarity with a validated threshold, the diff is defensible; if it does a naive dictionary comparison, some fraction of the “multimodal-only” features are artifacts of retraining noise. I cannot confirm which from the abstract, and this is the first thing I would look for in the paper.

Experimental integrity: The shape of the evidence is right — three model families is real breadth, and pairing degradation on the target with a flat control task is the correct design. Two reservations. First, general VQA is a weak specificity control: a lot of VQA is answerable from coarse scene gist, so it can survive damage to spatial or OCR circuits without that proving much. A broader suite (MMBench, a text-only benchmark to check the LM was not dented, an over-refusal check alongside the ASR drop) would carry far more weight. Second, the steering gains are small: +3.6% and +1.8% over a single-layer baseline. Single-layer steering is a soft comparator, and part of the delta may simply be that MMDiff touches more layers rather than that it touches better directions — a multi-layer version of the naive baseline is the missing ablation. The ablation results (-12%, -17%, -24%) are the stronger half of the paper because selective damage is harder to fake than a two-point accuracy gain. Also absent from the abstract: random-feature and matched-non-diff-feature ablation controls, which are what separate “we found the causal features” from “clamping any dozen active features hurts.”

Writing quality: The abstract is doing marketing work — “three uses” framing suggests breadth was prioritized over depth, and three behaviors across three model families in one paper usually means each cell gets a paragraph. The section that would most elevate the paper is a validity analysis for the diff itself: how features were matched across dictionaries, what the false-positive rate looks like under a null (two SAEs on the same model, different seeds), and how much of the discovered set survives that null. That single section would convert the paper’s central move from plausible to established.

Verdict: weak accept — genuinely useful pipeline framing with the right bidirectional causal test, held back by a fragile core assumption about cross-SAE comparability and steering gains too small to carry the “control” half of the title.

Takeaways

• Diff as a search-space filter, not a result. The reusable pattern is not the diff itself but its role: use a cheap between-checkpoint delta to shrink the candidate set before expensive causal testing. This transfers to any fine-tuning audit — LoRA adapters, domain post-training, RLHF stages, safety patches. Ask “what changed” first so you only run interventions on the changed part.
• Contrast at token granularity. Sequence-averaged activation statistics hide behaviors that are computed at a few positions. If you have ever gotten a null result from a contrastive-activation analysis, try it per-token before concluding the feature is not there.
• Two-sided causal claims. Ablation that degrades the target *plus a control task that stays flat plus steering that improves the target. Any one of those alone is weak evidence. Cheap to add, and it is what distinguishes this from feature-browsing papers.
• Steer at multiple layers. The single-layer-injection convention in the steering literature looks like a historical accident rather than an optimum. Spreading the intervention across layers is nearly free to try.
• Reframe multimodal jailbreaks as feature-level, not prompt-level. If the vision pathway added features that safety training never constrained, then patching prompts is downstream of the actual problem. That framing is worth borrowing even if you never touch an SAE.
• What not to borrow uncritically: comparing two independently trained SAEs. If you build on this, tie the dictionaries together (shared init, crosscoder, or validated matching) and run a same-model different-seed null before believing any “new feature.”