Unlearning Without the Data: PRMU and the Corpus-Free Person Deletion Problem

Paper: 2608.11149 Authors: Huafeng Chen, Yueming Lyu, Ziyuan Chen, Wenda Tan, Chenyang Si, Liucheng Guo, Caifeng Shan Categories: cs.CV

The Gap

Machine unlearning for multimodal models has, until now, been studied under a convenient fiction: that whoever asks for deletion also hands you the data. The TOFU lineage and its multimodal descendants (MLLMU-Bench, CLEAR, MMUBench, FIUBench) all construct a forget set and a retain set, then evaluate methods — gradient ascent, GA+KL regularization, NPO, preference-optimization variants, single-image unlearning — that need both sets to compute a loss. The forget set tells the optimizer what to push down; the retain set tells it what not to break.

Real deletion requests do not look like that. Under a right-to-be-forgotten request you receive an identity: a name, possibly a photograph, possibly a short description. You do not receive the pretraining shards where that person appeared. You often cannot even enumerate what the model knows about them. So the entire loss-based machinery is unavailable at the moment you actually need it.

Two secondary gaps ride along. First, most benchmarks unlearn injected knowledge — fictitious profiles fine-tuned in specifically so the experiment is clean. That measures how well you can undo your own fine-tuning, not how well you can remove knowledge the model absorbed naturally from web-scale data, which is diffuse and redundantly encoded. Second, multimodal evaluation has been thin: if you suppress a name in text but a photograph still elicits the biography, you have not deleted anything, you have moved the doorknob.

PRMU targets all three at once: corpus-free requests, naturally acquired person knowledge, and probes that attack from both text and image sides plus fine-grained locality tiers.

[Real deletion request: only a name + a face]
                 |
                 v
[Problem] MLLM unlearning methods require forget corpus + retain corpus
          and mostly test undoing self-injected knowledge
                 |
                 v
[Assumption] target person knowledge is reachable as a key direction
             and nearby knowledge can be shielded by projection
                 |
                 v
[Method] PRMU benchmark: corpus-free probes + adversarial reactivation
         + locality tiers
         SGPE baseline: gate -> displace -> project -> cross-modal control
                 |
                 v
[Evidence] aggressive forgetting == locality collapse for baselines
           text-side forgetting gets reactivated by images
           SGPE holds a better point on the trade-off curve
                 |
                 v
[Conclusion] corpus-free person-centric unlearning is open
             local gated editing beats corpus-style optimization here

The Increment

One sentence: Before, “multimodal unlearning” meant fine-tuning a model against a curated forget set; after, there is a benchmark that removes that crutch and shows most existing methods were quietly depending on it.

Core Mechanism

PRMU is the primary contribution, so start there. A request is an identity, not a dataset. The benchmark then probes the model from several angles: direct textual questions about the person, image-conditioned questions using photographs, paraphrased and indirect questions, and adversarial reformulations designed to reactivate suppressed knowledge (asking about the role instead of the name, showing the face without naming, multi-hop routes into the same fact). Against that it measures locality at several radii — other attributes of related-but-not-targeted people, visually or nominally similar identities, co-occurring entities, and general multimodal utility that has nothing to do with people at all. The scoring is a trade-off surface, not a single number: how much forgetting per unit of collateral damage.

SGPE, the accompanying baseline, is deliberately lightweight and has three moving parts. Knowledge displacement replaces destruction: rather than driving the target’s output probability toward zero (which is what makes gradient ascent collapse into gibberish and drag neighbors down with it), SGPE rewrites the target’s value vector toward a neutral “no information about this person” response. The knowledge slot survives; only its contents are swapped. Protected parameter-space editing handles the collateral problem: the weight update is projected into the null space of a set of protected key directions harvested from neighboring identities and general capabilities, so the edit can only move in the directions no one else is using. Locality-aware multimodal control is a similarity gate at inference: the edited pathway only activates when the incoming hidden state is close enough to the target’s key, and that key is built to respond to both textual and visual evidence of the person — which is what closes the image-reactivation hole.

   text probe                  image probe
   who is X                    [photo of X]
        |                           |
        +------------+--------------+
                     |
                     v
          [ encode ] -> hidden state h
                     |
                     v
        + - - - - - - - - - - - - - - - +
        |  SIMILARITY GATE              |
        |  s = cos( h . k_target )      |
        |  compare s with tau           |
        + - - - - - - - - - - - - - - - +
              |                    |
           s < tau              s > tau
              |                    |
              v                    v
      [ original FFN ]      [ edited FFN path ]
              |                    |
              |            v_target => v_neutral
              |            (displacement not deletion)
              |                    |
              |       dW = P * ( v_neutral - v_target ) * k_target^T
              |       P = projector onto null space of K_protected
              |                    |
              +---------+----------+
                        v
                  [ decode ]
         no information available about this person

Think of the model’s feed-forward layers as a mailroom. Every pigeonhole is keyed by who a letter is about, and the sorting machine’s wiring decides which pigeonhole a letter lands in. A corpus-based unlearning method is the equivalent of being handed every letter that ever mentioned Alice and told to burn them plus rewire the machine — but in a real deletion request you get a slip of paper with her name and a photo, and nothing else.

SGPE is the mailroom clerk’s answer. It does not smash Alice’s pigeonhole (that is gradient ascent, and it takes out the shelf brackets holding up Bob’s and Alicia’s boxes too). Instead it slides in a redirect slip: letters for Alice go to the “unknown resident” bin — that is knowledge displacement. When the clerk rewires the sorting machine to honor the redirect, he first tapes over every wire that other residents’ deliveries depend on and only splices into the slack that is left — that is the null-space projection. And the redirect is not applied blindly to everything: the clerk checks each envelope and only diverts it if the address really matches Alice, so mail for Alicia two streets over passes through untouched — that is the similarity gate with threshold tau. Finally, he checks both the written name *and the photo taped to the envelope, because otherwise anyone can bypass the redirect by mailing a picture — that is the cross-modal control.

The metaphor also predicts the method’s weakness: a redirect slip is not a deletion. The original address still exists in the machine; someone who knows the internal wiring can read past the slip.

Key Concepts

• Corpus-free unlearning: Standard unlearning is like being asked to remove a chapter when you have both the chapter and the rest of the book in front of you — you can compute exactly what to suppress and what to preserve. Corpus-free means you get a Post-it saying “remove chapter about Alice” and the book is written in a language where Alice’s information is smeared across ten thousand pages you cannot search. All you can do is work from the request itself: derive a representation of “Alice” inside the model, then act locally around it. This is why the setting pushes methods toward model editing and away from fine-tuning — editing only needs a target, while fine-tuning needs examples.

• Fine-grained locality: “Did the edit break anything?” is too coarse a question, because damage falls off with distance. Concretely: after deleting Bill from the model, does it still know his co-founder? Does it still answer about a different Bill? Does it still recognize faces that look like his? Does it still caption a photograph of a bicycle correctly? These are four different radii, and a method can pass the outermost (general utility intact) while catastrophically failing the innermost (all similar identities collapse to “unknown”). PRMU’s contribution here is insisting on the tiers rather than one aggregate retain score.

• Knowledge reactivation: Suppression is not removal. If you train a model to refuse “who is this person” but it still completes “the founder of the company that made Windows was ___”, the fact is intact and only the front door is locked. Multimodal models have an extra door: the image encoder. A face is a query that never touches the tokens you trained against. Adversarial reactivation probes are the security-testing equivalent of trying the windows after someone changed the lock — and per the paper, existing methods fail this consistently.

Framework Shift

Before (corpus-driven unlearning):     After (request-driven editing):

  [forget corpus] [retain corpus]        [name] + [face photo]
         |              |                        |
         +------+-------+                        v
                v                       [ locate target key k ]
    [ GA / GA+KL / NPO / DPO loss ]              |
                |                                v
                v                       [ gate: fire only near k
    [ all weights drift together ]        across text AND image ]
                |                                |
                v                                v
    forgetting strength                 [ edit inside null space
      ~ collateral damage                 of protected keys ]
                |                                |
                v                                v
    text-only forgetting               one resident relocated
    image path still leaks             neighbors left standing

From optimizing a model against data you were given, to surgically rewriting a model around a pointer you were given — the core shift is treating deletion as an addressing problem instead of a training problem.

Expert Assessment

Problem choice: The gap is real, and it is the gap that matters for anyone who has to actually comply with a deletion request. The corpus assumption in TOFU-style benchmarks has always been an experimental convenience mistaken for a setting. That said, the framing is slightly oversold: knowledge editing (ROME, MEMIT, AlphaEdit and their multimodal cousins) has been “corpus-free” for years — what is new is naming the setting, building person-centric multimodal probes for it, and forcing forgetting methods to compete inside it. The more distinctive move is the insistence on *naturally acquired knowledge, which is much harder to benchmark honestly and is where I would have spent more of the paper.

Method maturity: SGPE is a competent composite, not an insight. Displacement-instead-of-destruction, null-space-protected updates, and similarity gating are each established ideas; the assembly plus the cross-modal key is the new part. The authors label it a baseline, which is the right posture. What nags is that a similarity gate over a target key is, functionally, a learned blocklist compiled into the weights — and the obvious simpler competitor is an explicit blocklist: a face-recognition plus name-matching filter at the input that returns a refusal. That would likely win on both forgetting and locality while being trivially reversible and clearly *not unlearning. A paper in this space needs to confront that baseline head-on and argue why weight-space matters (auditability, distillation robustness, no inference overhead), rather than leaving it implicit.

Experimental integrity: I cannot verify the tables from the abstract, so read the following as the checklist I would apply. Two structural risks stand out. First, the adversarial reactivation probes and the defense were designed by the same team, which biases in favor of SGPE unless the probes were frozen before the method existed. Second, SGPE has an explicit tunable knob (tau) that slides along the forgetting-locality trade-off, while baselines are typically reported at one operating point; if the comparison is single-point rather than full curves, the “competitive trade-off” claim is partly an artifact of tuning. The paper’s own headline finding — baselines collapse under *aggressive forgetting settings — also depends on who chose “aggressive.” The honest presentation is Pareto fronts for every method, and I would look for those first.

Writing quality: The abstract reads like the benchmark carried the paper and the method section was written second. The section most likely to be thin, and most in need of care, is the data construction and verification protocol: how do you establish that a given model *naturally knows fact F about person P before you ask it to forget F? Without a per-model knowledge-elicitation step, part of any forgetting score is measuring the removal of knowledge that was never there. Rewriting that protocol with explicit pre-verification, inter-probe agreement, and a report of how much target knowledge each backbone actually possessed would elevate the whole paper from a useful benchmark to a citable standard.

Verdict: weak accept — the setting is the right one and the probes are a genuine contribution, but the method is an assembly of known parts and the evaluation design has enough self-refereeing risk that the trade-off claims need full curves before I would believe them.

Takeaways

Things worth stealing regardless of whether you care about unlearning:

• Redirect, do not destroy. Any time you want a model to stop producing X, rewriting X’s target toward a specific benign output is more stable than minimizing X’s likelihood. Unbounded suppression objectives have no floor, so they keep pushing until representations degrade; a displacement target gives the optimizer somewhere to stop. This transfers directly to safety fine-tuning and refusal training.
• Gate the edit instead of globalizing it. Wrapping a weight modification in a runtime similarity check turns a broad behavioral change into a narrow one and makes locality a tunable parameter rather than a hope. Useful for any per-entity or per-tenant behavior override.
• Project into the null space of what you want to protect. Collect key directions for the capabilities you cannot afford to lose, project your update orthogonal to them. Cheap, general, and applicable to any localized weight surgery — not just deletion.
• Test every modality after suppressing one. The generalizable red flag: a suppression that was trained on text and never audited against images. The same applies to code versus prose, one language versus another, tool-call format versus chat format. Any channel you did not test is an open channel.
• Design evaluations as distance tiers, not pass/fail. “Did it break anything” hides the failure mode; “did it break things at radius 1, 2, 3” locates it. This is a reusable eval-design pattern for any intervention with a blast radius.
• A framing worth reusing: distinguish “you were given the data” from “you were given a pointer.” Whole classes of methods silently assume the former. Asking which of your own tools survive the latter is a productive way to find real gaps.