ARMOR++: Multi-Agent Orchestration Exposes Transferability Gaps in Deepfake Detectors

Paper: 2607.15246 Authors: Christos Korgialas, Gabriel Lee Jun Rong, Dion Jia Xu Ho, Pai Chet Ng, Xiaoxiao Miao, Konstantinos N. Plataniotis Categories: cs.CV

The Gap

Deepfake detectors are supposed to be the immune system against synthetic media, but they have an Achilles’ heel: adversarial transferability. When you craft perturbations against one detector and deploy them on a different, unseen detector, performance collapses. Prior work falls into two camps — single-primitive transfer attacks (like PGD variants or frequency-based perturbations) that optimize one strategy hard but miss the diversity of inductive biases across architectures, and agentic approaches that use LLMs to pick attacks but lack semantic grounding and entropy-aware mixing. The specific frontier this paper targets: CNN-to-transformer transfer under strict no-query (blind) conditions. Existing methods treat the perturbation space as monolithic — one loss, one optimizer, one hope. They don’t ask: what does the image actually contain, and which parts matter most for fooling a detector? ARMOR++ says: let a vision-language model read the scene, let a language model plan the assault, and let five specialists execute in concert.

Deepfake detectors rely on fragile, architecture-dependent cues
    |
    v
Single-primitive attacks optimize one strategy but miss diverse inductive biases
    |
    v
Hypothesis: LLM-orchestrated multi-primitive framework with semantic priors
            can target heterogeneous biases simultaneously
    |
    v
Method: VLM spatial priors + LLM orchestrator + 5 complementary primitives
    |
    v
Evidence: AADD-2025 benchmark, blind-target ASR outperforms all baselines
    |
    v
Conclusion: Significant residual vulnerability remains in deployed detectors;
            agentic orchestration is a viable paradigm for red-teaming

The Increment

One sentence: Before this paper, adversarial attacks on deepfake detectors were either single-strategy optimizers or blind LLM pickers; after this paper, attacks have eyes (VLM semantic priors), a brain (LLM orchestration), and a diversified toolkit (five primitives mixed with entropy regularization).

Core Mechanism

ARMOR++ has three layers working in sequence: perception, planning, and execution.

Layer 1 — Perception: The Qwen2.5-VL vision-language model analyzes the input image and extracts spatial semantic priors. This means it identifies where the face is, where artifacts might live, which regions are semantically rich versus background noise. Think of this as generating a vulnerability map — not every pixel matters equally, and the VLM tells the system where to focus.

Layer 2 — Planning: The Qwen3 large language model acts as an orchestrator. It receives the semantic priors and makes three decisions: (1) which of the five attack primitives to activate for this specific image, (2) what hyperparameters each primitive should use (adaptive reparameterization rather than fixed settings), and (3) how to blend the resulting perturbations via entropy-regularized mixing. The entropy regularization is key — it prevents the mixing from collapsing onto a single dominant primitive, maintaining diversity in the attack signal.

Layer 3 — Execution: The five primitives each contribute a perturbation channel: dense gradient optimization (PGD-style, hits the loss surface hard), saliency-based methods (focus on detector-attention regions), spatial transformations (geometric warps that break spatial priors), frequency-domain perturbations (attack spectral features detectors rely on), and block-structured modifications (localized patches that exploit patch-based processing in transformers). These are merged into a single adversarial perturbation that targets multiple inductive biases at once.

Input Image
    |
    v
+-------------------+
| Qwen2.5-VL (VLM) |
| Semantic Priors   |
| (face, regions,   |
|  artifact maps)   |
+-------------------+
    |
    v
+-------------------+
| Qwen3 (LLM)      |
| Orchestrator      |
|                   |
| 1. Primitive      |
|    Selection      |
| 2. Hyperparameter |
|    Re-params      |
| 3. Entropy-reg.   |
|    Mixing Weights |
+-------------------+
    |
    +--------+--------+--------+--------+
    v        v        v        v        v
[Dense] [Saliency] [Spatial] [Freq]  [Block]
  Opt     Based      Trans    Domain  Struct
    |        |        |        |        |
    +--------+--------+--------+--------+
                  |
                  v
    +---------------------------+
    | Entropy-Regularized       |
    | Perturbation Mixing       |
    | (blend 5 channels with    |
    |  diversity constraint)    |
    +---------------------------+
                  |
                  v
         Adversarial Image
                  |
                  v
    Transfer to Blind Target Detector

The Heist Analogy: Imagine you’re planning a bank heist, but you’ve never been inside the bank and you can’t ask any questions about it.

The scout drone (Qwen2.5-VL) flies over the building and generates a detailed map — where the vault is, where the guards patrol, which walls are thin. This is the semantic prior: you now know the terrain, even if you don’t know the exact alarm system inside.

The mission commander (Qwen3 LLM) reviews the map and assigns roles. She doesn’t just pick one specialist — she calls in five, each with a different skill. The lockpicker (dense optimization) goes straight for the loss surface. The sniper (saliency-based) targets the exact spots where the alarm system is most sensitive. The contortionist (spatial transforms) finds ways to slip through geometric blind spots. The frequency jammer (frequency-domain) disrupts the spectral signatures the sensors detect. The demolition expert (block-structured) plants targeted charges on structural weak points.

The commander’s real genius is the coordination plan — she blends everyone’s approach so no single specialist dominates (entropy regularization), because she knows that a bank defended against lockpicking alone will catch the lockpicker, but a bank facing five simultaneous strategies has a much harder time. The final plan is one coordinated assault that exploits vulnerabilities across all defense layers at once. And critically, this plan works even against banks you’ve never scouted (blind transfer), because the five strategies target fundamentally different aspects of the defense architecture.

Key Concepts

• Blind Transfer (No-Query Adversarial Transfer): You craft perturbations using one detector (the surrogate), then apply them to a completely different detector (the target) without ever sending a single query to the target. It’s like practicing your heist on a model of Bank A, then robbing Bank B — no scouting allowed. This is the hardest setting because you get zero feedback on whether your attack is working until deployment. Most prior work cheats by assuming at least some queries; ARMOR++ operates strictly under this constraint.

• Inductive Bias Heterogeneity: Different neural network architectures “think” differently about images. CNNs build features through local convolutions; transformers attend globally via self-attention. This means they rely on *different forensic cues to detect fakes. An attack optimized purely for CNN-style features will miss transformer-style features. ARMOR++ addresses this by combining primitives that span multiple bias families — frequency-domain attacks hit spectral features regardless of architecture, while spatial transforms break the geometric assumptions both families make. The insight: no single attack primitive covers all biases, so you need a portfolio.

• Entropy-Regularized Mixing: When you combine multiple perturbation signals, there’s a natural tendency for the optimizer to favor the one that works best on the surrogate and ignore the others. This is dangerous because the “best on surrogate” primitive may transfer poorly. Entropy regularization adds a penalty that rewards the mixing weights for staying spread out — it forces diversity. Think of it as a portfolio diversification constraint in finance: don’t put all your money in one stock, even if it’s performing well today, because tomorrow’s market (the target detector) might punish that bet.

Framework Shift

Before (mainstream approach):        After (this paper):
                                     +-------------------+
Single attack primitive              | Qwen2.5-VL (VLM) |
(PGD / freq / spatial)               | Semantic Priors   |
    |                                +--------+----------+
    v                                         |
Fixed hyperparams                             v
    |                                +-------------------+
    v                                | Qwen3 LLM        |
Optimize one loss                    | Orchestrator      |
on surrogate                         | (select, adapt,   |
    |                                |  mix with entropy)|
    v                                +--------+----------+
Hope it transfers                             |
    |                               +--+--+--+--+--+
    v                               |D |S |Sp|F |Bl|
Single-channel                       |en|al|tr |re |oc|
perturbation                         |se|i |an |q  |k |
                                     +--+--+--+--+--+
                                          |
                                          v
                                 Entropy-reg. mixing
                                          |
                                          v
                                 Multi-channel perturbation
                                 (targets heterogeneous biases)

From single-primitive, fixed-parameter attacks to agentic multi-primitive orchestration with semantic grounding and diversity-preserving mixing, the core shift is treating adversarial transfer as a planning problem under architectural uncertainty rather than an optimization problem on a single loss surface.

Expert Assessment

Problem choice: This is a real and well-motivated gap. The CNN-to-transformer transfer problem in deepfake detection is genuinely underexplored, and as the field migrates to transformer-based detectors (which it is), this vulnerability becomes more consequential. The “agentic” framing is timely given the broader trend of using LLMs as planners, though one could argue the problem could be addressed with simpler ensemble methods. Still, the semantic prior contribution is a genuine differentiator — most adversarial attack papers treat images as flat tensors, not scenes with structure.

Method maturity: The architecture is clever but heavy. You’re deploying two large models (a VLM and an LLM) just to orchestrate perturbation generation — this is expensive and introduces dependencies on specific model families (Qwen). The five primitives themselves are not novel individually; the novelty is entirely in the orchestration. One concern: could a well-tuned ensemble with fixed weights and hand-crafted spatial priors achieve 80% of the gain at 10% of the cost? The paper would be stronger if it addressed this ablation directly and honestly.

Experimental integrity: The AADD-2025 benchmark is a reasonable choice and the evaluation includes both low- and high-quality image regimes, which is commendable. The blind-target ASR metric is the right one for this problem. However, I’d want to see more detail on: (1) computational cost comparison — how many GPU-hours per image? — and (2) whether the gains hold when the target detector uses adversarial training or input preprocessing defenses. The paper mentions “robust defensive configurations” but the abstract is vague on specifics. Red flag: if the baselines are weakly tuned, the comparison may be flattering.

Writing quality: The abstract is dense but well-structured. The weakest section (based on what’s visible) is likely the ablation study — papers like this live or die on showing which components matter, and I suspect the VLM prior is doing most of the heavy lifting while the LLM orchestration adds marginal gains at high cost. The authors should have led with a clearer “cost vs. benefit” narrative rather than burying it.

Verdict: weak accept — The agentic orchestration framing is genuinely novel for the adversarial deepfake detection domain, and the VLM-as-semantic-prior idea transfers well to other vision tasks. But the complexity-to-gain ratio needs tighter justification, and the reliance on specific LLM/VLM families limits immediate reproducibility.

Takeaways

Three concrete ideas worth stealing:

1. Semantic priors for adversarial attacks: Using a VLM to generate spatial vulnerability maps before crafting perturbations is a transferable idea. Any domain where you’re attacking vision systems (object detection, segmentation, medical imaging) could benefit from “what does the model see?” as a preprocessing step for attack design.

2. Entropy-regularized ensemble mixing: The diversity-preserving blending constraint is a general technique. Whenever you’re combining multiple strategies under distribution shift (the surrogate-to-target gap is a form of shift), penalizing concentration in your mixture weights is a principled move. This applies to ensemble robustness, domain adaptation, and even federated learning.

3. LLM as differentiable-free hyperparameter tuner: Using an LLM to set hyperparameters per-instance (rather than per-dataset) is an interesting alternative to Bayesian optimization when the search space is structured and the LLM has relevant priors. Worth exploring in other optimization-heavy settings where per-instance adaptation matters.