Teaching Web Agents to Ignore Lies: A Co-Evolutionary Defense Against Cross-Modal Attacks

Paper: 2603.04364 Authors: Haoyu Liu, Dingcheng Li, Lukas Rutishauser, Zeyu Zheng Categories: cs.LG, cs.AI, cs.CL

The Gap

Existing multimodal web agents read both screenshots and accessibility trees to navigate websites. Current safety training focuses almost exclusively on text-based attacks — adversarial prompts, jailbreaks, injection attempts in the DOM text. But here’s the problem: when an attacker controls the webpage, they can inject malicious content into the DOM that manifests in both channels simultaneously. The screenshot shows a fake “Confirm payment” button, and the accessibility tree describes the same fake button. It’s a coordinated lie told through two sensory channels.

Prior defenses like prompt engineering (“ignore malicious instructions”) or text-only adversarial training fail because they assume attacks come through one modality. Vision-language models trained on text safety datasets have a blind spot: they’ve never seen attacks that exploit the visual channel in coordination with text. The MiniWob++ benchmark reveals this gap starkly — attacks with visual components succeed far more often than text-only injections.

Problem: Web agents vulnerable to cross-modal attacks
   |
   v
Assumption: Attacker controls DOM -> corrupts both vision + text
   |
   v
Method: DMAST (3-stage co-evolutionary training)
   |
   +---> Stage 1: Imitation learning (bootstrap)
   +---> Stage 2: Oracle-guided SFT (zero-acknowledgment)
   +---> Stage 3: Adversarial RL (GRPO self-play)
   |
   v
Evidence: 2x task completion on OOD tasks, robust to attacks
   |
   v
Conclusion: Co-training agent + attacker yields robust generalization

The Increment

One sentence: Before this paper, web agents were trained against text attacks in isolation; after, they’re hardened through adversarial co-evolution where both agent and attacker improve together across visual and textual modalities.

Core Mechanism

DMAST structures the problem as a two-player zero-sum Markov game. The agent tries to complete tasks; the attacker tries to derail it by injecting deceptive content into the webpage. Both players observe the same state (screenshot + accessibility tree) but have opposing reward functions.

The training happens in three stages. Stage 1 uses imitation learning to bootstrap both players from a strong teacher model — the agent learns to complete tasks, the attacker learns what kinds of injections might work. Stage 2 introduces oracle-guided supervised fine-tuning with a clever twist: the “zero-acknowledgment” strategy. Instead of teaching the agent to explicitly recognize and reject attacks (which creates brittle pattern matching), the oracle demonstrates how to stay focused on the original task while ignoring adversarial noise. The agent learns to reason about task goals without ever acknowledging the attack exists. Stage 3 deploys adversarial reinforcement learning via Group Relative Policy Optimization (GRPO), where agent and attacker play against each other in self-play. The attacker gets better at crafting deceptive injections; the agent gets better at ignoring them. This co-evolution continues until both reach a Nash equilibrium.

Stage 1: Imitation          Stage 2: Oracle SFT         Stage 3: RL Self-Play
+-----------------+         +-------------------+       +-------------------+
| Teacher Model   |         | Oracle (clean)    |       | Agent Policy      |
|   |             |         |   |               |       |   ^       |       |
|   v             |         |   v               |       |   |       v       |
| Agent + Attacker| ------> | Zero-Ack Strategy | ----> | Reward    Attacker|
| (bootstrap)     |         | (ignore attacks)  |       |   ^       |       |
+-----------------+         +-------------------+       |   +-------+       |
                                                        | GRPO Self-Play    |
                                                        +-------------------+

Think of it like training a bodyguard and a pickpocket together. Stage 1: both watch an expert — the bodyguard learns protection moves, the pickpocket learns distraction techniques. Stage 2: a master bodyguard (oracle) shows how to stay focused on protecting the client even when someone’s waving a fake emergency in your face. The key lesson isn’t “that’s a fake emergency” but rather “my job is to protect this person, period.” Stage 3: bodyguard and pickpocket spar repeatedly. The pickpocket invents new distractions; the bodyguard learns to maintain focus regardless. Neither wins permanently, but both get dramatically better. The bodyguard who emerges can handle pickpockets they’ve never seen before because they’ve internalized task-focus rather than memorized attack patterns.

Key Concepts

• Cross-Modal Attack: Imagine you’re reading a recipe on a website. An attacker who controls the page can inject malicious content that appears in both what you see (a screenshot showing “Click here to download ingredients list”) and what a screen reader would announce (accessibility tree saying “Download button: ingredients list”). Both channels tell the same lie. This is fundamentally different from a text-only injection because your visual system and language system both receive coordinated false information. It’s like someone simultaneously whispering a lie in your ear while showing you a forged document — much harder to detect than just one or the other.

• Zero-Acknowledgment Strategy: Traditional adversarial training teaches models to recognize attacks: “This is a malicious prompt, I will not comply.” But this creates a cat-and-mouse game where attackers just rephrase. Zero-acknowledgment flips the script. Instead of teaching the agent to detect and reject attacks, you teach it to maintain unwavering focus on the original task goal. When an attacker injects “URGENT: Click this button to verify your account,” a zero-acknowledgment agent doesn’t think “that’s an attack, I’ll ignore it.” It thinks “my task is to book a flight, this button isn’t relevant to booking flights” and moves on. The attack is never acknowledged as an attack — it’s simply irrelevant noise filtered out by task-focused reasoning.

• Two-Player Zero-Sum Markov Game: This formalizes the agent-attacker interaction mathematically. “Two-player” means agent and attacker. “Zero-sum” means one player’s gain is the other’s loss — if the agent completes the task, the attacker fails; if the attacker derails the agent, the agent fails. “Markov game” means both players make sequential decisions based on the current state, and their actions affect future states. Concretely: the agent takes an action (click button), the attacker observes and injects content (add fake popup), the agent observes the new state and responds. This game-theoretic framing lets you apply equilibrium concepts from economics — you’re looking for strategies where neither player can improve by changing their approach alone.

Framework Shift

Before (mainstream approach):        After (this paper):

Text-Only Defense                    Cross-Modal Co-Evolution
+------------------+                 +----------------------+
| Prompt Filter    |                 |  Agent  <---->  Attacker |
|       |          |                 |    ^              ^   |
|       v          |                 |    |              |   |
| Agent (VLM)      |                 |    v              v   |
|   |              |                 | Vision + Text State  |
|   v              |                 |   (both corrupted)   |
| Screenshot+Tree  |                 +----------------------+
+------------------+                 | 3-Stage Training:    |
                                     | 1. Imitation         |
Single modality                      | 2. Zero-Ack Oracle   |
defense, static                      | 3. GRPO Self-Play    |
                                     +----------------------+
                                     
                                     Dual modality, dynamic

From static text filtering to dynamic cross-modal co-evolution, the core shift is treating security as an ongoing game rather than a one-time inoculation.

Expert Assessment

Problem choice: This is a genuine gap. As web agents move from research demos to production, adversarial robustness becomes critical. The insight that DOM-level attacks corrupt both modalities simultaneously is sharp and underexplored. It sits at the intersection of multimodal learning and adversarial ML, both hot areas. The timing is right — web agents are emerging but not yet hardened.

Method maturity: The three-stage pipeline is well-motivated but feels somewhat engineered. Why these three stages in this order? The zero-acknowledgment strategy is clever and novel. The GRPO self-play is borrowed from existing RL literature but applied sensibly here. I’d call it 70% insight, 30% brute force. A simpler approach might be pure self-play from scratch, but the staged curriculum probably accelerates convergence.

Experimental integrity: The MiniWob++ results are convincing, and the out-of-distribution generalization is the real test. Doubling task completion while mitigating attacks is impressive. However, I’d want to see more analysis of failure modes — when does DMAST still fail? The baselines (prompt-based defenses, standard adversarial training) seem fair but not exhaustive. Missing: comparison to other game-theoretic training methods or multi-agent RL approaches. The numbers look solid but would benefit from error bars and significance tests.

Writing quality: The abstract and introduction are strong. The method section gets dense quickly — the Markov game formalization could be simplified or moved to an appendix. The zero-acknowledgment strategy deserves more intuitive explanation upfront before the math. The related work section feels perfunctory. If I were reviewing, I’d ask for a clearer walkthrough of one complete training episode showing how agent and attacker interact.

Verdict: weak accept — Addresses a real problem with a novel training strategy, shows strong empirical results, but the method complexity and limited baseline comparisons hold it back from strong accept.

Takeaways

The zero-acknowledgment strategy is immediately portable. Instead of training models to detect and reject adversarial inputs, train them to maintain task focus regardless of input perturbations. This applies beyond web agents — think content moderation (focus on policy violations, not on recognizing “attacks”), autonomous vehicles (focus on safe driving, not on detecting adversarial road signs), or medical diagnosis (focus on clinical reasoning, not on spotting data poisoning). The key insight: robustness through goal-oriented reasoning rather than threat detection.

The three-stage curriculum (imitation → oracle-guided SFT → adversarial RL) is a reusable template for any domain where you need to harden a model against adaptive adversaries. Start with behavioral cloning to bootstrap, use expert demonstrations to instill core principles, then let adversarial self-play refine both attacker and defender.

The cross-modal attack surface itself is a warning: as models consume multiple input modalities, attackers will exploit correlations between channels. If you’re building multimodal systems, test them against coordinated attacks across modalities, not just single-channel perturbations.