Stateful Monitoring Catches Distributed Agent Attacks Across User Accounts

Paper: 2605.31593 Authors: Davis Brown, Samarth Bhargav, Arav Santhanam, Kasper Hong, Ivan Zhang, Matan Shtepel, Steffi Chern, Alexander Robey, Eric Wong, Hamed Hassani Categories: cs.CR, cs.AI

The Gap

Language model safety monitors score individual conversations in isolation. They look at one user’s transcript, flag it as safe or harmful, and move on. This works when an attacker uses one account to do something bad. But attackers have figured out they can split a harmful task across many accounts—each conversation looks innocent, but together they accomplish the attack. Current monitors are structurally blind to this because they never look across users.

Prior work on agent misuse focuses on jailbreaks and prompt injection within single sessions. Multi-agent systems exist, but safety research treats each agent independently. No one has demonstrated that distributed attacks actually work, or built defenses that reason over groups of users rather than isolated transcripts.

Problem: Attacker splits harmful task across N accounts
   |
   v
Assumption: Weak signals exist in each transcript
   |          that correlate across accounts
   v
Method: Real-time clustering + cross-account LM judge
   |
   v
Evidence: Catches distributed attacks 30% earlier,
   |       5x better than per-session monitors
   v
Conclusion: Safety monitoring needs cross-user state

The Increment

One sentence: Before this paper, safety monitors were stateless and per-session; after, we have evidence that stateful cross-account monitoring catches a new class of distributed attacks that evade standard defenses.

Core Mechanism

The monitor maintains a rolling window of recent agent transcripts across all users. As new messages arrive, a lightweight classifier extracts suspiciousness signals—things like “user asked about vulnerabilities” or “agent discussed exploit code.” These signals are weak: individually they’re noisy and trigger on benign security research. But the key insight is that attackers reuse patterns across their distributed accounts.

The system clusters these signals in real-time using embeddings of the suspicious excerpts. When a cluster grows dense enough (many similar signals from different accounts in a short time window), the monitor escalates to a language model judge. This judge sees the aggregated evidence across accounts and decides whether it’s coordinated misuse or coincidental legitimate activity. The clustering acts as a filter: most benign traffic never reaches the expensive LM judge, keeping latency low.

User transcripts (streaming)
   |
   v
[Lightweight classifier] --> Suspiciousness signals
   |                          (per message, weak)
   v
[Real-time clustering] --> Group similar signals
   |                       across accounts
   v
[Density threshold] --> Trigger when cluster grows
   |
   v
[LM judge] --> Flag coordinated misuse
   |           (sees cross-account context)
   v
Alert or block

Think of it like airport security. A standard monitor is like checking each passenger’s bag individually—if one person carries a suspicious item, you catch them. But what if ten people each carry one innocent-looking component, and together they assemble something dangerous? You’d miss it unless you’re watching for patterns across passengers.

The stateful monitor is like having a system that notices “five people in the last hour all had metal tubes in their bags” and flags that correlation. The lightweight classifier is the X-ray machine (fast, noisy). The clustering is the pattern recognition (grouping similar findings). The LM judge is the human security officer who investigates only when the pattern looks serious (slow, accurate, used sparingly).

Key Concepts

• Distributed agent attack: An attacker controls multiple user accounts and splits a harmful task (like finding zero-day vulnerabilities) across them. Each subagent gets a limited, benign-looking subtask: “research authentication in this codebase,” “check for input validation here.” No single transcript reveals the harmful intent, but the attacker coordinates the subagents’ outputs offline to complete the attack. Standard monitors see each conversation as safe because they lack the full context.

• Stateful monitoring: Instead of scoring each conversation independently, the monitor maintains state across users and time. It remembers recent suspicious signals, clusters them, and reasons about whether patterns across accounts indicate coordinated misuse. “Stateful” means the decision for user A’s transcript can depend on what users B, C, and D did recently. This breaks the per-session isolation that lets distributed attacks hide.

• Weak signal clustering: Individual suspiciousness signals are noisy—lots of false positives from legitimate security researchers, students learning about exploits, etc. But attackers reuse attack strategies across their accounts, creating correlated signals. Clustering groups similar signals (using embeddings of the suspicious text) so the monitor can detect “ten accounts all asked about SQL injection in the last hour” even if each individual query looked borderline. The cluster’s density becomes the strong signal.

Framework Shift

Before (per-session monitoring):        After (stateful cross-account):

User A --> [Monitor] --> Safe/Harmful   User A --\
User B --> [Monitor] --> Safe/Harmful   User B ---+--> [Clustering] --> [LM Judge]
User C --> [Monitor] --> Safe/Harmful   User C --/         |              |
                                         User D ------------|              v
(Each decision independent)                            (Shared state)   Alert if
                                                                        coordinated

From isolated per-session scoring to shared state across users, the core shift is treating safety as a collective property rather than an individual one.

Expert Assessment

Problem choice: This is a real gap. The authors built the first working distributed agent attack and showed standard monitors catch it only 20% of the time (vs. 100% for non-distributed attacks). The threat model is credible: attackers already distribute manual hacking across accounts to avoid rate limits and detection. As agents get better at cybersecurity tasks, this will matter.

Method maturity: The approach is pragmatic rather than elegant. Real-time clustering with a density threshold is straightforward, and the two-stage design (cheap classifier + expensive LM judge) is standard in production systems. The novelty is applying this to cross-account agent monitoring, not inventing new ML techniques. That’s fine—the contribution is showing the problem exists and that a simple defense works. I’d like to see ablations on the clustering algorithm (why embeddings? what about graph-based methods?) but the core idea is sound.

Experimental integrity: The evaluation uses simulated datacenter traffic, which is both a strength and weakness. Strength: they can control the attack/benign ratio and measure false positive rates at scale. Weakness: real attacker behavior might differ from their threat model (e.g., longer time delays between subagent calls, more sophisticated obfuscation). The baselines are fair—they compare against standard per-session monitors and show Pareto dominance. The 30% earlier detection claim holds up in their experiments, but the advantage narrows as benign traffic scales, which they acknowledge honestly.

Writing quality: The paper front-loads the threat (distributed attacks exist and evade defenses) before introducing the solution, which is good structure. The related work section is thin—more comparison to anomaly detection and fraud detection systems would strengthen the positioning. The red-teaming section (adaptive attackers reusing jailbreaks across accounts) is a nice touch but feels like an afterthought; it deserves more prominence since it shows the defense generalizes beyond the original threat model.

Verdict: weak accept — Solid problem identification and a working defense, but the method is incremental and the evaluation is simulation-heavy. The contribution is more “this problem is real and here’s a reasonable first solution” than “here’s a breakthrough technique.”

Takeaways

For practitioners building agent systems: Implement two-stage monitoring. Run a fast, noisy classifier on every request, then escalate only high-density clusters to a slow, accurate judge. This keeps latency low for 99% of traffic while catching coordinated misuse.

For security researchers: Attackers will distribute. If your threat model assumes one account per attacker, you’re missing a large attack surface. Start logging cross-account patterns now, even if you’re not actively monitoring them yet.

For ML safety teams: Stateful monitoring is underexplored. Most safety research focuses on making individual model calls safer (RLHF, input filters, output classifiers). This paper shows you also need to reason about usage patterns across users and time. The clustering approach here is simple; there’s room for better methods that handle adaptive attackers who add noise to decorrelate their signals.

Transferable technique: The weak-signal-clustering pattern applies beyond agent safety. Fraud detection, spam filtering, and abuse detection all face similar problems: individual signals are noisy, but coordinated actors create correlations. If you’re building any system where attackers can split their activity across accounts, consider maintaining cross-user state and clustering suspicious events in real-time.