LCGuard: Protecting Privacy in Multi-Agent LLM Systems Through Adversarial KV Cache Transformation

Paper: 2605.22786 Authors: Sadia Asif, Mohammad Mohammadi Amiri, Momin Abbas, Prasanna Sattigeri, Karthikeyan Natesan Ramamurthy Categories: cs.AI, cs.ET, cs.LG, cs.MA

The Gap

Multi-agent LLM systems have moved beyond natural language communication to sharing transformer KV caches directly—a more efficient channel that preserves richer task information. But here’s the problem: KV caches aren’t just task data. They encode the full context that generated them: sensitive inputs, intermediate reasoning, agent-specific information. When Agent A passes its KV cache to Agent B, it’s handing over an opaque memory dump that could leak private data without any textual trace.

Prior work on latent communication (KV sharing) focused on efficiency and task performance. Privacy? Unaddressed. Existing privacy techniques for LLMs target text outputs (differential privacy, output filtering) or model weights (federated learning). None of them handle the representation-level leakage problem when raw internal states move between agents.

Problem: KV caches leak sensitive info
    |
    v
Assumption: Leakage = reconstructability by adversary
    |
    v
Method: Adversarial training (sanitizer vs reconstructor)
    |
    v
Evidence: Lower reconstruction accuracy + maintained task performance
    |
    v
Conclusion: Safe KV sharing is feasible

The Increment

One sentence: Before this paper, sharing KV caches meant choosing between efficiency and privacy; after it, you can sanitize caches adversarially to get both.

Core Mechanism

LCGuard sits between agents as a learned transformation layer. When Agent A wants to share its KV cache with Agent B, LCGuard intercepts the cache and applies a representation-level transformation before transmission. The transformation has two competing objectives: preserve enough information for Agent B to complete the task, but destroy enough structure that an adversary can’t reconstruct Agent A’s sensitive inputs.

The system trains two neural networks in opposition. The sanitizer (LCGuard itself) learns transformations that map raw KV caches to sanitized versions. The adversarial reconstructor learns to recover sensitive inputs from sanitized caches. They play a minimax game: the reconstructor tries to maximize reconstruction accuracy, the sanitizer tries to minimize it while keeping task performance high. At equilibrium, the sanitizer has learned transformations that are useful for downstream tasks but useless for privacy attacks.

Agent A                    Agent B
   |                          ^
   | raw KV cache             | sanitized KV cache
   v                          |
[Sanitizer] <--adversarial--[Reconstructor]
   |            training         |
   +------------------------------+
        tries to recover
        sensitive inputs

Think of LCGuard as a diplomatic translator between two countries. Agent A speaks a language that contains state secrets embedded in its grammar and idioms. Agent B needs to understand the task-relevant message but shouldn’t learn the secrets. The sanitizer is the translator who learns to rephrase messages in a way that conveys the operational content but strips out the cultural/historical context that would reveal secrets. The reconstructor is a spy trying to reverse-engineer the original language from the translations. The translator gets better by studying what the spy can infer, then adjusting the translation protocol to close those inference channels. Eventually, the translations are operationally useful but cryptographically opaque to reconstruction attacks.

Key Concepts

• Reconstruction-based leakage: Privacy isn’t about what the cache “contains” in some abstract sense—it’s about what an adversary can extract. LCGuard defines leakage operationally: a cache is unsafe if an adversarial decoder can reconstruct sensitive inputs from it with high accuracy. This shifts the problem from “hide information” (vague) to “minimize reconstruction loss” (measurable). The adversarial reconstructor serves as a worst-case privacy auditor: if it can’t recover secrets, neither can real attackers with similar capabilities.

• Latent working memory: The paper treats shared KV caches as a form of working memory—temporary storage that agents use to coordinate without committing everything to explicit text. This framing is key: working memory should be ephemeral and task-specific, not a permanent record. LCGuard’s transformations act like a “forget gate” that lets task-relevant patterns through while erasing agent-specific traces. The cache becomes a shared scratchpad rather than a surveillance log.

• Adversarial training for privacy: Most privacy techniques add noise (differential privacy) or filter outputs (content moderation). LCGuard uses adversarial training: the sanitizer learns by fighting a reconstructor that actively searches for leakage channels. This is fundamentally different from static defenses. The reconstructor doesn’t just check if secrets are present—it learns the optimal attack strategy, forcing the sanitizer to close vulnerabilities it wouldn’t anticipate on its own. The result is a defense that adapts to the attack surface rather than guessing at it.

Framework Shift

Before (standard KV sharing):        After (LCGuard):

Agent A --[raw KV cache]--> Agent B  Agent A --[raw KV]--> [Sanitizer] --[clean KV]--> Agent B
    |                          |                                |
    +-- leaks context,         +-- preserves task info,         +-- adversarial
        reasoning, secrets         blocks reconstruction             reconstructor
                                                                     audits leakage

From transparent memory sharing to adversarially sanitized communication, the core shift is treating privacy as a learned transformation rather than a fixed filter.

Expert Assessment

Problem choice: Real gap. Multi-agent LLM systems are moving toward latent communication for efficiency, but the privacy implications are genuinely underexplored. This isn’t a manufactured problem—KV caches do encode sensitive context, and there’s no existing framework for sanitizing them. The timing is good: the problem is emerging but not yet crowded.

Method maturity: Solid adversarial formulation, but the approach is somewhat brute-force. The minimax game is well-motivated, but the paper doesn’t explore whether simpler transformations (e.g., dimensionality reduction, noise injection) could achieve similar privacy-utility tradeoffs with less computational overhead. The adversarial reconstructor is a strong privacy auditor, but it’s also a strong assumption—real attackers may not have the same training access. The paper acknowledges this but doesn’t test against weaker threat models.

Experimental integrity: Baselines are fair (standard KV sharing, no sharing). The reconstruction metrics (accuracy, BLEU, perplexity) are appropriate. Task performance comparisons are honest—LCGuard does incur some performance cost, and the paper reports it. One concern: the adversarial reconstructor is trained on the same distribution as the sanitizer. In practice, attackers might use different architectures or training strategies. The paper would be stronger with transferability experiments (train reconstructor on one model family, test on another).

Writing quality: The formalization is clear, but Section 3 (method) is dense and could benefit from a running example. The related work section is thorough but reads like a literature dump—cutting it by 30% and integrating key comparisons into the method section would improve flow. The experimental section is well-structured, but the ablation studies are buried in the appendix when they should be front and center.

Verdict: weak accept — Addresses a real emerging problem with a principled adversarial approach, but the method is somewhat heavy-handed and the evaluation could stress-test the assumptions more rigorously.

Takeaways

Operational privacy definition: The reconstruction-based leakage metric is transferable. If you’re building any system that shares learned representations (embeddings, hidden states, gradients), you can operationalize privacy as “can an adversary reconstruct sensitive inputs?” and use adversarial training to minimize it. This beats vague notions of “information hiding.”

Adversarial auditing: The reconstructor isn’t just a training opponent—it’s a privacy auditor. You can use this pattern in other domains: train an adversarial decoder to extract what you want to hide, then measure your defense’s effectiveness by how badly the decoder fails. This gives you a quantitative privacy metric instead of relying on intuition.

Latent communication as a design pattern: The broader idea—agents sharing internal states instead of text—is worth stealing. If you’re building multi-agent systems, consider whether latent communication (embeddings, attention maps, KV caches) could replace some natural language exchanges. It’s more efficient and preserves richer structure. Just remember: you’ll need something like LCGuard to keep it safe.