Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalization

Paper: 2602.15028 Authors: Shangding Gu Categories: cs.LG, cs.AI

Abstract

As large language models are deployed in increasingly privacy-sensitive and personalization-critical applications, understanding how context length affects their behavior becomes crucial. This paper presents PAPerBench, a comprehensive benchmark with ~29,000 instances spanning context lengths from 1K to 256K tokens (377K total evaluation questions), designed to jointly assess personalization quality and privacy leakage in LLMs. The evaluation reveals a consistent and counterintuitive finding: both personalization performance and privacy protection degrade as context length increases across state-of-the-art models. The authors provide theoretical analysis attributing this phenomenon to attention dilution in fixed-capacity Transformers, where soft attention mechanisms struggle to maintain focus as context scales. This “long context, less focus” scaling gap represents a fundamental limitation in current architectures.

Key Contributions

• Introduction of PAPerBench, the first large-scale benchmark jointly evaluating personalization and privacy across varying context lengths (1K-256K tokens)
• Empirical demonstration that both personalization effectiveness and privacy protection degrade consistently as context length increases across multiple state-of-the-art LLMs
• Theoretical analysis of attention dilution mechanisms, explaining performance degradation as an inherent limitation of soft attention in fixed-capacity Transformers
• Identification of a fundamental “scaling gap” in current LLM architectures where longer contexts lead to reduced focus and capability
• Open-source release of benchmark data and evaluation code to enable reproducible research

The PAPerBench Benchmark Design

PAPerBench is structured to systematically probe the intersection of context length, personalization, and privacy. The benchmark contains approximately 29,000 base instances, each evaluated at multiple context lengths, generating 377K total evaluation questions. Context lengths range from 1K tokens (short context) to 256K tokens (ultra-long context), covering the spectrum of modern LLM capabilities.

The benchmark evaluates two critical dimensions simultaneously:

Personalization Performance: Measures how well models utilize user-specific information embedded in context to provide tailored responses. Tasks include preference-based recommendations, user history-aware suggestions, and context-dependent decision making.

Privacy Protection: Assesses whether models inadvertently leak sensitive information from the context when responding to queries. This includes direct information extraction attacks, inference attacks, and membership inference scenarios.

The dual evaluation framework enables controlled analysis of the trade-offs and correlations between these two objectives as context scales.

Empirical Findings: The Degradation Pattern

Extensive evaluation across state-of-the-art LLMs reveals a striking and consistent pattern: both personalization quality and privacy protection deteriorate as context length increases. This finding holds across different model families, sizes, and architectures.

For personalization tasks, models show progressively weaker ability to leverage user-specific information as context grows. At 1K tokens, models effectively utilize personal preferences and history. However, at 128K-256K tokens, the same information becomes effectively “invisible” to the model, resulting in generic responses that ignore user context.

Privacy metrics show a parallel degradation. Shorter contexts enable models to better compartmentalize and protect sensitive information. As context expands, the probability of privacy leakage increases—models become more likely to inadvertently expose private details when answering queries, even when such information should remain confidential.

The correlation between these two degradation patterns suggests a common underlying mechanism rather than independent failure modes.

Theoretical Analysis: Attention Dilution

The paper provides theoretical grounding for the empirical observations through analysis of attention mechanisms in Transformers. The core insight centers on attention dilution under context scaling.

In standard Transformer architectures, attention weights are computed via softmax over all context positions:

$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$

As context length $L$ increases, the attention distribution becomes increasingly diffuse. For a fixed model capacity (fixed number of parameters and attention heads), the attention “budget” must be distributed across more positions. This creates a fundamental tension: longer contexts require more selective attention, but the soft attention mechanism inherently spreads probability mass across all positions.

The authors formalize this as attention dilution: even when relevant information exists in context, the attention weight assigned to critical tokens decreases as $O(1/L)$ in the worst case. This dilution effect is particularly pronounced for:

• Rare or unique information (like personal details) that appears infrequently in context
• Information that requires precise retrieval rather than aggregate understanding
• Tasks requiring fine-grained discrimination between similar context elements

The theoretical analysis suggests this is not merely an optimization or training issue, but an architectural limitation of soft attention in fixed-capacity models. As context scales, the model’s effective “focus” diminishes, leading to both weaker personalization (inability to attend to user-specific details) and weaker privacy (inability to selectively withhold sensitive information).

Implications and Future Directions

The “long context, less focus” phenomenon has significant implications for LLM deployment and development:

For Privacy-Critical Applications: The finding that privacy protection degrades with context length challenges the assumption that more context is always better. Applications handling sensitive data may need to carefully limit context length or employ alternative architectures.

For Personalization Systems: The degradation in personalization effectiveness suggests that simply extending context windows is insufficient for improving user-specific adaptation. New mechanisms for selective attention or explicit memory structures may be necessary.

For Architecture Design: The theoretical analysis points toward fundamental limitations of soft attention that cannot be resolved through scaling alone. Future architectures may need sparse attention patterns, explicit retrieval mechanisms, or hybrid approaches combining parametric and non-parametric memory.

For Evaluation Methodology: PAPerBench demonstrates the importance of joint evaluation across multiple objectives. Optimizing for context length alone without considering downstream task performance and safety properties can lead to misleading conclusions about model capabilities.

The benchmark and findings open several research directions: developing attention mechanisms that maintain focus at scale, designing architectures with explicit privacy-preserving components, and creating training objectives that balance context utilization with selective information processing.

Takeaways

1. Both personalization quality and privacy protection in LLMs degrade consistently as context length increases from 1K to 256K tokens
2. This degradation pattern holds across multiple state-of-the-art models, suggesting a fundamental architectural limitation rather than model-specific issues
3. Attention dilution in fixed-capacity Transformers provides theoretical explanation for the empirical observations—soft attention spreads focus too thinly across long contexts
4. The “long context, less focus” scaling gap represents a critical challenge for deploying LLMs in privacy-sensitive and personalization-critical applications
5. PAPerBench provides a reproducible benchmark with 377K evaluation questions for future research on scalable privacy and personalization in long-context models