Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Paper: 2602.21189 Authors: Anas Barakat, Souradip Chakraborty, Khushbu Pahwa, Amrit Singh Bedi Categories: cs.LG, cs.AI

Abstract

Pass@k is a critical evaluation metric for verifiable LLM tasks such as mathematical reasoning, code generation, and short-answer problems, where success is achieved if any of $k$ independently sampled solutions passes verification. While this metric has inspired inference-aware fine-tuning methods that directly optimize pass@k performance, a persistent paradox has emerged: improvements in pass@k often come at the cost of degraded pass@1 performance. This paper provides the first theoretical characterization of this trade-off, demonstrating that pass@k policy optimization can reduce pass@1 through gradient conflicts induced by prompt interference. The authors show that pass@k optimization implicitly reweights the training distribution toward low-success prompts, and when these prompts exhibit negative interference properties, the resulting gradient updates can diverge from the pass@1 optimization direction.

Key Contributions

• Theoretical Framework: Establishes a formal characterization of when and why pass@k optimization degrades pass@1 performance through gradient conflict analysis
• Prompt Interference Concept: Introduces the notion of “negatively interfering prompts” that cause gradient directions to diverge between pass@k and pass@1 objectives
• Implicit Reweighting Mechanism: Reveals that pass@k optimization inherently upweights difficult, low-success prompts in the gradient computation
• Empirical Validation: Demonstrates theoretical predictions on mathematical reasoning tasks using large language models

The Pass@k vs Pass@1 Trade-off

The pass@k metric evaluates whether at least one solution among $k$ samples succeeds, making it particularly relevant for scenarios where multiple attempts are feasible. However, pass@1 remains operationally critical due to:

• Latency constraints: Generating multiple samples increases inference time
• Cost considerations: Each additional sample incurs computational expense
• Verifier limitations: Not all tasks have perfect automated verifiers
• Fallback requirements: Systems need reliable single-shot performance

The paper formalizes this tension by analyzing the policy gradient objectives. For pass@1, the gradient encourages increasing the probability of correct solutions. For pass@k, the gradient is weighted by $(1 - p_{\theta}(x))^{k-1}$, where $p_{\theta}(x)$ is the success probability. This weighting factor is larger for prompts with lower success rates, effectively prioritizing improvement on difficult examples.

Gradient Conflict and Prompt Interference

The core theoretical insight centers on gradient conflict between pass@k and pass@1 objectives. The authors prove that pass@k gradients can point away from pass@1 gradients when:

$\cos(\theta_{k,1}) = \frac{\langle \nabla_{\theta} \text{pass@k}, \nabla_{\theta} \text{pass@1} \rangle}{||\nabla_{\theta} \text{pass@k}|| \cdot ||\nabla_{\theta} \text{pass@1}||} < 0$

This occurs through prompt interference: when optimizing on a mixture of prompts, gradients from different prompts can conflict. Specifically, “negatively interfering prompts” are those where improving performance on one prompt degrades performance on another. Since pass@k optimization upweights low-success prompts, if these prompts negatively interfere with high-success prompts, the overall gradient update can harm pass@1 performance.

The mathematical formulation shows that the pass@k gradient can be decomposed as:

$\nabla_{\theta} \text{pass@k} = \sum_{i} w_i^{(k)} \nabla_{\theta} p_i$

where $w_i^{(k)} \propto (1 - p_i)^{k-1}$ assigns higher weight to prompts with lower success probability $p_i$. When these heavily weighted prompts have gradients that conflict with the pass@1 direction, degradation occurs.

Experimental Validation

The authors validate their theoretical predictions through experiments on mathematical reasoning tasks using large language models. Key findings include:

• Observed Trade-off: Pass@k optimization consistently improves pass@k metrics while degrading pass@1 performance across different model sizes and datasets
• Gradient Analysis: Empirical measurements confirm negative cosine similarity between pass@k and pass@1 gradients during training
• Prompt-Level Effects: Low-success prompts indeed receive disproportionate gradient weight, and their optimization conflicts with high-success prompt optimization
• Magnitude of Degradation: The pass@1 degradation is not negligible, often representing several percentage points of accuracy loss

Takeaways

1. Optimizing pass@k directly can harm pass@1 performance due to fundamental gradient conflicts, not implementation issues or hyperparameter choices.

2. The root cause is prompt interference: pass@k optimization implicitly reweights toward difficult prompts, and when these prompts negatively interfere with easier ones, gradient directions diverge.

3. Pass@1 remains practically essential despite the appeal of pass@k metrics, as real-world deployments face latency, cost, and verifier coverage constraints.

4. Future optimization methods should explicitly address this trade-off, potentially through multi-objective optimization, prompt-aware gradient balancing, or constrained optimization that maintains pass@1 performance.

5. The theoretical framework generalizes beyond specific tasks, applying to any verifiable LLM application where multi-sample inference is considered.