Verification as a Scaling Axis: LLM-as-a-Verifier's Probabilistic Approach to Solution Judging

Paper: 2607.05391 Authors: Jacky Kwok, Shulu Li, Pranav Atreya, Yuejiang Liu, Yixing Jiang, Chelsea Finn, Marco Pavone, Ion Stoica, Azalia Mirhoseini Categories: cs.AI, cs.CL, cs.LG, cs.MA, cs.RO

The Gap

The field has spent enormous effort scaling three axes: pre-training compute (bigger models), post-training compute (RLHF, DPO), and test-time compute (chain-of-thought, search, best-of-N sampling). All three share an implicit assumption: if you can generate better candidates, you’ll get better outputs. But there’s a bottleneck everyone runs into — how do you reliably pick the best candidate from a pool?

Standard LM judges (e.g., “rate this solution 1-10”) produce discrete, poorly calibrated scores. They struggle to distinguish between solutions that are subtly different in quality. When you ask an LLM to pick a number from 1 to 10, you’re compressing a rich internal representation into a handful of bits. The judgment signal is lossy, noisy, and hard to aggregate across multiple criteria.

This paper identifies verification as a fourth scaling axis and proposes a concrete mechanism to unlock it.

Problem: Hard to pick best solution from candidates
         |
         v
Assumption: LLMs have richer judgment signal than
            discrete scores capture
         |
         v
Method: Compute expected score from logit
        distribution (continuous, not discrete)
        + scale across 3 dimensions:
          granularity, repetition, decomposition
         |
         v
Evidence: SOTA on 4 benchmarks (SWE-Bench 78.2%,
          Terminal-Bench 86.5%, RoboReward 87.4%,
          MedAgent 73.3%)
         |
         v
Conclusion: Verification is a viable scaling axis
            independent of the other three

The Increment

One sentence: Before this paper, judging LLM outputs meant prompting for a discrete score and hoping for the best; after this paper, you can extract a continuous probability distribution over scores directly from the model’s logits, giving you calibrated, decomposable, and repeatable verification signals.

Core Mechanism

The framework has three main components working in sequence. First, criteria decomposition: instead of asking “how good is this solution overall?”, the system breaks the evaluation into specific sub-criteria (e.g., for code: correctness, efficiency, readability). Each sub-criterion gets its own evaluation pass. Second, probabilistic scoring: rather than prompting the LLM to output a discrete score token, the method examines the full probability distribution over candidate score tokens in the model’s output logits. It computes a weighted expectation — essentially asking “what score does this model *believe the solution deserves, on average?” Third, repeated evaluation: the same evaluation is run multiple times (with temperature sampling), and scores are averaged, reducing variance from the stochastic generation process.

The ranking algorithm then takes all these continuous scores across sub-criteria and evaluation runs, aggregates them, and selects the best candidate. The key insight is that continuous scores preserve more information than discrete ones — a score of 7.3 vs 7.5 carries meaning that “both are 7” would lose.

Solution + Evaluation Criteria
              |
              v
    [Criteria Decomposition]
              |
     +--------+--------+
     |        |        |
     v        v        v
 Sub-crit 1  Sub-crit 2  Sub-crit K
     |        |        |
     v        v        v
 [LLM generates score tokens]
     |        |        |
     v        v        v
 [Extract logit probs over
  score tokens: p(1)...p(10)]
     |        |        |
     v        v        v
 [E[Score] = sum(i * p(i))]
     |        |        |
     v        v        v
  s1_cont   s2_cont   sK_cont
     |        |        |
     +--------+--------+
              |
              v
    [Repeat N times, average]
              |
              v
    [Ranking Algorithm]
              |
              v
        Best Candidate

Think of it like Olympic diving judging. In the old approach, each judge holds up a single cardboard number — “7.5” — and that’s all you get. You don’t know if the judge was torn between 7 and 8, or confidently picking 7.5. In this paper’s approach, each judge holds up a semi-transparent gradient sheet: mostly concentrated around 7.5 but with a little shimmer at 7.0 and 8.0. You can see their uncertainty. Now imagine the judges evaluate not just “overall impression” but separate criteria — entry angle, splash size, body control — each with their own gradient sheet. And they evaluate the same dive three times, each time watching from a slightly different angle. The sheets overlap and combine, giving you a much richer picture than any single number could. The ranking algorithm is the head judge who synthesizes all those gradient sheets across all criteria and all viewings to declare a winner. The power is in the continuous signal: a gradient sheet preserves everything the judge actually “thinks,” while a cardboard number throws most of it away.

Key Concepts

• Logit Expectation over Score Tokens: When an LLM generates text, at each position it assigns a probability to every possible next token. If you ask it to rate something 1-10, it doesn’t just pick one number — it has a probability for each. The expected score is the weighted average: multiply each possible score by how likely the model thinks it is, then sum. So if the model thinks “7” has 40% probability, “8” has 35%, and “6” has 25%, the expected score is 0.4**7 + 0.358 + 0.256 = 7.15. This is richer than just picking the most likely token (which would give 7). The 7.15 tells you the model slightly leans toward higher scores — information lost in the discrete approach.

• Criteria Decomposition: Instead of one big judgment, break it into pieces. For evaluating a coding agent’s output, you might separately check: (1) does it compile? (2) does it pass edge cases? (3) is the complexity reasonable? (4) is the style clean? Each gets its own probabilistic score. This matters because an overall score of 7 could mean “perfect correctness, terrible style” or “mediocre at everything” — decomposition makes the feedback actionable.

• Verification as a Scaling Axis: The paper’s framing move. Previously, the community talked about three axes of scaling: more pre-training data, more post-training compute, more test-time compute. This paper argues verification is a fourth independent axis — you can improve system performance by making your *judge better, not just your generator. The axes are somewhat orthogonal: a better judge helps even with the same generator, and vice versa.

Framework Shift

Before (mainstream approach):        After (this paper):

  Solution A --> [LLM Judge] --> 7   Solution A --> [LLM Logits]
  Solution B --> [LLM Judge] --> 8        |          |
  Solution C --> [LLM Judge] --> 7        |          v
                                          |     [Score Distribution]
  Pick highest discrete score             |     p(1)...p(10)
                                          |          |
  Problem: Poor separation,               |          v
  poor calibration,                       |     [E[Score] = 7.31]
  single-shot, single-criterion           |          |
                                          +---> [Repeat x N]
                                          +---> [Decompose x K]
                                                |
                                                v
                                          [Rank by aggregated
                                           continuous scores]

From compressing judgment into a single integer to preserving the full probabilistic signal across multiple evaluation dimensions — the core shift is treating the judge’s output distribution as first-class information rather than collapsing it to a point estimate.

Expert Assessment

Problem choice: This is a real gap. The verification bottleneck is genuine — everyone doing best-of-N sampling, agentic pipelines, or RL with LLMs has felt the pain of unreliable judges. Positioning verification as a scaling axis is a useful framing, though one could argue it’s a repackaging of “reward modeling” with a different lens. The distinction matters though: reward models require training data and a separate model, while this uses the LLM itself as judge but extracts more signal from it. It sits at the intersection of the test-time compute scaling literature (Snell et al.) and the LM-as-judge literature, finding a concrete optimization point between them.

Method maturity: Clever rather than brute force. The core insight — use the full logit distribution instead of the argmax token — is simple and, in retrospect, obvious. That’s usually a sign of a good idea. The three scaling dimensions (granularity, repetition, decomposition) are each individually known, but combining them under a unified probabilistic framework is the contribution. One concern: the method still relies on the LLM’s own calibration being reasonable for scoring tasks, which is not guaranteed across domains. Simpler approaches like majority voting or trained verifiers might still win in narrow domains where training data is abundant.

Experimental integrity: Strong on breadth — four diverse benchmarks (code, terminal, robotics, medical). SWE-Bench Verified at 78.2% is competitive but note this uses Claude 3.5 Sonnet as the base model, which is already a strong baseline. The ablation studies showing gains from each scaling dimension are convincing. The RL results (SAC and GRPO improvements) are interesting but feel somewhat appended — they’d benefit from more rigorous comparison against dedicated reward models. The Claude Code extension is a nice practical touch but reads more as a demo than a rigorous evaluation.

Writing quality: Clear and well-structured. The paper does a good job motivating verification as a scaling axis and explaining the probabilistic formulation accessibly. The weakest section is the RL application — it feels rushed and underdeveloped compared to the thorough verification experiments. The related work could be sharper in distinguishing from prior LM-as-judge approaches; the distinction sometimes blurs. A tighter Section 5 that either goes deeper on RL or is saved for a follow-up would strengthen the paper.

Verdict: weak accept — The core idea (logit expectation for continuous scoring) is clean, practical, and broadly applicable. The framing as a “new scaling axis” is slightly oversold but useful. The experiments are solid if not extraordinary. Main concern is whether the gains hold up when the base model changes or when domain-specific reward models are available.

Takeaways

Three concrete things to steal:

1. The logit expectation trick: Whenever you use an LLM as a judge, don’t take the argmax token. Instead, compute the expected value from the probability distribution over score tokens. This is a drop-in improvement that requires zero additional training — just a few lines of code to extract and weight the logit probabilities. Apply this to any evaluation pipeline that asks an LLM to score something.

2. Decompose before aggregating: If you’re evaluating something complex, split the evaluation into independent criteria, score each separately, then combine. This gives you both better scores (less variance per criterion) and actionable feedback (you know *why something scored high or low). This transfers directly to any quality assurance or review process.

3. Use continuous scores as RL rewards: The paper shows that continuous, fine-grained verification signals work better as reward signals for reinforcement learning than discrete judgments. If you’re doing RL with LLMs, consider replacing your binary pass/fail reward with a continuous verification score — it provides denser supervision and improves sample efficiency.