Confidence Has a Shape: Consilience for Verifier-Free Test-Time Scaling

Paper: 2608.09898 Authors: Lecheng Kong, Like Hui, Haitao Mao, Jun Huan Categories: cs.CL, cs.LG

The Gap

Test-time scaling is simple in principle: sample N rollouts, keep the good one. The whole difficulty is “keep the good one.” With a verifier — a compiler, a unit test suite, a learned value function — the problem is basically solved. Without one, you need a proxy signal, and the field has converged on a small menu: majority voting / self-consistency (needs an extractable, comparable answer, so it dies on free-form code), LLM-as-judge (another full forward pass per candidate, and the judge inherits the generator’s blind spots), and confidence-based scoring — rank rollouts by mean token log-probability, by perplexity, by self-certainty, or by the lowest-confidence window inside the trace (the DeepConf family).

Confidence-based scoring is the attractive branch because it is nearly free. You already have the logits. No extra sampling, no extra model, no task-specific parsing. It works across models and tasks with minimal access to internals.

This paper’s contribution is to show where that branch breaks, and it breaks in a specific and interesting way. On hard problems, ranking by aggregate confidence does not just degrade toward random — it actively selects wrong answers. The reason: for a difficult question, a rollout with uniformly high confidence from token one is a rollout that never considered an alternative. Flat high confidence is a signature of failed exploration, not of competence. Aggregate-confidence scoring is therefore systematically biased toward the confidently wrong.

The proposed fix is to stop treating confidence as a scalar summary and start treating it as a trajectory with a diagnostic shape: doubt early, certainty late.

[ Problem ]
  no verifier at test time
  must pick 1 of N rollouts using only internal signals
      |
      v
[ Prior art ]
  score = aggregate over all tokens
  mean logprob / perplexity / self-certainty / min-window conf
      |
      v
[ Observed breakdown ]
  on hard tasks the winners are the flat-high-confidence rollouts
  those are exactly the ones that never branched
  ==> confidently wrong answers get promoted
      |
      v
[ Assumption ]
  good cognitive search has a SHAPE
  low confidence early: exploratory branching
  high confidence late: convergence on a solution
      |
      v
[ Method: consilience ]
  split trace into early and late
  combinatorial score that punishes early certainty
  and strictly demands late certainty
      |
      v
[ Evidence ]
  graduate-level mathematics + free-form code generation
  outperforms confidence and voting baselines
      |
      v
[ Conclusion ]
  the informative part of confidence is its temporal asymmetry
  not its average

The Increment

One sentence: Before, verifier-free selection asked “how confident is this rollout”; after, it asks “did this rollout earn its confidence” — the same logits, read as a time series instead of a number.

Core Mechanism

Start where every confidence method starts. Sample N rollouts for a prompt. For each rollout you have a per-token confidence series c(t) — some monotone function of the token distribution, typically the log-probability of the emitted token or a negative-entropy style self-certainty. Prior methods collapse c(t) into one number: mean, min, or the minimum over sliding windows. Collapsing is exactly where the information dies, because mean and min are order-invariant. A trace that doubts then commits and a trace that commits then doubts get identical scores.

Consilience keeps the ordering. Split the trace into a head region and a tail region, aggregate confidence separately in each, and combine them asymmetrically: the tail term rewards high confidence (the model must actually land somewhere), while the head term is penalized for high confidence (a model that was sure before it did any work is suspect). The abstract describes this as a combinatorial metric that “actively penalizes high initial confidence while strictly demanding final certainty” — read it as a monotone combination S = g(tail) - lambda * h(head), where the “strictly demanding” phrasing suggests the tail acts closer to a hard gate than a soft additive term. Rank by S, return the argmax. Cost over vanilla best-of-N confidence: one extra pass over an array you already have.

The critical structural point is that this comparison happens within a question. All N rollouts face the same problem, so difficulty is held fixed. That matters, because across questions “low early confidence” mostly measures how hard the question is — an easy arithmetic problem should be answered confidently from token one, and penalizing that would be nonsense. Within a fixed question, low early confidence measures something else: which of these N attempts actually opened up the space before closing it.

prompt
  +--> rollout 1 : c1 c2 c3 ... cT
  +--> rollout 2 : ...
  +--> rollout N : ...

read each rollout as a confidence trajectory

  conf
  1.0 |                  ...._____   commit
      |          ___/~~~~
  0.5 |     ___/
      |  __/    doubt
  0.0 +--------------------------------> t
        GOOD SHAPE: explored then converged

  conf
  1.0 |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  0.5 |
  0.0 +--------------------------------> t
        BAD SHAPE: never doubted
        flat high == no branching == likely wrong on hard tasks

  conf
  1.0 |  ___
      | /   \____                 ....
  0.5 |            \___     ..../
  0.0 +--------------------------------> t
        BAD SHAPE: ends hedging == no convergence

scoring:
  HEAD = early segment of c(t)
  TAIL = late segment of c(t)

  A = agg( HEAD )    "was it sure before it thought"
  B = agg( TAIL )    "did it land on something"

  S = g( B )  -  lambda * h( A )
      ^^^^^^         ^^^^^^^^^^^
      demand         penalize

rank by S  ==>  argmax  ==>  final answer

The metaphor: this is a detective case review, and the review board never gets to check whether the arrest was actually correct.

Each rollout is a detective who worked the same case. The board cannot verify the verdict — no confession, no DNA, no verifier. All it has is each detective’s own log of how certain they felt, hour by hour.

The old policy was to promote whichever detective was most confident on average across the whole investigation. On a routine case that is fine. On a genuinely hard case it selects for the worst possible officer: the one who walked into the room, decided it was the husband, and never wavered. Nothing in his log ever dips, so his average certainty is the highest in the department. His flat log is not evidence of skill; it is evidence that he never canvassed anyone else.

Consilience reads the log’s shape. A good detective’s certainty starts low — three suspects on the board, contradictory timelines — and ends high, because independent threads eventually pointed the same way. That is what the word means, in Whewell’s original sense: independent lines of inference jumping together. So the board applies two rules. First, hedging at the end disqualifies you; an investigation that closes with “probably the husband, maybe the brother” is not a solved case. Second, certainty at the start counts against you, because on a hard case, day-one certainty is tunnel vision wearing the costume of competence. The head term is the tunnel-vision penalty; the tail term is the closed-case requirement; lambda is how harshly the board treats a cop who never doubted.

Key Concepts

• Token confidence: When an LLM writes a token, it first produces a probability over every possible next token. If it puts 0.99 on “4” after “2 + 2 =”, it is confident. If the top three candidates each get roughly 0.3, it is not. Confidence is just that number, tracked token by token, and averaged or otherwise summarized. The thing to internalize is what it actually measures: how predictable the *next word was given everything written so far. It is a fluency measurement, not a truth measurement. A model marching down a wrong but internally coherent chain of reasoning is highly confident at every step, because each wrong step follows smoothly from the last wrong step. This is precisely why average confidence fails as a correctness proxy on hard problems.

• Temporal asymmetry: Take a rollout’s confidence trace and reverse it. Mean confidence: unchanged. Minimum confidence: unchanged. Perplexity: unchanged. Every mainstream confidence score is blind to time order — which means it throws away the entire distinction between “figured it out” and “fell apart.” Temporal asymmetry is any score that changes when you reverse the sequence. The paper’s specific claim is narrower and testable: the useful asymmetry is rising, and rising specifically from a low start. Concretely, on a hard geometry problem, rollout A opens with “hmm, could be similar triangles, or maybe coordinates” (low confidence) and closes with a clean numeric answer (high confidence); rollout B opens with “The answer is clearly 12 by symmetry” and stays smooth throughout. Old scoring picks B. Consilience picks A.

• Verifier-free vs. verifier-based: With a verifier, test-time scaling is a search problem with ground truth at the leaves — run the unit tests, keep what passes. Verifier-free means the only oracle you have is the model’s own internal state, which is correlated with correctness but also correlated with familiarity, template-ness, and how confidently it can be wrong. Every VF-TTS paper is fundamentally an argument about which internal statistic is least contaminated by those confounds. This paper’s argument: not the level, the slope.

Framework Shift

Before (mainstream confidence VF-TTS):   After (consilience):

  rollout: [==================]            rollout: [=====|........|=====]
                    |                                HEAD           TAIL
                    v                                  |              |
       collapse to a scalar                            v              v
       mean / min / perplexity                   penalize high    demand high
       (order-invariant)                                 \           /
                    |                                     \         /
                    v                                       v     v
       pick max total confidence                       S = shape match
                    |                                          |
                    v                                          v
       flat-high rollout wins                     explored-then-converged wins
       == the one that never                      == the one whose certainty
          explored                                   was earned

  confidence as a LEVEL                     confidence as a TRAJECTORY

From “how sure is the model” to “how did the model become sure,” the core shift is treating the derivative of confidence as the signal and the level as merely its endpoint.

Expert Assessment

Problem choice: Real gap, well-placed. Confidence-based selection is currently the default cheap knob for verifier-free scaling, and “it degrades on hard problems” is a live, widely-felt failure. The diagnosis — flat high confidence as a marker of absent exploration — is a genuinely nice observation and is worth the paper on its own even if the metric does not survive. It also sits at a productive intersection with the reasoning-model literature, where backtracking behavior (“wait,” “alternatively”) is now understood as load-bearing. Consilience is essentially a logit-level proxy for that behavior. The risk is that the paper is one observation plus one heuristic, which is a thin core for the framing weight it carries.

Method maturity: Clever, but closer to a well-chosen feature than to a framework. Two things nag. First, the obvious cheaper competitor: just count backtracking markers in the trace, or measure confidence variance, or fit a slope. If “number of ‘wait’ tokens” recovers most of the gain, consilience is an elaborate way to detect exploration. The paper needs that ablation, and the abstract does not tell me whether it exists. Second, defining “initial” is genuinely delicate. The opening tokens of any CoT are boilerplate — “Okay, so I need to find the value of…” — which is near-deterministic and therefore near-maximally confident regardless of whether real exploration follows. Any head window that includes the preamble measures template-ness, not doubt. Where exactly the head starts, and how sensitive results are to that choice, is the load-bearing detail. I would also want to know how many hyperparameters (window sizes, lambda, aggregation choices) were tuned, and whether tuning was done on held-out tasks. A shape-matching score with three knobs can be fit to any two benchmarks.

Also, the name is a slight misdirection. Consilience in Whewell’s sense is independent lines of evidence converging — which sounds like cross-rollout agreement, i.e. self-consistency. The actual mechanism is within-rollout temporal shape. Evocative, but it will confuse readers.

Experimental integrity: Two task families (graduate-level math, free-form code generation) with the code setting well chosen, since that is exactly where majority voting is unavailable and the method has room to matter. The claim I would press hardest is “catastrophically break down,” which demands a difficulty sweep — show me the same benchmark stratified easy-to-hard and the crossover point where aggregate confidence goes from helping to actively hurting. Without that, “catastrophic” is rhetoric. The baseline I most want to see is confidence-weighted majority voting, and better still, consilience used *as the weight in voting rather than as a hard argmax; if the authors compared only against plain best-of-N confidence variants, the comparison is soft. Also worth checking: length confounds (longer traces both explore more and have more low-confidence tokens) and whether gains persist at large N or wash out. Note that I am reading the abstract, not the tables, so treat these as the questions I would take into review rather than as findings.

Writing quality: The corner-cutting is visible from the abstract alone: “a combinatorial metric” is doing a lot of work while telling the reader nothing. The single highest-leverage rewrite would be a short, explicit section giving the formula, the window definitions, the hyperparameter values, and a sensitivity plot — that one section converts the paper from “interesting story with a heuristic” to something reproducible. Second-highest: replace the assertion of breakdown with a difficulty-stratified figure. The phenomenology is the strongest asset here and it deserves a figure, not an adjective.

Verdict: weak accept — the diagnostic insight is sharp and cheap to adopt, but the metric looks hand-shaped and the evidence base is thin relative to the strength of the claims.

Takeaways

Things that transfer:

1. Any scalar you compute over a trace is throwing away order. Mean, min, max, and perplexity are all reversal-invariant. If you are ranking agent trajectories, RL rollouts, retrieval chains, or tool-use episodes by an averaged score, you are unable to distinguish “converged” from “fell apart.” Adding head-vs-tail split features is close to free and can be bolted onto whatever scorer you already run.

2. High uniform confidence is a warning light, not a green light. This is directly usable as a monitoring signal in agentic loops: a sub-agent whose confidence never dips on a hard subtask probably never considered an alternative. Route those for re-sampling or escalation. You do not need the paper’s exact metric for this — flat-and-high is a cheap red flag.

3. Compare within-item, not across-item. The whole method only makes sense because all N rollouts share one prompt, which holds difficulty constant. This is a general recipe for making badly-calibrated internal signals usable: find a comparison set where the dominant confound is fixed, then read the residual variation.

4. Asymmetric scoring beats symmetric aggregation when the good and bad regimes have different signatures. Demanding high late confidence while penalizing high early confidence is a template you can reuse anywhere a process should start uncertain and end certain — search, calibration schedules, annealing diagnostics, curriculum monitoring.

5. For RL people: this suggests a trajectory-shape reward or filter. Preferring rollouts whose confidence rises could bias policies toward exploratory-then-decisive reasoning, which is a different target from rewarding final correctness alone. Speculative, and the paper does not do it, but it is the obvious next move.