CoDE-Stop: When to Stop Thinking by Watching Confidence Dance

Paper: 2604.04930 Authors: Parsa Hosseini, Sumit Nawathe, Mahdi Salmani, Meisam Razaviyayn, Soheil Feizi Categories: cs.CL, cs.AI, cs.LG

The Gap

Large reasoning models like o1 and DeepSeek-R1 generate long chain-of-thought sequences to solve complex problems. Current approaches either let models reason to a fixed maximum length or use simple heuristics to stop early. The problem: correct solutions often emerge early but models keep generating tokens, while incorrect reasoning paths spiral into unproductive loops. Prior early stopping methods (like reward model scoring or fixed-length cutoffs) don’t leverage the temporal dynamics of how confidence evolves during reasoning.

Problem: Wasted computation in long reasoning chains
    |
    v
Observation: Confidence behaves differently in correct vs incorrect paths
    |
    +---> Correct: High confidence reached early, then stable
    |
    +---> Incorrect: Low confidence, erratic fluctuations
    |
    v
Method: Track confidence dynamics, stop when pattern indicates answer found
    |
    v
Evidence: 25-50% token reduction, maintained accuracy across benchmarks
    |
    v
Conclusion: Confidence trajectory is a reliable early stopping signal

The Increment

One sentence: Before this paper, models reasoned to fixed lengths or used static scoring; after, we can watch confidence dynamics in real-time to stop reasoning the moment a stable answer emerges.

Core Mechanism

CoDE-Stop monitors intermediate answers during chain-of-thought generation. At each reasoning step, the model can produce a candidate answer. The method extracts confidence scores for these intermediate answers and tracks how confidence changes over time. When confidence reaches a high threshold and stabilizes (stops increasing significantly), reasoning terminates.

The system has three components: (1) an answer extraction module that identifies candidate answers in the reasoning chain, (2) a confidence estimator that scores each candidate, and (3) a stopping criterion that analyzes the confidence trajectory. The confidence estimator uses the model’s own token probabilities—specifically, it looks at how confidently the model generates the answer tokens.

Reasoning Chain Generation:
    |
    v
[Step 1] ---> Extract Answer_1 ---> Confidence: 0.3
    |
    v
[Step 2] ---> Extract Answer_2 ---> Confidence: 0.7
    |
    v
[Step 3] ---> Extract Answer_3 ---> Confidence: 0.85
    |                                      |
    v                                      v
[Step 4] ---> Extract Answer_4 ---> Confidence: 0.87
                                           |
                                           v
                                    Delta < threshold
                                           |
                                           v
                                    STOP & Return Answer_4

Think of CoDE-Stop like a chef tasting soup while cooking. A novice chef keeps adding ingredients and tasting repeatedly, never sure when it’s done. An experienced chef tastes, notices the flavor has reached the right balance, tastes once more to confirm it’s stable, then stops. CoDE-Stop is that experienced palate—it doesn’t just check if the answer is good (high confidence), it checks if the answer has stopped improving (confidence plateau). The “tasting” is confidence measurement, the “flavor balance” is answer quality, and the “stability check” is the dynamics analysis. Just as the chef saves time and avoids over-seasoning, CoDE-Stop saves tokens and avoids overthinking.

Key Concepts

• Confidence Dynamics: Instead of looking at a single confidence score, this tracks how confidence changes across reasoning steps. Imagine watching a stock price chart versus just seeing today’s price. The chart tells you if the price is rising, falling, or plateauing—that trajectory contains information the single number doesn’t. In reasoning, a confidence that jumps from 0.3 to 0.8 to 0.85 to 0.86 signals “we found it and we’re sure,” while a confidence bouncing between 0.4 and 0.6 signals “still searching.” The dynamics reveal the model’s internal certainty about whether it has solved the problem.

• Intermediate Answer Extraction: During chain-of-thought reasoning, models don’t just think—they periodically state candidate answers. For a math problem, the model might write “so the answer could be 42” at step 5, then “actually, recalculating gives 37” at step 8. CoDE-Stop parses these intermediate statements to identify answer candidates. This is different from just looking at the final output—it’s like watching a student’s scratch work to see when they first wrote down the correct answer, even if they kept working afterward.

Framework Shift

Before (fixed-length reasoning):        After (CoDE-Stop):

Input --> [Reason] --> [Reason] -->    Input --> [Reason] --> Extract Answer
          [Reason] --> [Reason] -->              [Reason] --> Extract Answer
          [Reason] --> [Reason] -->              [Reason] --> Extract Answer
          [Reason] --> Output                       |            |
                                                    v            v
          (Fixed N steps)                      Confidence   Confidence
                                               Trajectory   Analysis
                                                    |            |
                                                    +-----+------+
                                                          |
                                                          v
                                                    Stop when stable
                                                          |
                                                          v
                                                       Output

From blind iteration to informed termination, the core shift is using the model’s own confidence trajectory as a real-time stopping signal rather than relying on external length limits.

Expert Assessment

Problem choice: This is a real gap. As reasoning models scale, inference cost becomes prohibitive. The observation that correct reasoning reaches high confidence early is empirically grounded and practically important. It sits at the intersection of efficiency and capability—a sweet spot in current LLM research.

Method maturity: Elegant simplicity. The method requires no training, no additional models, just monitoring existing signals. However, the reliance on answer extraction assumes models consistently produce parseable intermediate answers, which may not hold across all reasoning formats. The confidence estimation using token probabilities is straightforward but could be brittle with different decoding strategies.

Experimental integrity: Baselines are reasonable (fixed-length, reward model scoring). The evaluation spans multiple models and benchmarks, which is good. However, the paper doesn’t deeply explore failure modes—when does confidence plateau on wrong answers? The 25-50% reduction is impressive but the variance across problem types isn’t thoroughly analyzed. I’d want to see more on the calibration of confidence scores.

Writing quality: The motivation is clear and the method is well-explained. The confidence dynamics analysis (correct vs incorrect trajectories) is the paper’s strongest contribution but could be expanded. The related work section feels thin—more comparison with adaptive computation methods would strengthen positioning. The experimental section is solid but lacks error analysis.

Verdict: weak accept — Practical method with clear benefits and minimal overhead, but limited theoretical depth and incomplete failure mode analysis. The core insight about confidence dynamics is valuable and the results are convincing enough for practitioners to try it.

Takeaways

Steal the confidence trajectory monitoring pattern. Even outside reasoning models, tracking how a model’s certainty evolves over generation steps can signal when to stop sampling, when to switch strategies, or when to request human input. The key transferable idea: don’t just threshold on confidence magnitude, threshold on confidence velocity (rate of change). This applies to iterative refinement systems, multi-agent debates, or any process where you’re unsure how many iterations to run. Implement it by logging intermediate outputs and their probabilities, then plot the trajectory—you’ll often see the “found it” plateau pattern emerge naturally.