Base Models Can Reason By Taking a Cue From Training Data

Paper: 2610.06851 Authors: Sophie L. Wang, Amil Dravid, Rulin Shao, Kevin Farhat, Sewon Min, Alexei A. Efros Categories: cs.AI, cs.CL, cs.LG

The Gap

The prevailing narrative in frontier AI development is that reasoning is synthesized during post-training. Under this view, base language models merely predict web text distributions; it is reinforcement learning (RL) with verifiable rewards or self-play (e.g., DeepSeek-R1, OpenAI o1 style training) that teaches the network how to search, backtrack, and construct chain-of-thought derivations.

However, this assumption overlooks the structural composition of the pre-training corpus itself. Rigorous mathematical proofs, code reviews, and competitive programming walkthroughs on the web do not appear at random; they are preceded by characteristic conversational openers, formatting cues, and transitional markers.

Could it be that the base model has already learned these reasoning pathways during pre-training, and is merely waiting for the right trigger token to steer its generation trajectory into those specialized document subspaces?

   PROBLEM: DOES RL TEACH REASONING, OR MERELY RETRIEVE IT?

   Dominant View:
   Base Model (Unstructured Prediction) 
              |
              v  Expensive RL / Verifiable Rewards (RLVR)
   Synthesizes Search & Reasoning Abilities from Scratch
              |
              v  MATH-500: 42% -> 80%

   --------------------------------------------------------------

   THIS PAPER'S HYPOTHESIS & EXPERIMENTAL INTERVENTION:

   Base Model + Starting Token Cue (e.g., ".\n\nOkay" or "Alright,")
              |
              v  Forced Prefix
   MATH-500 Accuracy Jumps Immediately:
   - Olmo-3-7B: 42% -> 78% (Recovers 90% of RL gain without training!)
   - Qwen3-14B: 72% -> 87%
              |
              v
   Causal Pretraining Interventions:
   - Inject arbitrary token "chicken" as prefix to math derivations
   - "chicken" becomes an equally potent reasoning trigger!
   - "Think duck duck goose" matches "Think step by step"
              |
              v
   CONCLUSION: RL primarily boosts probability of pre-existing prefix cues

The Increment

One sentence: By demonstrating that fixing minimal opening token cues allows un-fine-tuned base models to match the mathematical and coding accuracy of RL-trained checkpoints, this paper proves that complex reasoning policies reside latent in pre-trained weights and are routed through document-distribution associations formed during pre-training.

Core Mechanism

The authors conduct a series of controlled causal interventions on base model generations across standard mathematical reasoning benchmarks (MATH-500, GSM8K) and competitive coding:

1. Prefix Cue Forcing: Rather than allowing the base model to sample its opening tokens, the authors force-feed candidate starting tokens at the beginning of the completion.
   • Appending ".\n\nOkay" to the prompt before generation instantly elevates Olmo-3-7B’s MATH-500 pass@1 from 42% to 78%.
   • Appending "Alright," elevates Qwen3-14B from 72% to 87%, nearly matching its fully RL-tuned sibling.
2. Mechanism of RL: When analyzing the checkpoint before and after reinforcement learning, the authors observe that RL drastically sharpens the model’s likelihood of emitting these exact reasoning cues as its initial tokens. Once those initial tokens are emitted, the autoregressive rollout follows the reasoning manifold that pre-training had already established.
3. Causal Pretraining Manipulation:
   • To rule out innate linguistic semantics of English words like “Okay,” the researchers pre-trained synthetic models from scratch.
   • By prepending arbitrary nonsense words—such as "chicken" or "quack"—to mathematical and algorithmic walkthroughs in the pre-training data, those arbitrary tokens become fully functional reasoning triggers.
   • Similarly, instructing the model with "Think duck duck goose" elicited identical chain-of-thought derivations as the canonical "Think step by step" if that phrase co-occurred with technical problem solutions.
4. Internal Representation Routing: Probing intermediate hidden states reveals that forcing a reasoning cue immediately shifts the network’s internal representations toward the subspace occupied by technical lecture notes, textbooks, and forum solutions, away from casual social media chatter.

   ACTIVATION ROUTING THROUGH PREFIX TOKENS

   Query: "Find all real roots of P(x) = x^3 - 3x + 1..."
                      |
                      v
   +-------------------------------------------------------------+
   | Unconditioned Base Model: Samples generic conversational tokens |
   | -> Routes to casual web chat manifold -> Fails / Hallucinates |
   +-------------------------------------------------------------+
                      |
                      v  Inject Prefix Cue: ".\n\nOkay"
   +-------------------------------------------------------------+
   | Conditioned Base Model: Hidden states snap into Math/Code   |
   | document manifold -> Emits rigorous derivation -> Solves!   |
   +-------------------------------------------------------------+

The load-bearing structural metaphor is a repertory theater actor backstage.

• The actor has spent years memorizing the complete works of Shakespeare, quantum physics lectures, and barroom stand-up comedy (pre-training).
• When pushed onto a pitch-black stage without props or direction, the actor starts bantering like a tavern patron because tavern talk is the most common default mode of human speech.
• You do not need to send the actor to graduate school for four years (reinforcement learning) to teach them quantum mechanics. You simply hand them a tweed jacket and chalk (the opening token cue ".\n\nOkay"). The moment they put on the jacket, the entire professor persona activates instantly, delivering a flawless physics derivation from memory. RL does not teach the physics; it simply teaches the actor to grab the tweed jacket by default when hearing the word “problem.”

Key Concepts

• Reasoning Cue: A small set of opening tokens (such as punctuation followed by conversational transitional phrases) that conditions the Transformer’s attention onto pre-training document distributions containing step-by-step technical problem solving.
• Causal Data Association: The associative link formed in next-token prediction between discourse markers that introduce problem solutions in training text and the structured logic that follows.
• Manifold Routing vs. Skill Acquisition: The distinction between a model learning how to logically deduce new theorems versus a model navigating to a pre-existing latent capability space.

Framework Shift

Before (The Post-Training Emergence Paradigm):
  Pre-training: Ingests raw knowledge without reasoning ability.
  RL Post-Training: Teaches the model search policies, self-correction, and reasoning logic.

After (The Cue-Driven Document Routing Paradigm):
  Pre-training: Ingests both knowledge AND sophisticated reasoning patterns from high-quality text.
  RL Post-Training: Primarily aligns the opening token probabilities to point into the reasoning manifold.
  Proof: Forcing the cue on the base model recovers ~90% of RL's benchmark gains for free!

From “RL creates reasoning out of whole cloth,” the core shift is that base models already possess profound reasoning circuits formed during pre-training, which can be summoned by simply steering the model’s opening discourse markers.

Expert Assessment

Problem choice: Groundbreaking and provocative. The AI community has spent billions of compute hours fine-tuning and running RL on models under the assumption that base models cannot reason on their own. Exposing the latent reasoning capability of unaligned checkpoints reshapes our understanding of LLM intelligence.

Method maturity: The causal data intervention experiments are airtight. By substituting arbitrary non-words (“chicken”) into pretraining data and observing the emergence of the identical cue effect, the authors decisively eliminate confounding semantic arguments.

Experimental integrity: Tested across distinct model families (Olmo, Qwen) and rigorous benchmarks (MATH-500, coding suites). Safety evaluations further confirm that the same cue-based routing governs compliance versus refusal behaviors.

Writing quality: Exemplary clarity, lucid storytelling, and sharp empirical precision.

Verdict: strong accept — One of the most clarifying mechanistic interpretability and pre-training papers of the year, dismantling key myths surrounding the necessity of heavy post-training for basic reasoning tasks.

Takeaways

• When evaluating or deploying base models, do not let them sample opening tokens unconstrained; experiment with prefix-cue forcing (".\n\nOkay", "Alright,", "Let's think carefully:") to unlock immediate reasoning jumps without fine-tuning.
• During pre-training data curation, ensure that high-quality technical and mathematical reasoning content is paired with clean, consistent introductory markers.
• Re-evaluate post-training RL pipelines: measure whether your RL algorithm is truly discovering novel problem-solving topologies or merely optimizing the likelihood of starting token triggers.