S1: Simple Test-Time Scaling

Overview

S1: Simple Test-Time Scaling demonstrates that test-time compute scaling — allowing a model to “think longer” during inference — can be achieved with remarkably simple techniques. The authors show that by fine-tuning a Qwen2.5-32B model on only 1,000 carefully curated examples and applying a technique called “budget forcing,” they can match or exceed OpenAI o1-preview on competition math benchmarks.

This challenges the assumption that test-time scaling requires complex reinforcement learning pipelines like those used in DeepSeek-R1.

Key Contributions

1. Curated Training Data

The authors distilled 59,000 reasoning traces from Google’s Gemini Flash Thinking model, then carefully filtered them down to just 1,000 examples based on three criteria:

• Quality: Only traces with correct final answers
• Difficulty: Prioritizing harder problems that require more reasoning
• Diversity: Covering a broad range of domains (math, science, coding, puzzles)

This small but high-quality dataset proved sufficient to teach the model extended reasoning behavior.

2. Budget Forcing

Budget forcing is the core technique that controls how much computation the model uses at test time. It works through two mechanisms:

• Forced termination: Appending an end-of-thinking token to stop reasoning when a maximum token budget is reached
• Forced continuation: Suppressing end-of-thinking tokens and injecting “Wait” to force the model to keep reasoning when a minimum budget hasn’t been met

This simple approach creates a controllable knob for test-time compute without any architectural changes.

3. Sequential vs Parallel Scaling

The paper compares two approaches to test-time scaling:

• Sequential scaling: Let the model think longer on a single attempt (budget forcing)
• Parallel scaling: Generate multiple shorter attempts and pick the best (majority voting)

Key finding: Sequential scaling is more compute-efficient for difficult problems, while parallel scaling works better for easier ones. The crossover point depends on problem difficulty.

Results

• s1-32B matches OpenAI o1-preview on AIME24 (American Invitational Mathematics Examination) and MATH500 benchmarks
• Budget forcing provides up to 27% improvement over greedy decoding on competition math
• The model achieves this with only 1,000 training examples and 26 minutes of fine-tuning on 16 H100 GPUs

Architecture

The approach is built on three components:

1. Base model: Qwen2.5-32B-Instruct (already capable of reasoning)
2. Training data: 1,000 curated reasoning traces with explicit thinking tokens
3. Inference control: Budget forcing to modulate test-time compute

No changes to the model architecture are needed — it’s purely a training data and inference strategy innovation.

Significance

S1 demonstrates that:

1. Data quality trumps quantity — 1,000 examples can match systems trained on orders of magnitude more data
2. Simple techniques work — No complex RL pipelines needed for test-time scaling
3. Test-time compute is controllable — Budget forcing provides a simple, effective knob
4. Open research is viable — The entire approach is reproducible with publicly available models and modest compute