s1: Simple test-time scaling

The problem

OpenAI's o1 demonstrated that a language model can reason much better if it is allowed to think for longer before answering, but the company did not publish how it achieved this ^[1]. The authors of this paper — Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candes and Tatsunori Hashimoto — set out to find the simplest approach that delivers both things at once: test-time scaling, the idea that spending more computation while generating an answer makes that answer better, and strong reasoning ability ^[1]. The paper was submitted to arXiv on 31 January 2025 as 2501.19393 ^[1].

How it works

The recipe has two ingredients: a small, carefully built dataset, and a way to control how long the model thinks.

The dataset is called s1K: 1,000 questions, each paired with a reasoning trace, that is, the step-by-step working that leads to the answer ^[1]. The questions were not gathered at random. They were selected on three criteria — difficulty, diversity and quality — and the authors checked with ablations that each criterion actually contributes to the final result ^[1]. An ablation here means removing one element and re-running the experiment to see whether performance drops, so the criteria are justified by measurement rather than intuition.

The second ingredient is budget forcing, a method for controlling test-time compute without any extra training ^[1]. It works at the moment of generation in two directions. If the model tries to stop thinking and give its answer too early, the researchers forcibly append the word "Wait", which pushes the model to keep going and double-check its answer; if instead the thinking has gone on long enough, they forcibly end it and move to the answer ^[1]. In effect, the length of the model's internal reasoning becomes a dial that can be turned up or down at inference time.

The resulting model, s1-32B, is simply Qwen2.5-32B-Instruct fine-tuned with supervised learning on s1K and then run with budget forcing ^[1]. Supervised fine-tuning means training the model on example questions paired with their worked solutions, so that it learns to produce similar traces itself.

Results

On competition mathematics, s1-32B beats o1-preview by up to 27% on the MATH and AIME24 benchmarks ^[1]. Those two benchmarks consist of olympiad-style problems, where the model has to produce a final answer through multi-step reasoning rather than recall a fact.

Budget forcing also scales at inference time in the way the authors hoped. Applied to s1-32B, it raises the score on AIME24 from 50% to 57%, which is higher than the model reaches without any test-time intervention ^[1]. In other words, the same weights produce a better answer when the method is allowed to extend the thinking.

The efficiency numbers are striking. The supervised fine-tuning took just 26 minutes on 16 NVIDIA H100 GPUs using PyTorch FSDP, a framework for distributing a training job across many GPUs ^[1]. The paper contrasts this with 394 H100 GPU hours for fine-tuning on 59K samples, where s1-32B needed only 7 ^[1]. The gap comes from two directions: a far smaller dataset, and a much shorter run.

Why it matters

The model, the data and the code are all open-sourced ^[1]. That matters because the capability o1 revealed was, at the time, not accompanied by a published method; s1 offers a complete, reproducible recipe that others can inspect and reuse ^[1].

The broader lesson concerns where the gains come from. Instead of a large corpus of training examples or a complicated reinforcement-learning pipeline, the paper pairs a thousand curated questions with a text-level trick for controlling how long the model deliberates. Both halves are cheap, and both are described in enough detail to be copied. The work therefore serves as a baseline: a minimal configuration that already reaches competition-mathematics performance comparable to a much larger closed system, against which more elaborate approaches can be measured.