Absolute Zero: Reinforced Self-play Reasoning with Zero Data

The problem with human-written questions

Reinforcement learning with verifiable rewards (RLVR) trains a language model by rewarding answers that a program can check automatically — a numerical result that matches, or code that runs and returns what it should. Nobody has to grade the model's reasoning step by step, which is why the approach has been widely used for reasoning training. But the questions themselves still come from people. Earlier zero-setting RLVR work, the authors note, avoids labelling the reasoning process but still depends on manually curated collections of questions and answers ^[1].

The paper argues that this dependence is a ceiling. High-quality human examples are scarce, which limits how far the approach can scale ^[1]. There is a second, subtler worry: tasks written by humans may offer limited learning potential to a system that goes beyond human intelligence ^[1]. If the trainee is meant to outgrow its teachers, the supply of problems should not be capped by how many good problems people can write.

Proposing your own curriculum

Absolute Zero is the paradigm the authors propose: a single model proposes the tasks that maximise its own learning progress, and then improves by solving them, with no external data at all ^[1].

The concrete system built on the idea is the Absolute Zero Reasoner (AZR). Its central trick is to use one code executor for two jobs at once: validating the code reasoning tasks the model proposes, and verifying the answers the model gives ^[1]. A code executor is simply a program that runs code and reports what happens — whether it returns a value, raises an error, or never finishes. That makes it a single unified source of verifiable reward ^[1]. A proposed task can be checked for being well formed and solvable before it enters the curriculum, and an answer can be checked by running it.

A rough illustration: the model might propose a small programming problem about computing a value from a list, the executor tries it to confirm it is a real, solvable problem, and the same executor later judges whether the model's solution behaves correctly. Because the same model plays both roles — proposer of tasks and solver of them — the curriculum is not fixed in advance but self-evolves ^[1]. As the solver improves, problems that were once hard stop being informative, and the proposer is pushed towards problems at the edge of what the model can currently do. That is what "maximise its own learning progress" means in practice: a task is worth training on to the extent that the model still has something to learn from it.

Results

Trained entirely without external data, AZR reaches overall state of the art on coding and mathematical reasoning ^[1]. It beats zero-setting models that rely on tens of thousands of in-domain human-curated examples ^[1], and outperforms the previous best model by 1.8 absolute percentage points on average ^[1].

The main experiments train on Qwen2.5-7B and Qwen2.5-7B-Coder, giving Absolute Zero Reasoner-base-7B and Absolute Zero Reasoner-Coder-7B ^[1]. Scale helps: gains grow with model size across the 3B, 7B and 14B coder models, and the method applies across model classes ^[1].

The authors also check that self-training has not collapsed the model's output variety. In pass@k evaluation — generate k answers to the same problem and ask whether at least one is correct — they went up to k=512, and found that AZR keeps answer diversity, outperforming the base model in 4 of 5 cases ^[1]. This matters because a self-taught model that drifts towards producing one repetitive answer would look fine on some measures while becoming much less useful.

Why it matters

The interesting move is where the training signal comes from. In earlier pipelines a person decides what the model practises; here the model proposes, a program judges, and the solving is the training ^[1]. Because the judge is a code executor rather than a human, the loop does not need a fresh batch of hand-written problems to keep going; the data is generated inside the system ^[1].

The authors frame the work as a way past the scarcity of human examples ^[1]. Whether a model can keep supplying itself with useful problems once it is better than its human-written starting points is exactly the question such a design is built to probe.

The paper

"Absolute Zero: Reinforced Self-play Reasoning with Zero Data" is by Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng and Gao Huang. It was submitted to arXiv on 6 May 2025 as 2505.03335 ^[1].