OLMo 2

What it is

OLMo 2 is the second generation of the OLMo family of fully open language models from the Allen Institute for AI (Ai2), released on 26 November 2024 in two sizes, 7B and 13B parameters^[1]. "Fully open" is meant literally here: the release includes base and instruct weights, the pretraining and post-training data, code, training recipes, intermediate checkpoints and the OLMES evaluation framework^[1]. The weights and code are published under the Apache 2.0 licence, and the models have a 4,096-token context window^[3].

Most open-weight models ship only the finished weights, so others can run them but cannot see what went in. Releasing the data and the recipes as well makes it possible to trace how a model was built and to rerun the process.

How it was trained

Pretraining ran over up to 5 trillion tokens in two stages. The first was about 3.9 trillion tokens of a corpus called OLMo-Mix-1124; the second was a mid-training mix, Dolmino-Mix-1124, of 843 billion tokens, used to continue training on higher-quality material^[1].

Ai2 said it invested specifically in training stability, because long pretraining runs are prone to loss spikes — sudden jumps in the training loss, the number that measures how far the model's predictions are from the target — and such spikes correlate with lower final quality^[4]. The models used staged training with interventions during late pretraining to keep the run on track^[4].

Post-training

The instruct versions were post-trained with Ai2's Tülu 3 recipe, which combines supervised fine-tuning (SFT), preference tuning and reinforcement learning with verifiable rewards^[1]. In broad terms, supervised fine-tuning teaches the model from example answers, preference tuning pushes it toward the kind of response people rate more highly, and reinforcement learning with verifiable rewards is applied where an answer can be checked automatically — for instance, a maths result that is either right or wrong.

Reported results

Ai2 said that OLMo 2 7B outperforms Llama-3.1 8B, and that OLMo 2 13B outperforms Qwen 2.5 7B, despite lower training FLOPs; it also reported that OLMo 2 13B Instruct outperformed Qwen 2.5 14B Instruct^[1]. The institute described the models as the best fully open language models to date^[1]. These are Ai2's own comparisons against named baselines.

Documentation

The models come with a technical report titled "2 OLMo 2 Furious", published on arXiv as 2501.00656^[2]. Together with the public data, checkpoints and the OLMES evaluation framework, the report is part of the same idea: a model whose construction can be inspected and repeated, not just used^[1].

Where it sits

A 32B model followed on 13 March 2025, extending the family upward in scale, and Olmo 3 succeeded the family in November 2025^[1]. OLMo 2 therefore occupies the middle of a series: it is the generation that carried Ai2's fully open approach from a first release to a family of sizes with a documented training pipeline and a matching instruction-tuned variant.