OLMo 2 32B

What it is

OLMo 2 32B is a large language model with 32 billion parameters and the largest member of the OLMo 2 family, released by Ai2 on 13 March 2025.^[1] A parameter is one of the numerical weights a network learns during training, so the parameter count is a rough measure of size; at 32 billion, the model belongs to the mid-sized class that can be trained and adapted without frontier-scale resources. What sets it apart is the packaging: Ai2 released all of the data, code, weights and training details along with the model.^[1]

How it was trained

The model was trained on up to 6 trillion tokens drawn from a pretraining mix of 3.9 trillion tokens — about 1.5 epochs, meaning one and a half passes over the same body of text.^[1] Tokens are the chunks of text, roughly word fragments, that a model reads and predicts. A further mid-training stage followed on the Dolmino mix, 843 billion tokens in all.^[1] Splitting training into a broad pretraining phase and a smaller, curated mid-training phase is now a common pattern: the first builds general competence, the second steers the model before it is tuned for instructions and chat.

The run took place on Augusta, a 160-node Google Cloud cluster with 8 H100 GPUs per node, at over 1,800 tokens per second per GPU.^[1] That corresponds to roughly 38% MFU — model flops utilization, the share of an accelerator's peak arithmetic throughput the training actually uses. The 32B model was then post-trained with Tulu 3.1.^[2]

Results

Ai2 described OLMo 2 32B as the first fully open model to outperform GPT-3.5-Turbo and GPT-4o mini on a suite of multi-skill academic benchmarks.^[1] The qualifier matters: the claim concerns fully open models, meaning systems released with their data, code and weights. On the same benchmarks, Ai2 said the model matches or beats Qwen 2.5 32B and Mistral 24B, and approaches Qwen 2.5 72B and Llama 3.1/3.3 70B.^[1] The last two are roughly twice its size, so coming close to them puts a 32B open model near a level usually associated with a larger class. Ai2 also reported that it reached performance similar to Qwen 2.5 32B at roughly one-third of the training cost.^[1]

What it offers

All three OLMo 2 sizes could be fine-tuned on a single H100 GPU node and were available on the Ai2 playground.^[2] Fine-tuning on one node puts adaptation within reach of a university lab or a small team, not only organisations with large clusters. Releasing data, code and training details means the recipe can be inspected and reused rather than taken on trust.

OLMo 2 32B therefore sits at a particular point in the short history of open language models: a mid-sized system whose full training pipeline is public, and which Ai2 reports as competitive with widely used models of the same size and close to some twice as large.^[1]