Hugging Face releases SmolLM2

Background

Language models are usually described by how many parameters they contain — the numbers learned during training, which effectively set how much the model can hold. A 135M model has roughly 135 million parameters; a 1.7B model roughly 1.7 billion. Well-known chat models are far larger than this, but a persistent line of work asks how much capability can be packed into models small enough to run on a laptop or a phone, where memory, power and response time are all tight.

What was released

Hugging Face released SmolLM2 in November 2024 in three sizes: 135M, 360M and 1.7B parameters^[3]. The weights are published under the Apache 2.0 license^[1], a permissive open-source license that allows commercial and non-commercial use, modification and redistribution as long as the licence text and notices are kept. In practice that means a team can download the model, run it, fine-tune it and ship it inside a product without negotiating a separate agreement first.

How the 1.7B model was trained

The 1.7B model was trained on about 11 trillion tokens^[1]. A token is the unit a language model reads and predicts, usually a word fragment; English text averages somewhere under one word per token, so trillions of tokens amount to far more text than any person could read. That is the central bet of the small-model line of work: capacity is the scarce resource, so the data budget is spent generously instead.

Alongside the base models, the release included an instruct model built with supervised fine-tuning followed by Direct Preference Optimization on UltraFeedback^[4]. Supervised fine-tuning trains the model on example instructions paired with good answers, which teaches it the shape of a request-and-response exchange. Direct Preference Optimization, usually shortened to DPO, then takes pairs of answers to the same prompt — one preferred, one not — and adjusts the model to make the preferred one more likely. It is a simpler alternative to training a separate reward model and optimising against it with reinforcement learning.

Results reported

Hugging Face reported that the model outperforms Qwen2.5-1.5B and Llama3.2-1B^[2], two models of comparable scale from other organisations, which gives readers a yardstick for what the size class can do. The model card reports that the 1.7B variant improved over SmolLM1-1.7B especially in instruction following, knowledge, reasoning and mathematics^[4]. The gains, in other words, were not spread evenly across every kind of task but concentrated in the abilities a user notices most when talking to a chat model.

The paper and the datasets

A technical paper followed on 4 February 2025, released together with three datasets: FineMath, Stack-Edu and SmolTalk^[2]. Publishing the data alongside the weights matters, because the quality of a small model depends heavily on exactly what it was trained on. Without the mixture, others can neither reproduce the result nor build a different model on the same foundation.

Why it matters

SmolLM2 sits at the point where open licensing, small size and a documented training recipe meet. Models of a billion or two parameters are small enough that many researchers and hobbyists can run them on ordinary hardware, and an Apache 2.0 licence removes the legal friction that otherwise slows that down. The instruct variant and the published datasets make the family usable for two different audiences at once: people who want a working assistant they can run and adapt, and people who want to study how such a model is assembled. The comparison against Qwen2.5-1.5B and Llama3.2-1B also gives the field a common reference point for how much a well-trained small model can close the gap on larger ones.