Hugging Face releases the SmolLM small language models

Background

Hugging Face is best known as a hub where developers share machine-learning models, datasets and demos. Many widely discussed models of the time were very large, needing powerful servers to run. Small models aim at a different target: they are light enough to run on a phone or a laptop without sending data to a remote service. The SmolLM release was built around that goal ^[1].

What happened

On 16 July 2024 Hugging Face released SmolLM in three sizes: 135M, 360M and 1.7B parameters ^[1]. A parameter is an adjustable number inside a model that stores part of what the model has learned, so the figures measure how large each model is. The models target on-device use, which means they are designed to run on the user's own hardware ^[1].

The models were trained on SmolLM-Corpus, a dataset that combines three sources: synthetic Cosmopedia v2 textbooks and stories, Python-Edu code, and FineWeb-Edu web text ^[1]. The training corpus was published alongside the models, so anyone can see what the models learned from.

Key details

The 1.7B model was trained on 1 trillion tokens, and the two smaller models on 600 billion tokens each ^[1]. A token is a small unit of text, and a trillion of them is an enormous amount of reading for a model. Training on that much data is one way a small model can gain ability without growing.

Hugging Face said each size outperformed rival models in its class. For the 1.7B model, the comparison included Qwen2-1.5B and Phi-1.5 ^[1]. These are the developers' statements. A fair reading should check the evaluation details and the date of the comparison, since rivals and their scores change.

Impact

The release gave developers a set of small open models they could download and test without large computing budgets. Publishing the corpus with the models meant that the training choices could be examined, not just the final weights. This openness matters for a reader who wants to understand how a model was made, because the data is often the least visible part of a system.

The SmolLM family also set up the work that followed. The next generation, SmolLM2, arrived in November 2024 and built on this line of small models. For anyone who follows open AI, the release is a clear example of a company putting effort into the data and the size of models rather than only into raw scale.