SmolLM

What SmolLM is

SmolLM is a family of small language models released by Hugging Face on 16 July 2024 in three sizes: 135 million, 360 million and 1.7 billion parameters.^[1] Parameter count is a rough measure of a model's capacity — the number of weights it learns during training — so all three sit far below the large models that dominate headlines, and are built to run on a device rather than in a data centre.^[1] Running locally buys offline operation and keeps the user's text on the machine; the cost is the breadth that comes with far more parameters.

A corpus built for the job

The models were trained on far more text than their size would suggest: the 135M and 360M models on 600 billion tokens, and the 1.7B model on 1 trillion tokens.^[1] A token is a chunk of text, roughly a short word or part of a word, so these counts mean hundreds of billions of fragments — many times the number of parameters in each model.

That text was assembled for the project as the SmolLM-Corpus, in three parts: Cosmopedia v2, 28 billion tokens of synthetic textbooks and stories generated by Mixtral; Python-Edu, 4 billion tokens taken from The Stack; and a deduplicated version of FineWeb-Edu, 220 billion tokens.^[1] The mix is deliberate in kind: filtered educational web text for general knowledge, code for structured material, and generated textbook-style prose alongside them. The model card names the training data as Cosmo-Corpus and lists English as the only language.^[3]

Reported results

Hugging Face compared each model with others in its own size class. SmolLM-135M was reported to beat MobileLLM-125M; SmolLM-360M to beat every model under 500 million parameters, including Qwen2-500M; and SmolLM-1.7B to beat models under 2 billion parameters, including Phi-1.5 and Qwen2-1.5B.^[1] The model card describes these comparisons as being made within size categories, on benchmarks that test common-sense reasoning and world knowledge.^[3]

Read closely, the claims are relative. Each result says the model did better than similarly sized peers; none of them says it competes with systems an order of magnitude larger. That framing is what makes the numbers useful — for someone choosing a model to run on a laptop or a phone, the relevant question is which small model is best, not whether a small model beats a large one.

Open weights and everyday use

The weights were published openly under the Apache 2.0 license on the Hugging Face Hub.^[2] Checkpoints for all three sizes are on the Hub and load through the transformers library,^[3] so a developer can download one and run it with the same code used for other models in that ecosystem. The license is a permissive one, which matters for anyone who wants to ship a model inside a product rather than only study it. Publishing three sizes rather than one lets a developer trade memory and speed against capability without changing tooling: the 135M model for the tightest devices, the 1.7B model where more room is available.

Where it sits

SmolLM was succeeded by SmolLM2 in November 2024,^[2] a quick follow-up that indicates how fast this corner of the field moved. The interest of the original release lies in the combination: a small model, a documented and partly synthetic training corpus, and claims of competitive quality inside a narrow parameter band — with the weights and the recipe released together.