TII releases the Falcon3 model family

What was released

On 14 December 2024, TII published the Falcon3 family of language models ^[1]. The release covers four sizes — 1B, 3B, 7B and 10B parameters — and each size appears in two forms: a base model, trained on text to predict what comes next, and an instruct model, further tuned to answer requests and follow instructions ^[1]. Shipping both forms matters because the two serve different users: the base model is a starting point for further training, while the instruct model can be used directly.

The 10B and depth up-scaling

The largest model in the family was not designed as a separate architecture. TII describes the 10B as a depth up-scale of the 7B base model rather than a model designed on its own ^[1].

Depth up-scaling is a way of building a bigger model out of a smaller one. A transformer's depth is the number of layers stacked one after another; each layer passes its output to the next, so a deeper model can carry a representation through more stages of computation and generally holds more parameters. Instead of specifying a new architecture and training it from random starting weights, depth up-scaling extends an existing model's stack of layers, producing a deeper — and therefore larger — model from a design that is already known to work. The approach saves the cost of designing and validating a new architecture, at the price of inheriting the structure and any weaknesses of the model it grew out of.

Training the up-scaled model still took substantial compute: 2 trillion tokens of training data and 1,024 H100 GPUs ^[1]. Tokens are the units of text — roughly word pieces — that a model is trained on, so 2 trillion tokens is a measure of how much text the model saw during training.

Reported results

TII said the models were state of the art at release among open models of their size ^[1]. This is a claim about a specific comparison group — openly released models in the same size range — and it reflects the position at the time of release, not a permanent ranking.

For the largest instruct model, the published model card reports a score of 78.17 on IFEval, as measured on the Open LLM Leaderboard ^[1]. IFEval tests whether a model obeys instructions whose compliance can be checked automatically — for example, a request to answer in a fixed format or under a word limit — and scores how often those checkable constraints are met. A leaderboard score is useful for comparison, but it measures a narrow slice of behaviour rather than overall quality.

Licence

One choice stands out for anyone planning to build on the family. The models kept the Falcon-LLM Licence rather than moving to Apache or MIT ^[1]. Apache and MIT are the standard permissive open-source licences, whose terms are well known and widely accepted by legal teams and package ecosystems. Staying on a project-specific licence means users have to read the terms themselves to find out what is permitted and what obligations apply, and some organisations treat non-standard licences as a reason to look elsewhere.

Where it fits

Falcon3 sits in the small-to-mid range of open models, from 1B up to 10B parameters. That range is where models are cheap enough to run on modest hardware and to fine-tune on limited budgets, and where a single release spanning several sizes lets a user pick the one that fits their compute. The family's interest lies less in any single number than in the combination: a full size ladder, both base and instruct forms, a documented up-scaling route to the largest model, and a licence that keeps it outside the Apache and MIT mainstream.