Stable LM

What Stable LM is

Stable LM is a suite of language models from Stability AI. The first models in the suite were launched on 19 April 2023 as alpha versions with 3 billion and 7 billion parameters, and Stability said that 15-billion and 65-billion-parameter models would follow ^[1]. Parameter count is a rough measure of size — it is the number of numerical weights the network adjusts while training — so a 3-billion-parameter model sits on the small side and a 65-billion-parameter one on the large side.

How it was released

The base models were published on Stability's GitHub under the CC BY-SA-4.0 license, which allows commercial and research use ^[1]. What was published was the model weights themselves rather than access to a hosted service, so others could take the models and build on them; the license is a share-alike one, meaning it also sets the terms under which modified versions may be passed on.

Two kinds of model

The release came in two tracks with different terms. The base models are general text predictors: trained on large amounts of text, they learn to continue a passage, and they can be used for commercial or research purposes subject to the CC BY-SA-4.0 license ^[1]. Alongside them Stability published instruction-tuned research models, fine-tuned on Alpaca, GPT4All, Dolly, ShareGPT and HH data and released under CC BY-NC-SA 4.0 for research only ^[1]. Instruction tuning is the step that turns a text predictor into something that answers a request: the model is trained on example instructions paired with good responses. Stability says its instruction-tuned research models were fine-tuned following Stanford Alpaca's procedure ^[2].

Training data

Stability said training used an experimental dataset built on The Pile, about three times larger, with 1.5 trillion tokens of content ^[1]. A token is the unit in which models read text — a word or a piece of one — so the figure describes the volume of text the models were trained on.

Later updates

The repository was updated after the initial alpha release. On 5 August 2023 Stability released patched StableLM-Alpha v2 models with 3B and 7B parameters ^[2]. On 29 September 2023 it released StableLM-3B-4E1T under CC BY-SA-4.0 ^[2]. Both later updates sit in the same GitHub repository as the original release ^[2].

Where it fits

Stable LM is best understood as an open-weights release with a split license: the base models are available for commercial use under a share-alike license, while the instruction-tuned variants are research-only. That split lets an organisation publish base weights broadly while keeping tighter terms on the tuned versions built on top of them, and for anyone using the models it means the license attached to a particular model, not the family name, decides what is permitted.

The trajectory within a few months — an alpha release, patched v2 models, then a newly named 3B model — shows the family being revised rather than replaced, with the small 3B and 7B sizes remaining the centre of the published work while the announced 15-billion and 65-billion-parameter models were described as still to come ^{[1] [2]}.