Gemma 2

What Gemma 2 is

Gemma 2 is the second generation of Google's Gemma family of open models, announced on 27 June 2024 in two sizes: 9 billion and 27 billion parameters^[1]. Both are text-to-text, decoder-only large language models — the architecture behind most chat-style language models, in which the network reads a piece of text and generates the continuation one token at a time, each token conditioned on everything before it. The input is text; the output is English-language text^[3].

The parameters in the name are the weights the model learns during training. More of them generally means more capability and more computing power needed to run the model, which is why the size of a release is usually a trade-off rather than a simple ranking.

Where it came from

Gemma 2 extends the Gemma line of openly released models. Google said the new generation was officially released to researchers and developers globally^[1], and it shipped under the Gemma license, which Google said allowed commercial use and monetisation^[1]. That licensing point matters for the open label: weights that can be downloaded are only useful to a company if it is also allowed to build products on top of them, so the terms are part of the release, not an afterthought.

How it is built

Two design details are documented for Gemma 2. It uses grouped-query attention, an arrangement in which several query heads share one set of key and value projections, cutting the memory and bandwidth that attention needs at inference time while keeping most of the quality of full multi-head attention. It also keeps a context window of 8,192 tokens^[2] — the amount of text the model can consider at once, roughly a few thousand words.

The training scale is given in Google's model card: the 27B model was trained on 13 trillion tokens and the 9B model on 8 trillion tokens^[3]. A token is a fragment of text, often a word or part of a word; trillions of them represent a very large amount of reading, and the larger model in the pair saw the larger share.

What Google claimed for it

Google's pitch centred on the 27B model. The company said it offered competitive alternatives to models more than twice its size^[1] — that is, that a model this small could be compared with much larger ones. It also said the 27B model ran inference efficiently at full precision on a single Google Cloud TPU host, an NVIDIA A100 80GB, or an NVIDIA H100 GPU^[1]. Full precision here means the weights were not compressed into a lower-precision format in order to fit; being servable on one accelerator host is a practical statement about what a team needs in order to deploy it. Google also said a 2.6-billion-parameter Gemma 2 model was planned^[1], aimed at settings where even 9 billion parameters are too much.

Where it sits in the line

Gemma 2 became a base for later releases: the vision-language PaliGemma 2 models, in December 2024, were built on Gemma 2^[2], pairing the language model with an image encoder so that pictures as well as text can be handled. Within the Gemma family, the second generation widened the line from an initial open release into a set of sizes and derivatives; for people who wanted a downloadable model they could run themselves under a license allowing commercial use, it was one of the options to weigh against other open families.