Gemma

What Gemma is

Gemma is a family of lightweight open models from Google DeepMind, released on 21 February 2024 and built from the same research and technology used to create Gemini.^[1] Google said the models were "available worldwide, starting today."^[1]

"Open" here means the trained weights are published so that others can download and run the models themselves, rather than only reaching them through a hosted service. That matters for researchers without large budgets, for developers who need to run a model on their own hardware or keep data in-house, and for anyone who wants to inspect and adapt a model rather than just call it.

Sizes and variants

Gemma came in two sizes, Gemma 2B and Gemma 7B, each with a pre-trained and an instruction-tuned variant.^[1] The pre-trained version is trained on large amounts of text to predict what comes next, which gives it general language ability but no particular habit of following requests. The instruction-tuned version is trained further on examples of instructions paired with good answers, so it responds usefully to a prompt instead of merely continuing text. Having both in each size lets developers choose between a raw base model to fine-tune and a model that works out of the box.

The models have an 8,192-token context window and handle text only.^[2] The context window is the amount of text the model can keep in view at once: everything in the prompt and the answer has to fit inside it. A larger window allows longer documents, longer conversations and more few-shot examples in a single call. Text-only means the models do not take images or audio as input.

Claims about quality

Google said Gemma was best-in-class for its size among open models and that it surpassed significantly larger models on key benchmarks.^[1] Benchmarks are standardised test sets, usually scored automatically, that let people compare models on tasks such as question answering or reasoning. They are useful for quick comparison but narrow: a high score says a model does well on those particular tests, not that it is better at everything. The comparison to larger models is worth reading with that in mind, and the claim here is Google's own description of its models rather than an independent finding.

Licence and safety tooling

The weights were released under the Gemma Terms of Use, which permit responsible commercial usage and distribution for all organisations regardless of size.^[1][3] That phrasing matters: a small team and a large company work under the same permission, so a startup or a university lab can build a product on the models without negotiating a separate agreement.

Google released a Responsible Generative AI Toolkit with the models, including tools for building safety classifiers and debugging model behaviour.^[1] A safety classifier is a separate model that screens inputs or outputs and flags material that a developer has decided should be filtered; debugging tools help developers work out why a model produced a particular output. Shipping these alongside the weights reflects the fact that once a model can be downloaded, the guardrails have to travel with it.

Where to run it

They were available on Kaggle, Colab, Hugging Face and Vertex AI, with support for Keras 3.0, PyTorch, JAX and TensorFlow, and optimised for NVIDIA GPUs and Google Cloud TPUs.^[1] Support across several frameworks means developers can stay in the ecosystem they already use rather than rewriting their pipeline around one library. Google also provided ready-to-use Colab and Kaggle notebooks, alongside integration with Hugging Face, and easy deployment on Vertex AI and Google Kubernetes Engine (GKE).^[1]

Place in the line

Variants followed, including CodeGemma for code and RecurrentGemma in April 2024, which is based on the Griffin architecture rather than a pure Transformer.^[2] Transformer-based models rely on attention over the whole sequence; an alternative architecture is a way of testing whether the same quality can be reached with different, sometimes cheaper, computation. Gemma was succeeded by Gemma 2 in June 2024.^[2] Taken together, the release marked Google putting a Gemini-derived, commercially usable model family into the open-weights ecosystem, with a licence, safety tooling and framework support packaged around it rather than a set of weights alone.