Llama 3.2

What it is

Llama 3.2 is a family of Meta's openly available models announced on 25 September 2024 at Meta Connect. It has two groups. The 11B and 90B models are vision models, meaning they can take images as input as well as text. The 1B and 3B models are text-only and are built for edge and mobile devices, where computing power and memory are limited. Each size comes in a pretrained version and an instruction-tuned version.^[1]

Origins

The 11B and 90B models were the first Llama models to support vision. Their design is an adapter: a trained set of cross-attention layers feeds an image encoder into the Llama 3.1 language model. Cross-attention is a way for one part of a model to look at another part's output, here the image features. The language model's weights stay frozen, which means they are not changed during this training. As a result the text abilities are unchanged, and the vision models can be used as drop-in replacements for Llama 3.1.^[1]

The small models were made differently. They came from structured pruning of Llama 3.1 8B, which removes parts of the network, followed by knowledge distillation from the logits of Llama 3.1 8B and 70B. Distillation trains a smaller model to match the output probabilities of larger ones. Both small models support a 128K-token context and were pretrained on up to 9 trillion tokens, with a December 2023 knowledge cutoff. They support eight languages.^[1][2]

Main features

Model size. The 1B model has 1.23 billion parameters, and the 3B model has 3.21 billion. Both support a 128K-token context window.^[1][2]

Benchmark results. For the instruction-tuned 1B and 3B models, the scores were 49.3 and 63.4 on MMLU, 59.5 and 77.4 on IFEval, 44.4 and 77.7 on GSM8K, 30.6 and 48.0 on MATH, and 25.7 and 67.0 on BFCL V2.^[2] The gap between the two columns shows how much the extra two billion parameters changed the results.

Comparisons Meta made. Meta said the vision models were competitive with Claude 3 Haiku and GPT-4o mini on image recognition. It said the 3B model outperformed Gemma 2 2.6B and Phi 3.5-mini on instruction following, summarisation and tool use.^[1]

Device support. Qualcomm and MediaTek hardware supported the small models from day one, with optimisations for Arm processors.^[1] Arm processors are the chips used in many phones and laptops.

Llama Stack and safety. The release included the first official Llama Stack distributions, which are packaged setups for running Llama models in different places: a single node, the cloud, on-device through PyTorch ExecuTorch, and on-premises. It also included Llama Guard 3 11B Vision and Llama Guard 3 1B as safety models.^[1]

Availability. The models were available on llama.com and Hugging Face, and through more than 25 partners, including AMD, AWS, Databricks, Dell, Google Cloud, Groq, IBM, Intel, Microsoft Azure, NVIDIA, Oracle Cloud and Snowflake.^[1]

Licence. Llama 3.2 was released under the Llama 3.2 Community License. Quantised versions of the 1B and 3B models, with an 8K-token context, followed on 24 October 2024.^[2]

Its place in development

Llama 3.2 extended the family from text into images and into small devices. Its successor was Llama 3.3 70B, released on 6 December 2024.^[3]

Because the vision models were drop-in replacements for Llama 3.1, a developer who already used Llama 3.1 for text could move to the vision models without rebuilding the text part of the application.^[1]