Meta releases Llama 3.2 with vision and on-device models

Background

Until this release, the Llama models that Meta had published accepted text only. Meta's larger aim with Llama 3.2 was to bring the family to images and to smaller devices. Meta released it at its Meta Connect event, where it presented the model as an open, customisable option for edge AI, meaning AI that runs on the device itself instead of in a remote data centre.^[1]

What happened

On 25 September 2024 Meta released Llama 3.2 in two groups. The first group has two vision models, with 11 billion and 90 billion parameters. The second has two text models, with 1 billion and 3 billion parameters. The text models are meant for devices with limited hardware.^[1]

Key details

The first Llama models that see. The 11B and 90B models were the first Llama models that understand images. They do this by attaching a cross-attention adapter to Llama 3.1, which lets the image information feed into the existing language model.^[1] A cross-attention adapter is a set of extra layers that lets the language model look at features from a second input, here the picture.

Small models for devices. The 1B and 3B models were made by pruning and distilling Llama 3.1. Pruning removes parts of a large model, and distillation trains the smaller model to copy the behaviour of the larger one. Both models support a 128K context, meaning they can take in a long input in one pass. They run on hardware from Qualcomm and MediaTek, two companies that make phone chips.^[1]

Comparisons. Meta said that the vision models were competitive with Claude 3 Haiku and GPT-4o mini on image understanding tasks.^[1]

Tooling. The release included the first official Llama Stack distributions, which are packaged sets of tools for running Llama models in an application. It also included Llama Guard 3 Vision, a safety classifier that can review images.^[1]

Quantised releases. On 24 October 2024 Meta published quantised versions of the 1B and 3B models. Quantisation stores the model's numbers at lower precision, which makes the files smaller and faster to run on limited hardware.^[2]

Impact

Llama 3.2 showed that the Llama family could cover three kinds of use in one generation: text on small devices, vision for larger models, and a set of tools for building applications around them. Developers could download the models under Meta's licence and run the smaller ones on phones and laptops rather than in the cloud. The release also brought image understanding into the open Llama line for the first time.^[1]