Meta releases Llama 2 for research and commercial use

Background

Llama 2 is the next generation of the models Meta had introduced earlier as LLaMA. Fine-tuning means further training a pretrained model on a smaller set of examples, such as dialogue, to change how it behaves. Its weights, the numerical values that a trained model uses, can be downloaded, so others can run the model on their own hardware and fine-tune it. Publishing weights is not the same as open-source software in the strict sense. The terms vary by release, so the license matters as much as the availability of the files.

What happened

On 18 July 2023 Meta released Llama 2, free for research and commercial use. The release included model weights and starting code for the pretrained models and for the fine-tuned conversational versions.^[1] The paper, “Llama 2: Open Foundation and Fine-Tuned Chat Models,” was posted to arXiv the same day. It has 68 authors, and its first author is Hugo Touvron.^[2]

Microsoft was named Meta's preferred partner for Llama 2. The models were offered in the Azure AI model catalog and optimised to run locally on Windows, and they were also available through AWS, Hugging Face and other providers.^[1]

Key details

The released sizes were 7B, 13B and 70B parameters.^[1] Each was pretrained on 2.0 trillion tokens with a 4k-token context length, meaning the amount of text the model can consider at once.^[3] Only the 70B model used grouped-query attention, a technique that reduces the memory needed during text generation.^[2]

The fine-tuned Llama 2-Chat models are optimised for dialogue. Their fine-tuning data included over one million new human-annotated examples.^[2]

According to the model card, Llama 2 was trained between January and July 2023, with pretraining data ending in September 2022. Training used a cumulative 3.3 million GPU hours on A100-80GB GPUs.^[3]

Meta said the fine-tuned models were red-teamed through internal and third-party adversarial testing, meaning testers actively searched for failures and harmful outputs. Meta also published a Responsible Use Guide and an Acceptable Use Policy.^[1]

The models were released under a custom commercial license, and downloading them requires accepting Meta's terms.^[3]

Impact

By allowing commercial use of downloadable weights, the release let companies run a large model on their own infrastructure, or call it through a cloud provider, rather than relying only on a closed interface.^[1] Because the family came in three sizes, a user could weigh cost against capability, using the 7B model for lighter workloads and the 70B model where more capability was needed.

The combination of a custom license and an acceptable-use policy reflects a middle ground: the weights are available, but their use is governed by contract terms rather than a fully unrestricted open license. How people judge that trade-off depends on how they weigh openness against risk.