Llama 2: Open Foundation and Fine-Tuned Chat Models

What the paper is

"Llama 2: Open Foundation and Fine-Tuned Chat Models" is a paper led by Hugo Touvron and a long list of co-authors from Meta, submitted to arXiv on 18 July 2023^[1]. It does two things at once: it develops and releases a family of large language models, and it documents in detail how the conversational versions of those models were trained^[1].

The problem it addresses

A language model is, at bottom, a system trained on large amounts of text to predict what comes next. That objective produces fluent text, but not necessarily a useful assistant: a model trained only to continue text may ramble, ignore the user's intent, or produce content its developers would rather it refuse. Turning such a base model into something that holds a conversation well requires further training on dialogue, and shaping its behaviour towards safety requires more work still. A recurring question in the field is how much of that gap can be closed by models whose construction is described openly rather than kept behind a service interface. The paper's contribution sits squarely on that question.

What was released

The paper develops and releases Llama 2, a collection of pretrained and fine-tuned large language models ranging in scale from 7 billion to 70 billion parameters^[1]. Parameters are the numerical weights a model learns during training; the count is a rough measure of model size and capacity, which is why a family spanning this range offers options at very different computational costs. The fine-tuned members of the family are called Llama 2-Chat, and they are optimised for dialogue use cases^[1]. The distinction matters: the pretrained models are general-purpose starting points, while the chat models are the ones shaped for back-and-forth interaction with a user.

How the chat models were built

A substantial part of the paper is devoted to the route from a pretrained model to a chat model. The authors give a detailed description of their approach to fine-tuning and safety improvements in Llama 2-Chat^[1]. Fine-tuning, in general terms, means continuing to train an already pretrained model on a narrower and more carefully curated set of examples, so that its behaviour shifts towards the desired kind of response. Safety work in this setting typically means teaching the model to decline or redirect harmful requests, and to answer in ways that human reviewers judge appropriate. The paper treats these as engineering steps to be specified, not as side effects, and sets out what was done.

Results

The abstract reports that the models outperform open-source chat models on most benchmarks the authors tested^[1]. Benchmarks here are standardised test sets with fixed scoring, which let different models be compared on the same tasks. Benchmarks alone do not capture how a model feels to use, so the authors also ran human evaluations, asking people to judge model outputs for helpfulness and safety; on the basis of those evaluations, they say the models may be a suitable substitute for closed-source models^[1]. That is a claim about the authors' own comparisons, and it is framed as a possibility rather than a settled result.

Why the details were published

The authors state that the aim of publishing these details is to let the community build on the work and contribute to the responsible development of large language models^[1]. Making the fine-tuning and safety recipe legible serves two audiences at once: researchers who want to reproduce, extend or critique the method, and anyone trying to judge what a model of this kind will and will not do. For a class of system whose behaviour is hard to inspect from the outside, a written account of how it was shaped is part of what makes the release usable and assessable, and it places the work in a lineage of openly described foundation models that others can measure themselves against.