Toolformer: Language Models Can Teach Themselves to Use Tools

The problem it addresses

Large language models are trained on one basic objective: given the text so far, predict the next token. That objective is powerful, but it is not the same as computing or looking something up. The Toolformer paper points to a paradox here: language models struggle with basic functionality such as arithmetic or factual lookup, tasks where much simpler and smaller models excel.^[1] A calculator returns an exact product; a retrieval system returns a stored answer. A language model, by contrast, produces whatever token seems most likely next, which may be fluent and still wrong.

The approach

Toolformer is a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction.^[1] An API, or application programming interface, is a defined way for one program to ask another program to do something and hand back an answer. The point of the design is that all four decisions are learned rather than hard-coded: nobody writes a rule saying that a string of digits must be multiplied.

A rough picture of what this looks like in practice: while writing a passage, the model reaches a spot where it needs an arithmetic result, emits a call to the calculator, receives the number back, and continues the text carrying that number along as if it had known it all along. The same shape of behaviour applies to the other tools.

How it was trained

What makes the method notable is that the training procedure is self-supervised and requires nothing more than a handful of demonstrations for each API.^[1] In self-supervised learning the supervision comes from the data itself rather than from human-labelled examples, so a small number of demonstrations is enough to show the model what a tool call looks like; after that, the model generates and evaluates candidate calls on its own. The model used as the starting point was the pretrained GPT-J model with 6.7B parameters, and it was trained on a subset of CCNet.^[1]

Tools and results

A range of tools was incorporated: a calculator, a Q&A system, two different search engines, a translation system, and a calendar. With them, Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks without sacrificing its core language modeling abilities.^[1] Zero-shot here means the model is asked to perform a task it was not specifically trained on, with no task-specific examples supplied at test time. The second half of that sentence matters as much as the first: learning to call tools did not come at the cost of ordinary text prediction, so the model did not become worse at the thing it was originally built for.

Why it matters

The contribution is less a single benchmark number than a recipe. Earlier systems that used external tools usually had the tool use designed by people: a program decides when to query a database and feeds the result to the model. Toolformer turns that decision into part of what the model learns, so that the choice of tool, the timing of the call, the arguments and the integration of the answer all belong to the model's own next-token prediction. The paper frames this as language models teaching themselves to use tools.^[1] That framing raises a general question about where capability should come from — bigger models alone, or models that can hand off to specialised, smaller, more reliable pieces of software when a task calls for it.