Tool calling

What it is

A language model writes its answer one token at a time, and normally everything the answer needs — the arithmetic, the lookup, the conversion — has to come out of its own parameters. Tool calling adds a second path. The model is allowed to hand a sub-task to an external program, such as a calculator or a search engine, and then use whatever that program returns inside its answer ^[1]. The program does the work; the model decides that the work is needed, describes it in a form the program accepts, and folds the result back into the text it is writing.

What a call looks like

A call is not a separate channel. It is text the model emits, in a format agreed on in advance: the model writes the name of a tool together with the arguments it wants to pass, the surrounding software pauses generation, runs the program, and inserts the output back into the context. Generation then continues with the result in view. Ask what 24 percent of 3,150 is, and the model may, instead of guessing, emit a calculator call carrying the expression, receive 756, and write the sentence around that number. The same shape covers a query sent to a search engine or a date sent to a calendar.

The alternative arrangement puts that decision in the surrounding program: a fixed pipeline decides when to retrieve or when to compute. Tool calling moves the decision to the model itself, which is both what makes the idea attractive and what makes it hard to train.

Where it came from

Schick and co-authors set out the idea in a paper called Toolformer, published in February 2023 ^[1]. Their starting observation was that language models struggle with basic functionality such as arithmetic or factual lookup, where much simpler and smaller models excel ^[1]. The paper's contribution was therefore a training recipe rather than a hand-written rulebook. The model was taught to decide for itself which APIs to call, when to call them, what arguments to pass, and how to incorporate the results into future token prediction ^[1]. Training was self-supervised: beyond a handful of demonstrations for each API, nothing had to be labelled by hand ^[1]. Candidate API calls are selected while the training data is being constructed, rather than at run time by an outside rule ^[1].

The toolset in the paper was deliberately ordinary: a calculator, a question answering system, two different search engines, a translation system and a calendar ^[1]. None of these is impressive on its own. The point is that the model, not the pipeline, chooses when each one is worth consulting.

What it showed

The trained model achieved substantially improved zero-shot performance across a variety of downstream tasks, and was often competitive with much larger models ^[1]. Zero-shot here means the tasks were not trained on: the model simply arrives with its tools and its judgement about when to use them. The gains came without sacrificing the model's core language modelling abilities ^[1] — a model that used a calculator but got worse at ordinary text would be a poor trade.

Costs and limits

Two costs are built into the recipe. Candidate API calls have to be selected during data construction, and a small number of demonstrations has to be kept for each tool, so the cost scales with the number of tools ^[1]. The set of tools is therefore fixed when the training data is built: adding a calculator, a second search engine or a calendar means going back to that step ^[1]. What the approach buys in return is a model that treats a tool as one more option in its own generation, and that has learned to weigh the option rather than being told by a script.