Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

The problem

A large language model is usually given a prompt that contains a question, often preceded by a few worked examples. When those examples show only the question and the final answer, the model has to produce the answer in one step. Many tasks are easy that way; multi-step arithmetic and logic problems are not, because one wrong intermediate quantity spoils the result. The paper, submitted to arXiv on 28 January 2022 by Jason Wei and colleagues on the Brain team at Google Research and published at NeurIPS 2022, asks whether the examples themselves can be written differently in order to draw out reasoning.^[1]

The method

The change is confined to the prompt. Instead of question-and-answer pairs, the examples are written as worked solutions: each one sets out the intermediate steps that lead to the final answer, not just the answer itself.^[1] A model that has seen several such examples tends to imitate the format on the new question, writing out its own intermediate steps in ordinary language before stating a result. In the experiments described in the paper, eight examples of this kind were placed in the prompt.^[1]

The authors called this chain-of-thought prompting. Its appeal is that it requires no change to the model's weights and no extra training: the same model, given a differently written prompt, produces a chain of reasoning where before it produced a bare answer.^[1]

The results

The gains were large. Chain-of-thought prompting brought substantial improvements across arithmetic, commonsense and symbolic reasoning tasks.^[1] On GSM8K, a benchmark of math word problems, PaLM 540B moved from 17.9% accuracy with standard prompting to 56.9% with chain-of-thought prompting, and with eight chain-of-thought examples it reached state-of-the-art accuracy on that benchmark, above a fine-tuned GPT-3 equipped with a verifier.^[1]

The effect depended on model size. The benefit appeared mainly in large models; small models produced fluent but wrong reasoning chains, text that reads like a sensible derivation yet arrives at an incorrect answer.^[1]

Why it matters

The paper is a starting point for the reasoning models that followed, which are trained to think step by step rather than to answer directly.^[2] One piece of that later work is described in the DeepSeek-R1 report: OpenAI's o1 models are described there as the first to scale inference by lengthening the chain-of-thought reasoning process, and DeepSeek-R1 itself is trained with reinforcement learning to produce long chains of thought.^[2] The intermediate reasoning that chain-of-thought prompting elicits with a handful of examples thus becomes, in later systems, something a model is trained to produce at length.