Prompt chaining

A big task — answering questions over a long document, multi-step processing, flows that need intermediate checks — crammed into a single prompt usually fails somewhere. Prompt chaining takes another route: split the task into subtasks, let each prompt's output become part of the next prompt, and approach the final result step by step. This page shows how to split document QA into a two-link chain, plus when and how to break tasks into chains.

What prompt chaining is

Prompt chaining decomposes a task into subtasks: identify the subtasks, give the prompt for one subtask to the language model, and use the result as part of a new prompt ^[1]. The chain applies transformations or other processing to the generated response until the desired output is reached ^[1].

The problem it solves is concrete: some complex tasks cannot be done well with a single, very detailed prompt, no matter how carefully it is written ^[1]. Once split into a chain, each link does exactly one thing, the blast radius of an error shrinks, and you can insert checking or cleaning steps between links.

Beyond performance, chaining increases the transparency, controllability, and reliability of LLM applications: problems are easier to locate, and the performance of each stage can be analyzed and improved separately ^[1]. It works well for LLM-driven conversational assistants and personalized user experiences ^[1]. In operations terms: when a document-reading chain drifts in quality after a month in production, a single-prompt app leaves you guessing from the final output, while a chain lets you inspect each intermediate stage to see whether the document changed, quotes were missed, or the response prompt stopped working.

Document QA as a two-link chain

Take the most common case: answering a question from a long document. Feed the document and question to the model at once, and it often misses the relevant paragraphs. The better design is two prompts ^[1].

The first prompt extracts quotes related to the question. A {{文档}} placeholder holds the document, quotes are wrapped in <quotes></quotes>, and if nothing relevant is found the prompt demands the explicit reply "未找到相关引文！" ^[1]:

你是一个很有帮助的助手。你的任务是根据文档回答问题。第一步是从文档中提取与问题相关的引文，由####分隔。请使用<quotes></quotes>输出引文列表。如果没有找到相关引文，请回应“未找到相关引文！”。

####
{{文档}}
####

The task involves a long context, so the example uses OpenAI's gpt-4-1106-preview model; other long-context LLMs such as Claude work with these prompts too ^[1]. The extraction step returns a quote list ^[1]:

<quotes>
- Chain-of-thought (CoT) prompting[27]
- Generated knowledge prompting[37]
- Least-to-most prompting[38]
- Self-consistency decoding[39]
- Complexity-based prompting[41]
- Self-refine[42]
- Tree-of-thought prompting[43]
- Maieutic prompting[45]
- Directional-stimulus prompting[46]
- Textual inversion and embeddings[59]
- Using gradient descent to search for prompts[61][62][63][64]
- Prompt injection[65][66][67]
</quotes>

The second prompt receives those quotes together with the full document and answers the question from them ^[1]:

根据从文档中提取的相关引文（由<quotes></quotes>分隔）和原始文档（由####分隔），请构建对问题的回答。请确保答案准确、语气友好且有帮助。

####
{{文档}}
####

<quotes>
- Chain-of-thought (CoT) prompting[27]
- Generated knowledge prompting[37]
- Least-to-most prompting[38]
- Self-consistency decoding[39]
- Complexity-based prompting[41]
- Self-refine[42]
- Tree-of-thought prompting[43]
- Maieutic prompting[45]
- Directional-stimulus prompting[46]
- Textual inversion and embeddings[59]
- Using gradient descent to search for prompts[61][62][63][64]
- Prompt injection[65][66][67]
</quotes>

Its output ^[1]:

文档中提到的提示技术包括：

1. Chain-of-thought (CoT) prompting[27]
2. Generated knowledge prompting[37]
3. Least-to-most prompting[38]
4. Self-consistency decoding[39]
5. Complexity-based prompting[41]
6. Self-refine[42]
7. Tree-of-thought prompting[43]
8. Maieutic prompting[45]
9. Directional-stimulus prompting[46]
10. Textual inversion and embeddings[59]
11. Using gradient descent to search for prompts[61][62][63][64]
12. Prompt injection[65][66][67]

每一种技术都采用独特的策略来增强或明确与大型语言模型的互动，以产生期望的结果。

After two links, the answer is built on the quotes screened by the first step, not on the model's fuzzy impression of the whole document. The quotes can also be cleaned before entering the second link, for example stripping the citation flags; following the same pattern, you can append a link that removes citation flags from the response before it reaches the end user ^[1].

The cost of a chain

Splitting is not free; count three bills first. Every extra link means an extra call, so latency and cost grow roughly linearly with the chain. Intermediate results can balloon: passing both quotes and the full document into the second link makes that call larger than the first. And the handover becomes your engineering work — output formats, variable passing, and failure retries between links are code you now maintain. Keep the chain short: merge steps where you can, and reserve a separate link only for stages that genuinely need their own check or format.

Three principles for splitting a chain

When cutting a task into a chain, three rules help:

1. Each link does one kind of operation: keep extraction, rewriting, checking, and summarizing in separate links, so a failure points to a specific stage.
2. Agree on an explicit handover format between links — wrapping the payload in markers like <quotes></quotes> tells the next prompt where its input begins and ends.
3. Give critical links an empty-result path: if the first link finds nothing, it says so outright instead of inventing an answer ^[1].

When to use a chain

Two signals justify chaining: the task needs several different kinds of transformations, or a single prompt cannot cover every detail however hard you try. In the first case, split the task; in the second, you at least gain controllability, since every intermediate input and output can be inspected on its own. In the other direction, do not split a task into a chain for architecture's sake: if one call handles it reliably, the extra cost and latency buy nothing. More chaining examples built around the Claude LLM can be found in Anthropic's prompt chaining documentation ^[1].

Adapted from the prompt chaining section of DAIR.AI's *Prompt Engineering Guide*^[1], released under the MIT License.