Tree of Thoughts (ToT) Prompting

Who this is for and what you will get

This is for readers who already write ordinary prompts but find that multi-step problems, or problems that call for comparing several approaches, produce inconsistent results. After reading it, you can turn the Tree of Thoughts idea into a single prompt and try it on your own question.

What Tree of Thoughts is

Chain-of-thought prompting asks a model to write out its reasoning one step at a time. Tree of Thoughts (ToT) goes one step further. It treats the problem as a tree, where each node is a “thought”: a coherent piece of language that represents one intermediate step toward a solution.^[1]

Instead of following a single path, ToT has the language model generate thoughts and also evaluate them. Search algorithms then decide which branch to continue and when to backtrack. Breadth-first search (BFS) and depth-first search (DFS) are two such general strategies, and neither is tied to any particular problem.^[1] Yao et al. (2023) proposed the framework for tasks that need exploration or strategic look-ahead, where simple prompting techniques handle poorly.^[1][2]

How the Game of 24 works

The paper uses the Game of 24, a math reasoning task, to demonstrate ToT. Setting up ToT for a task means deciding two numbers in advance: how many thought steps the problem is split into, and how many candidates are kept at each step. For the Game of 24, the problem is split into three thought steps. Each step produces an intermediate equation, and the best five candidates are kept.^[1]

For each candidate, the model judges whether it can reach 24. The answer must be one of “sure”, “maybe” or “impossible”, and each step’s evaluation is sampled three times.^[1] The authors explain the purpose: keep partial solutions that can be verified as correct with a little look-ahead (“sure”), remove impossible partial solutions with common-sense checks such as “too big” or “too small”, and keep the rest as “maybe”.^[1] On this task ToT runs breadth-first search. The results reported in the paper show that ToT clearly outperforms other prompting methods.^[1]

A one-prompt version of ToT

Running the full method requires code that performs the search and manages the tree. Hulbert (2023) condensed the main concepts into a short prompt that asks the language model to evaluate intermediate thoughts within a single prompt.^[4] An example prompt looks like this:

假设三位不同的专家来回答这个问题。
所有专家都写下他们思考这个问题的第一个步骤，然后与大家分享。
然后，所有专家都写下他们思考的下一个步骤并分享。
以此类推，直到所有专家写完他们思考的所有步骤。
只要大家发现有专家的步骤出错了，就让这位专家离开。
请问...

The prompt asks the model to imagine three experts answering the question. Each expert writes their first step and shares it, then the next step, and so on until every step is written. If an expert’s step is found to be wrong, that expert leaves. The design aims to remove flawed steps early. Replace the final line, “请问...”, with the question you want to solve.

How to try it

1. Pick a question that needs several reasoning steps, such as a math problem or a decision that compares several options.
2. Open the chat interface of the model you use and start a new conversation.
3. Paste the whole prompt and replace the final line with your question.
4. Send it, then check whether the answer follows the structure of experts writing steps in turn, and whether any expert is judged wrong and leaves.

Limits and common questions

The output gets longer. Each step generates and evaluates several candidates, so ToT uses more generation than a direct answer.^[1]

The structure has to be defined. ToT needs the number of thought steps and the number of candidates kept at each step set for each task. For a different kind of problem, those numbers usually have to be chosen again.^[1]

The one-prompt version is simplified. It condenses the main idea into a single prompt, with evaluation happening inside one response. It is not the same as the BFS or DFS search procedure in the paper.^[4]

The reported gains come from specific tasks. The paper’s experiments, including the Game of 24, do not show that ToT wins on every problem.^[1]

A related approach

Long (2023) proposed a similar core idea, using a search tree over multiple rounds of dialogue to strengthen an LLM’s ability to solve complex problems.^[1][3] The main difference is how the search is controlled. Yao et al. (2023) use DFS, BFS or beam search, while Long proposes a “ToT Controller” trained with reinforcement learning, which decides when to backtrack and how far back to go.^[1][3]

A controller like this could learn from new datasets, or through self-play the way AlphaGo learned, rather than relying only on brute-force search. General search strategies do not adjust to a specific problem in the same way. Even with a frozen LLM, a ToT system built on reinforcement learning could keep improving and learn new knowledge.^[1]

Next steps

For the full method, read the paper itself.^[2] For Long’s version of the idea, see the arXiv paper.^[3] For the condensed prompt, Hulbert’s repository is the place to start.^[4]

This article is adapted from the Tree of Thoughts (ToT) page in DAIR.AI’s *Prompt Engineering Guide*^[1], released under the MIT License.