Planning in AI agents

What it is

Planning in an AI agent is the work of turning a goal into intermediate steps and deciding what to do next, instead of generating a final answer in one pass.^[1][2] The emphasis is on decomposition and action selection. For a task that needs several pieces of evidence, an agent may first decide to search, read the result, then decide whether to search again or stop. Planning matters because single-pass generation can carry an early mistake into later steps; a plan creates points where the model can inspect, revise, and continue.^[1][2]

How it works

A common planning loop is: represent the goal, propose intermediate steps, choose the next action, observe the result, and update the plan. ReAct is one concrete form of this loop: the model generates reasoning traces and task-specific actions in an interleaved manner, rather than separating reasoning from acting.^[1] The reasoning traces help the model induce, track, and update action plans and handle exceptions; the actions let it query external sources such as knowledge bases or environments.^[1]

ReAct was published by Yao et al. in October 2022.^[1] On question answering over HotpotQA and fact verification over Fever, it interacted with a simple Wikipedia API and overcame the hallucination and error propagation common in chain-of-thought reasoning.^[1] On the interactive decision-making benchmarks ALFWorld and WebShop, it outperformed imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.^[1]

Tree of Thoughts, published by the same authors in May 2023, generalises chain-of-thought prompting into exploration over coherent units of text that serve as intermediate steps.^[2] The model can consider multiple reasoning paths, self-evaluate choices, look ahead, and backtrack when necessary.^[2] The framework targets tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role, which left-to-right decoding handles poorly.^[2] On the Game of 24, GPT-4 with chain-of-thought prompting solved only 4% of tasks, while Tree of Thoughts reached a 74% success rate.^[2]

Where it came from

ReAct and Tree of Thoughts are two formulations that made planning explicit in language-model reasoning. ReAct interleaves reasoning traces with task actions.^[1] Tree of Thoughts extends the intermediate step from a single chain into multiple paths that can be explored.^[2] Together they shift the model from answering immediately to planning, acting, and revising while working.

Where it is used

Planning appears in agents that call tools, retrieve information, operate environments, or complete multi-step tasks. The usual shape is a loop: decide the next step, execute it, read feedback, update the plan.^[1] ReAct’s interleaving of retrieval and action and Tree of Thoughts’ exploration of multiple paths correspond to two common needs in such systems.^[1][2]

Limits

Planning is not always better when it is heavier. Exploring many paths, repeatedly self-evaluating, and taking external actions add computation and calls; self-evaluation can be wrong, external tools can fail, and simple tasks may not need elaborate planning.^[1][2] The value of planning is to make hard tasks into checkable steps, not to apply the heaviest search to every problem.