Context engineering for agents with LangChain

Who this is for and what you will be able to do

This guide is for developers who can already call a large language model and now want an agent that behaves reliably. After reading it, you should be able to describe what information each step of an agent hands to the model, decide what belongs in the prompt, in tools, and in the steps between calls, and use LangChain middleware to implement one or two of those choices in code.

What context engineering means

When an agent fails, the usual cause is not that the model is not smart enough. The model did not receive the context it needed. LangChain names two reasons an LLM fails: the underlying model is not capable enough, or the right context was not passed to it. The second reason is the more common one, and it is the main reason agents are unreliable^[1].

Context engineering means providing the right information and tools, in the right format, so the LLM can accomplish a task^[1]. LangChain calls this the number one job of AI engineers^[1].

The agent loop

A typical agent loop has two steps^[1]:

• Model call: the LLM receives a prompt and the available tools, and returns either a response or a request to execute tools.
• Tool execution: the tools the LLM requested run, and their results are returned.

The loop continues until the LLM decides to finish^[1]. To make an agent reliable, you control what happens at each step and what happens between steps.

Three kinds of context

LangChain divides what you control into three groups^[1]:

• Model context covers what goes into each model call: instructions, message history, tools, and the response format. It is transient and affects only that call.
• Tool context covers what tools can access and produce. Tools read from state, store, and runtime context, and they can write results back. This context is persistent.
• Life-cycle context covers what happens between model calls and tool calls, such as summarization, guardrails, and logging. Changes at this layer are also persistent^[1].

Data comes from three places^[1]. Runtime context is static configuration scoped to a conversation, such as user IDs, API keys, database connections, permissions, and environment settings. State is short-term memory scoped to a conversation, such as current messages, uploaded files, authentication status, and tool results. Store is long-term memory that spans conversations, such as user preferences, extracted insights, and historical data^[1].

Defining a tool the model can use

Tools are the interface between the model and external systems. Each tool needs a clear name, a description, argument names, and argument descriptions. These details guide the model when it decides whether to call a tool and what arguments to pass^[1]. The Python example below defines a tool that searches orders by status:

from langchain.tools import tool

@tool(parse_docstring=True)
def search_orders(
    user_id: str,
    status: str,
    limit: int = 10
) -> str:
    """Search for user orders by status.

    Use this when the user asks about order history or wants to check
    order status. Always filter by the provided status.

    Args:
        user_id: Unique identifier for the user
        status: Order status: 'pending', 'shipped', or 'delivered'
        limit: Maximum number of results to return
    """
    # Implementation here
    pass

The docstring states what the tool is for, and the Args section describes each parameter. The model sees these descriptions, so write them precisely. Listing the allowed status values directly in the description is one example.

Too many tools confuse the model

Too many tools can overwhelm the model's context and increase errors. Too few tools limit what the agent can do^[1]. The fix is dynamic tool selection: choose which tools a given model call can see, based on authentication state, user permissions, feature flags, or the stage of the conversation^[1].

The examples show two patterns. Before a user is authenticated, only tools whose names start with public_ are kept. While a conversation has fewer than 5 messages, the advanced_search tool is hidden^[1]. Tools can also write to state. A tool can return Command(update={"authenticated": True}) after a successful password check, which records the login in state^[1].

Transient and persistent changes

A change made in wrap_model_call affects only that model call and does not alter what is saved in state. To persist a change in Python, return an ExtendedModelResponse with a Command from wrap_model_call, or update the conversation history from life-cycle hooks such as before_model, after_model, or wrap_tool_call. In TypeScript, wrapModelCall can return a Command directly^[1].

When you inject extra context, append it to the end of the messages. LangChain's examples do this because models pay more attention to the final messages^[1].

Long conversations and summarization

Long conversations increase token usage and latency. SummarizationMiddleware addresses this. When the conversation exceeds a token limit, it summarizes older messages with a separate LLM call, replaces them with a summary message in state, and keeps recent messages intact^[1]. The example below sets the trigger and the retention window:

from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware

agent = create_agent(
    model="gpt-6.1-sol",
    tools=[...],
    middleware=[
        SummarizationMiddleware(
            model="gpt-5.4-mini",
            trigger={"tokens": 4000},
            keep=("messages", 20),
        ),
    ],
)

In this code, summarization starts at 4000 tokens, and the last 20 messages are kept. The summary is permanent. In later turns the model sees the summary instead of the messages it replaced^[1]. Choose your own thresholds for your workload rather than copying these values.

Limits and common questions

• Once summarization runs, the replaced messages are no longer sent to the model. If later questions depend on early details, raise the threshold or move those details into store^[1].
• The store examples use InMemoryStore from langgraph.store.memory^[1]. It suits experiments, but its contents do not survive a process restart, so long-term data needs a persistent store.
• LangChain recommends starting with static prompts and tools, adding one context feature at a time, and monitoring model calls, token usage, and latency^[1].

Next steps

The LangChain middleware documentation covers the full set of built-in middleware, including SummarizationMiddleware and LLMToolSelectorMiddleware, along with the hooks available to custom middleware^[1]. The examples are written in both Python and TypeScript. The TypeScript versions use createAgent, createMiddleware, and initChatModel from the langchain package^[1].

Adapted from LangChain's documentation page *Context engineering in agents*^[1], released under the MIT License.