Memory for AI agents

Who this is for and what you can do after reading

This guide is for developers adding memory to agents, or hoping their agents will get better at knowing users over time. After reading, you should be able to tell short-term memory from long-term memory, know when session context is lost, and choose a starting point among Mem0, Cognee, Azure AI Search and knowledge agents.^[1]

Why agents need memory

Two things stand out about AI agents: they can call tools to complete tasks, and they can improve over time. Memory is the foundation of agents that improve themselves.^[1]

Without memory, AI applications are often stateless. Each interaction starts from scratch, and the agent forgets earlier context and preferences, which makes the experience repetitive and frustrating.^[1]

With memory, an agent can learn from past actions and outcomes, keep context across an ongoing conversation, respond to needs based on historical data, and act more independently using stored knowledge. The aim is to make agents more reliable and more capable.^[1]

The types of memory

Working memory works like scratch paper for a single task or line of reasoning. It holds the information needed to compute the next step, and it pulls out key items such as requirements, proposals, decisions and actions, even when the full chat history is long or truncated. A travel booking agent might keep a request such as I want to book a trip to Paris in its current context.^[1]
Short-term memory keeps information for one conversation or session, so the agent can refer back to earlier turns. If a user asks what a flight to Paris costs and then asks about accommodation there, short-term memory tells the agent that there means Paris.^[1]
Long-term memory persists across many conversations and sessions. It stores user preferences, past interactions and general knowledge, which makes personalization possible. For example, an agent can record that a user enjoys skiing and outdoor activities, likes coffee with a mountain view, and wants to avoid advanced slopes after a past injury, then use that in future trip planning.^[1]
Persona memory helps an agent keep a consistent personality or role. If the agent is designed as an expert ski planner, persona memory reinforces that role and shapes its responses to match it.^[1]
Workflow or episodic memory records the sequence of steps in a complex task, including successes and failures. If a flight booking fails because the seat is unavailable, the agent can record the failure and try other flights next time.^[1]
Entity memory extracts people, places, things and events from conversations to build a structured picture of what matters. From a past trip, an agent might extract Paris, the Eiffel Tower and dinner at Le Chat Noir, then offer to book that restaurant again later.^[1]
Structured RAG extracts dense, structured information from conversations, emails and images. It uses the structure of the information, not only semantic similarity. For example, flight details such as destination, date, time and airline can be parsed from an email, which makes a precise question such as which flight was booked to Paris on Tuesday answerable.^[1]

Session memory with AgentSession

In the Python SDK samples for Microsoft Agent Framework, short-term memory maps to AgentSession, created with agent.create_session(). As long as the same session is reused, the framework keeps the conversation context available.

That context is not saved when the session ends or the application restarts. Facts and preferences that must survive across sessions need long-term memory, usually in a database, a vector index or another persistent store.^[1]

Two approaches to long-term memory

Mem0 is a persistent memory layer that turns stateless agents into stateful ones. It runs a two-phase pipeline. In the extraction phase, an LLM summarizes the conversation history and pulls out new memories. In the update phase, an LLM decides whether each memory should be added, modified or deleted. Memories are kept in a hybrid data store that can include vector, graph and key-value databases.^[1]

Cognee is an open-source semantic memory for agents. It turns structured and unstructured data into knowledge graphs that can be queried and are backed by embeddings. Its dual-store design combines vector similarity search with graph relationships, so it can find similar content and also understand how concepts relate. Its hybrid retrieval blends vector similarity, graph structure and LLM reasoning. It keeps short-term session context and long-term memory in one connected graph.^[1]

The sample notebooks cover both approaches: one pairs Mem0 with Azure AI Search inside Microsoft Agent Framework, and the other builds structured memory with Cognee.^[1]

Using Azure AI Search for memory

If you prefer not to use a dedicated memory tool, Azure AI Search can serve as the backend for storing and retrieving memories. It can hold user-specific travel memories, product catalogs or other domain knowledge, so answers are grounded in your own data. It also supports Structured RAG, which extracts and retrieves dense, structured information from large sources such as conversation histories, emails and images.^[1]

Making agents self-improving

A common pattern adds a knowledge agent. It watches the conversation between the user and the main agent and does four things:^[1]

It decides whether any part of the conversation is worth saving, such as general knowledge or a user preference.
It extracts and summarizes the key takeaway or preference.
It stores that information in a knowledge base, often a vector database, so it can be retrieved later.
When a new query arrives, it retrieves relevant stored information and appends it to the user's prompt, giving the main agent the context it needs. This works much like RAG.

Controlling latency and cost

To avoid slowing down the user, a cheaper, faster model can first check whether information is worth storing or retrieving. The heavier extraction and retrieval steps run only when needed. As the knowledge base grows, less frequently used information can move to cold storage to control costs.^[1]

Things to watch for

Short-term memory is not long-term memory. Context that lives only inside a session is gone when the session ends.^[1]
Retrieval is the critical step in memory. Full memory management also covers generating, storing, updating and forgetting (deleting) information.^[1]
This guide covers concepts and choices, not code.