Agent memory

What agent memory is

A language model can only see a limited stretch of text at a time — its context window. Agent memory is the part of an agent system that stores information beyond that window so it can be recalled later^[1]. The model itself sees only what is in front of it; the memory system is what handles everything else.

How it works: an operating-system analogy

MemGPT, published by Packer et al. in October 2023, proposed virtual context management, a technique inspired by the hierarchical memory systems of operating systems^[1]. An operating system gives a program the appearance of more memory than is physically present by moving data between fast and slow storage; MemGPT applies the same move to the context window, providing the appearance of large memory resources through data movement between fast and slow memory^[1]. It manages different memory tiers so that extended context is available within the model's limited context window^[1]. Control flow between the system and the user is handled with interrupts^[1].

A small illustration: an agent working through a long document could keep a short summary of earlier chapters inside the window while the full text sits outside it; when a question needs a detail from the beginning, that passage is moved back in. The window only ever holds what is needed at that moment, but to the user it looks as though the whole document is at hand.

Where it came from

The work frames the limited context window as the core constraint that any agent memory design has to work around^[1]. Virtual memory suits the problem because it faces the same shape of difficulty: fast storage is small, slow storage is large, and scheduling is what hides the gap between them.

What was tested

The authors evaluated the design on document analysis, where MemGPT analysed large documents that far exceed the underlying model's context window^[1]. They also evaluated multi-session chat, where MemGPT could create conversational agents that remember, reflect and evolve through long-term interaction with users^[1].

Why it matters, and where it stops

For an agent, holding information beyond the window is what separates a one-off answer from work that continues across sessions. The cost is that material has to be chosen, summarised and moved in and out of the window, and a tiered memory makes the system more complicated than calling a model directly. The starting point stays the same: the limited context window is the constraint that shapes the whole design^[1].