AI Agent Memory: The Definitive Guide

This guide is for anyone building or working with AI agents, looking for an in-depth introduction to AI agent memory. We'll cover what agent memory is, how it compares to the other ways of bringing knowledge into LLMs, and different AI memory architectures — from vector stores to knowledge graphs. Along the way, we'll also show how these ideas are implemented in Cognee, an open-source memory layer for AI agents.

TL;DR

• LLMs are stateless by design. Every call starts from zero. Your agent only "knows" what's in the context window, so continuity within and across sessions requires memory.
• AI agent memory is an external, writable data layer. It captures data accumulated across tasks, sessions, and external sources, and provides shared context that can be reused by one AI agent or a fleet of agents.
• AI agent memory is more than a database. The memory layer also decides what gets stored and how — using meta-reasoning, temporal context, and a shared world model.
• AI agent memory can use different backends: vector stores (support semantic search but no relationships), plain files (simple but don't scale), knowledge graphs (connected and explainable but more complex to implement).
• Complex AI agents and multi-agent systems need hybrid memory. No single data store can handle all aspects. You'd often need to combine several, and connect memory with existing systems so that multiple agents can exchange data and communicate.
• AI memory must be actively managed, not just accumulated. It's not a static snapshot: it needs consolidation, updates when facts change, and eventually forgetting.
• Cognee implements these principles as an open-source memory layer. It combines vector, graph, and relational stores and exposes simple primitives for AI agents to operate on memory, supporting long-term, continual learning.

Why AI agents need memory

Before we talk about AI agent memory, let's first look at the problem it solves.

By design, an LLM is stateless. This means that every call to the model is completely independent: the LLM reads your input and returns a response. If you send another request, it's like the first one never happened.

But if you want to build an AI system that can complete tasks over time — such as a conversational assistant or an AI agent — it needs to be stateful. It has to remember information from previous interactions. Otherwise, it won't be able to handle tasks that span multiple steps or work with information that doesn't fit inside the LLM's context window.

In practice, we usually talk about AI agent memory on two timescales:

• Short-term memory (sometimes called a session buffer) keeps track of what's happening right now. It stores the context of the current interaction — like previous chat messages, intermediate reasoning steps, tool outputs, or the state of a task. This allows the agent to continue working toward the specific goal without losing progress.
• Long-term memory persists across sessions. It allows the agent to remember facts and user preferences from past interactions, earlier task outcomes, and record effective ways of solving similar problems. This gives the agent continuity, personalization, and the ability to improve over time instead of starting each new session from scratch.

This memory is external to the model. Adding it doesn't require changing the LLM. Instead, memory can exist as a separate component that the AI agent consults before each call. It searches it for any useful information from the past, and adds the results to the next request.

So the delivery mechanism stays the same — the LLM still receives a single prompt. The difference is that the prompt is now assembled from multiple sources: the current user message, the ongoing session buffer, and any long-term memories relevant to the task.

And the hard part is in building the system that can support this functionality: capture, store and retrieve all the necessary data at the right time.

Short-term memory can often be implemented directly inside the AI agent harness. For example, you can add a step that summarizes what has happened so far each time the context window starts to fill up, and then inserts that summary back into the prompt. Though this gets tricky for long-running tasks: summarization is lossy, and often it's not enough on its own. The agent may produce valuable artifacts you don't want to compress, such as intermediate drafts or the results of expensive computations, so you need some place to store them anyway.

Long-term memory usually requires an external memory layer from the start. Here, you often need to persist not only user preferences, history of runs, and previous learnings, but also make external context available to the agent — like company data sources, knowledge bases, APIs, and other information that changes over time. So keeping summaries inside the agent context is no longer enough — you need a dedicated layer that manages what gets stored, how it's organized, and how the right context gets retrieved for each request.

That memory layer becomes even more important as agent work becomes multi-player. We're moving toward a world where AI agents run tasks that take days and involve whole teams — often multiple people and multiple agents working on connected problems, like different tasks inside one company. That requires a shared state, so the memory layer has to support long-running, collaborative work, not just a single agent.

If you've built software before, this problem may feel familiar. Web servers don't remember users either, so we add databases, caches, and session stores around them. But memory for AI agents is more than a storage problem.

• The read and write patterns are dynamic. In a web application, you usually know what data will be written and queried, so you design the schema around it. With AI agents, data is generated and retrieved as they work, so you don't know upfront what they'll learn or which pieces of information will matter later.
• The data is varied and unstructured. Instead of rows and columns, agent memory may contain code, conversations, documents, decisions, tables, intermediate results, and other artifacts in different formats that don't fit neatly into a fixed schema.
• The information evolves. Much of what becomes memory is created by AI agents as they work, and facts change over time: yesterday's plan can be overwritten, a user preference can change, or new information can supersede what was stored before.

So an AI agent memory layer has to do more than persist data. It also has to make a lot of decisions on top of it: what's worth storing in the first place, how to represent it and make it discoverable, and when it should be updated or removed. Those are memory design problems in their own right, separate from the question of where the data resides.

In the rest of this guide, we explore different approaches, architectures, and tools for implementing AI memory.

What is AI memory?

One thing worth clarifying is that memory in AI can refer to several things, and people often use the same term for very different mechanisms.

That's partly a result of history. Bringing extra "knowledge" into LLMs — domain, company, or user-specific — is something developers have worked on since day one. Over time, these approaches evolved: from training knowledge into the model, to engineering it into the prompt, to storing it in external systems. Many of these ideas collide under the "AI memory" umbrella, even though they solve different parts of the problem.

Let's untangle the terms you may come across. The table shows an overview:

These aren't competing options — a complex agentic system may well use all four. But each represents a different way of getting data to the model, so let's go through them one by one.

LLM memory

The most confusing term of the bunch is "LLM memory" — people use it to mean at least three different things.

The first is the LLM's "knowledge" acquired during training. In everyday use, even developers say "LLM memory" when they mean exactly this — what the model knows. (Like: "This model doesn't even remember who the current president is!")

However, this is not a memory in any real sense. This built-in "knowledge" helps answer general questions, but it contains nothing about your users or the tasks the AI agent solves.

In simplified terms, an LLM is a transformer that predicts the next token: during training, it compresses statistical patterns from its training data into billions of weights, and at inference it uses those weights to produce likely continuations. There's no lookup, no stored record of any fact — when the model "remembers" the president, it's reproducing a pattern, not consulting a database. That's also why it can easily reproduce a pattern that's outdated or was never true.

But that naturally raises the question: if the model's learned representation of the world lives in its weights, why not update those weights to add new domain-specific knowledge?

Indeed, one way to add new knowledge is to fine-tune the LLM. Or, at the extreme, to train a custom LLM from scratch. Both have been used to make AI systems more domain-aware: for example, BloombergGPT trained a 50B model on decades of financial data, and more recently Cursor trained Composer, its own coding model, optimized for agentic work in large codebases.

But this training or fine-tuning is a one-time, static injection of data, which means:

• It can't hold a session or task state. The model's knowledge stops at training time — you can bake in evergreen information specific to your company or workflow, but nothing from a new user session or a running task will be captured.
• It is slow and expensive to update. Adding a single new fact requires training again — you can't just append an entry to a database.
• It is hard to inspect or delete. You can't see or selectively remove what the LLM absorbed, and you risk degrading its general abilities if you try.

So while fine-tuning has its uses — especially teaching style, format, or adapting to niche tasks — it doesn't solve AI agent memory by itself.

The term "LLM memory" also appears in another context: research on memory mechanisms inside LLMs. This work ranges from managing the LLM's working memory at inference time — how it handles what's already in the context as sequences grow long — to designing new architectures with memory built in. Titans (Behrouz et al., 2024), for example, adds a neural memory module that learns what to keep as it goes, and Google's follow-up Nested Learning work (NeurIPS 2025) continues this direction.

But all of this is deep work on model internals, mostly relevant if you train LLMs rather than build AI agents on top of them. And even if future models become better at retaining information on their own, you'll still need application-level mechanisms to manage user data, company knowledge, and long-running tasks.

Finally, "LLM memory" is sometimes used to describe what we now call agent memory. Through 2023-2025, many developers used it as an umbrella label to describe systems that added external storage and retrieval to the LLM, allowing it to keep track of conversations or persist information across sessions. For example, early memory systems like MemGPT (Packer et al., 2023) and MemoryBank (Zhong et al., 2023) framed themselves this way.

In today's terms, we'd rather call them early instances of agent memory — it was unsettled back then because even "AI agent" had no stable definition. A recent survey, "Memory in the Age of AI Agents" (Hu et al., 2025), summarizes this evolution and suggests a clean split: reserve "LLM memory" for what's inside the model, and call everything external agent memory.

So next, let's look at the approaches that can supply knowledge and memories without modifying the model itself.

Context engineering

Regardless of where the information comes from, the only way to get task-specific data to an AI agent is to put it in the prompt and fit it in the LLM's context window.

Context engineering is the process of deciding what goes into that window, in what form, and order. The AI application or agent harness assembles this context before every LLM call.

Within an agent task or a chat session, this often means re-sending the entire conversation history or task context. When it grows too large, the application compresses the earlier parts before adding them back into the prompt.

At first, the context window size was a major limitation. Early LLMs could process only a few thousand tokens, so every token mattered. But context windows have grown rapidly — today's models can handle hundreds of thousands of tokens, enough to fit entire codebases or document collections into a single request. That makes it possible to support fairly long conversations and workflows using context alone.

But even so, a large context window isn't the same as memory:

• The context disappears when the session ends. Unless the application stores it somewhere else, every new conversation still starts from scratch.
• Long prompts are expensive. As the amount of context grows, inference becomes more expensive and often slower, since large amounts of text are re-sent each time.
• Models don't use long contexts perfectly. Keeping everything in context doesn't always improve performance. Information buried in the middle may receive less attention (a phenomenon known as "lost in the middle"), so selecting the right context still matters.
• The context window is finite. No matter how large it becomes, long-running tasks eventually produce more information than fits into a single request, forcing the application to summarize, compress, or discard parts of it.

So context engineering is important, since it solves one part of the problem: how to build the best possible request for the current LLM call. But it doesn't answer where information should live between requests, how it accumulates over time, or how it should be managed. For that, we need external layers.

RAG vs memory

RAG (Retrieval-Augmented Generation) is one specific way of supplying external information to the context window. When the agent receives a request, it retrieves relevant information from an external knowledge source and includes it in the prompt before calling the LLM.

The source can be anything the agent can query: a vector database of embedded documents, a SQL database, a search index, or an external API. For example, an AI support agent might retrieve relevant sections of a product manual before answering a question, or an internal analytics assistant could query the latest sales figures from a CRM system.

On the surface, this looks a lot like agent memory. In both cases, information lives outside the model, is retrieved when needed, and ends up in the prompt. But the difference is what gets stored and how it gets there.

In classic RAG, the store contains an existing knowledge corpus — documents someone wrote, records already in a database, or other information that exists independently of the agent. This works well for bounded workflows, like grounding a support chatbot's answers. But the corpus is rarely organized for multi-purpose agentic workflows: it stores chunked documents, not connected facts. An agent trying to assemble complex context from it can end up issuing query after query, piecing together fragments that a memory system would return in one step.

On top, the agent reads from it, but doesn't write back to it. That's the key distinction — RAG solves the retrieval part, but it doesn't capture anything from the interaction itself. It doesn't remember what the user asked, what decisions were made, or what approaches worked well.

The moment your system starts writing back — capturing facts, recording previous interactions, or storing the outcomes of its own work — you've moved beyond RAG into agentic memory.

Agentic memory

This brings us to the category this guide is about — AI agent memory. Here, the agent not only queries information but also accumulates it over time. It writes relevant data to a dedicated memory store and retrieves it again whenever it's needed.

For example, suppose a user asks an AI sales assistant for this quarter's sales report, including the EU sales figures. The assistant could still use RAG to retrieve the latest numbers from the company CRM. At the same time, it could also query its own memory and discover that the user prefers European sales to be reported in euros and broken down by country. That preference wasn't stored in any company document — it was learned through previous interactions and written to memory by the assistant itself. The final response then combines the factual sales data from RAG with the personalized context from memory.

If the interaction produces new information that's likely to be useful in the future — for example, a new user preference or the outcome of a task — the agent can write that back to memory, making it available for future requests.

What is AI agent memory?

Now that we've introduced the idea, let's make the definition more precise.

AI agent memory is a dedicated data layer that captures and organizes information accumulated across AI agent tasks and sessions — both coming from user interactions as well as external data sources, like pre-existing company data. This memory provides a shared, persistent context that can be reused by a single AI agent or a fleet of agents whenever it's relevant. This context continues to evolve over time as new information is added, existing data is updated, and outdated pieces are removed or archived.

How is it different from RAG or context engineering? The memory system doesn't replace the model context window — it supplies it. And it also works alongside RAG: agents can retrieve data from documents, databases, and APIs, while memory retains what they learn over time.

Is the AI memory layer just a database? No — and it's worth distinguishing the memory layer from the underlying storage. The storage system is simply where the data lives — and your memory can have different data stores at once. The memory layer sits on top of all of them: it controls what information gets stored and in what form, and it exposes operations that let connected systems work with the memory — write, retrieve, update, forget.

Architecturally, the memory layer is often exposed to agents as a tool, though it can also be called by the agent harness directly. The agent writes to it as it works and queries it whenever it needs additional context. Being a separate component, the memory system can also serve multiple agents and humans at once — for example, if it operates as a shared "company brain."

Is agentic memory just stored data? The memories themselves can take many forms: facts about a user, previous successful or failed approaches, intermediate work products, or discovered relationships. So in a way, it's data — but it's useful to contrast memory with plain information: memory is not just bits that got stored, it's a way for the system to understand, over time, what happened, where, and how. So memory has a few important elements:

• Time component. Memories carry a time dimension: when something happened or was believed to be true, and what changed or superseded it.
• Personal context. Memories are tied to a specific user, agent, or team — not generic records, but context about someone and their work.
• Structure. For memory to become memory, raw entries need to be organized into a coherent whole — entities, relationships, categories — not just a pile of records.
• Abstraction. Memory needs to hold compressed forms of the raw data — like summaries or takeaways — so the system doesn't have to re-read everything to use what it knows.
• Meta-reasoning. Memory isn't just captured mechanically — the system has to reason about incoming context: what it means, how it connects to what's already there. That reasoning determines both what gets stored and how it's organized.

We'll come back to these properties when discussing memory architectures. But before that, it's useful to look at the types of information a memory system can store, since they influence some of the implementation choices.

Types of AI memory

Human memory vs. computer memory

Many of the concepts used to describe AI memory come from existing disciplines. When explaining what an AI system remembers, we often compare it to one of two familiar models:

• Human memory, as described by psychology and neuroscience — how brains and neurons store what we experience and know.
• Computer memory, as it works in classic computing — how data is written, stored, and made available to programs.

However, human and computer memory work on different principles, starting with where the processing happens.

Computers separate storage from processing. Data is written to a location, stays there unchanged, and is fetched when a program needs it. Every location has an address, so a program can go straight to the piece it wants and read it back exactly as it was written.

In the human brain, there is no such split. The same neurons that process information also store it, and memory is distributed across the whole organ. Neuroscientists call the physical trace of a memory an engram, and it famously resists being located. It looks less like a file in a folder, and more like a hologram that is everywhere and nowhere at once.

In effect, the brain is memory — it is located in every synapse and neuron. (This lecture covers the topic well.) And there are as many kinds of memory as there are neurons and synapses in the brain. So any distinction between memory types is a simplification. We create these categories because they're useful for reasoning about memory systems, not because they perfectly reflect how memory actually works.

Long-term vs. short-term memory

This is the division we've already introduced, and it comes from the same borrowed source: the short-term/long-term memory model was formalized in 1960s cognitive psychology, which had itself taken the metaphor from the computers of that era.

However, as a model of human memory, this division between STM (short-term memory) and LTM (long-term memory) is now considered simplistic. Computational neuroscience describes the brain's mechanisms differently — O'Reilly and Munakata split them into two:

• Activation-based memory is sustained neural firing: flexible, small, and gone once the thinking stops.
• Weight-based memory is a change in the strength of connections between neurons: durable, with very high capacity, but slow to form.

Short-term memory isn't a separate box in this picture — it's the currently active portion of long-term memory, the same neurons in a different state. So while the STM/LTM distinction remains useful for describing behavior, the idea of two completely separate memory stores doesn't really hold up for humans.

For agentic memory, though, the distinction is often architectural. Short-term and long-term memory can literally be different components: a session buffer managed inside the agent harness, and long-term memory stored externally.

They also differ in how much information they keep and how it's managed. Short-term memory can afford to be verbose — for example, keeping the full conversation history. Long-term memory requires more deliberate choices about what should be retained and how it should be organized for future use. For example, after each session you might choose to store the full transcript, extract only key facts, keep a summary, or discard it entirely. Those decisions can be hard-coded into the memory system, or made dynamically based on the specific interaction.

Types of memory content

Another way to divide memory is by the types of memories it can contain. Psychology has a well-known version of this:

• Episodic — memories of specific events and experiences, like a recent conversation.
• Semantic — general facts and knowledge, like knowing that Barcelona is a city in Spain.
• Procedural — knowing how to do things, like filling in a tax form.
• Working — what's actively in mind for the current task, like holding intermediate numbers while doing arithmetic.

A lot of the discussion about AI memory tries to adapt these categories, with mixed success — AI systems don't have to mirror human cognition. One recent survey proposes a functional split instead, closer to how AI agents actually operate:

• Factual memory — facts about the world and the user.
• Experiential memory — skills, strategies, and lessons drawn from past trajectories.
• Working memory — task-scoped state within the current horizon.

There are probably better ways to group this for specific use cases. But the useful takeaway from all these taxonomies is the same: a memory system will likely hold different kinds of content — and, just like humans, AI agents need to connect them when working on a task.

AI memory architecture

Now let's look at how AI agent memory is actually implemented.

At the high level, any simple memory system needs three pieces:

• A data store where memories are persisted.
• Write logic that decides what memories get stored and how — from naively dumping everything to an extraction or summarization step, or something more elaborate.
• A retrieval interface that lets the AI agent query and find the relevant memories.

The store you choose has a lot of influence on the other two: what write logic makes sense, and what retrieval can do. So let's go through the main architectures by store type — vector databases, files, knowledge graphs, and hybrid systems.

Vector database memory

A common starting point for memory architecture is a vector database. It follows the RAG pattern: memories are stored as embeddings in a database like Chroma, Pinecone, or pgvector — and retrieved through semantic search whenever the agent needs them.

The write path starts by deciding what's worth remembering. In practice, this is often an LLM call that extracts short facts from the interaction (like "the user prefers sales reports in EUR format") or generates summaries, rather than storing raw transcripts.

Each memory is converted into an embedding before being stored. An embedding model turns the memory into a vector — a long list of numbers representing its semantic meaning. The vector is stored together with the original text and metadata such as the user, session, or timestamp. Memories with similar meaning end up close together in the vector space.

The read path uses semantic similarity search. The agent's query is embedded the same way, and the database returns the closest stored memories, sometimes after filtering by metadata such as the current user or session.

For example, when the agent searches for "Q3 sales numbers," it retrieves all semantically related memories. That might include revenue figures, notes from a sales meeting, or last year's quarterly report — ranked by closeness to the original query.

These vector-based memory systems are easy to set up and have mature tooling. But relying on semantic similarity alone has limits:

• No entities. A vector database stores representations of text chunks, not objects. There's no single entity for "EU sales data". Instead, all related information is spread across many entries, and search simply returns those similar to your query.
• Lookups by a specific name, code, or ID are far less reliable. Embeddings group texts by meaning, but exact identifiers carry very little of it. In vector space, "Q3 sales" and "Q4 sales" could be nearly identical, and a search for "Q3 figures" can return data for another quarter, even though the distinction is business-critical.
• No relationships. Each memory is matched against the query on its own, so retrieval can miss a relevant entry (like the explanation of how the "billing export" works) simply because it doesn't resemble the initial request semantically.
• No notion of truth or recency. You can store timestamps as metadata, but similarity search itself doesn't use them: an outdated fact and its correction can rank side by side. Detecting conflicts and updates is logic you have to add.

In practice, teams working with vector databases patch these gaps by adding extras on top: metadata filters, hybrid search (querying by both keywords and embeddings), query expansion (rewriting the request into variants to surface more relevant memories), and reconciliation logic. Each such patch is a step toward the more complex memory systems we'll discuss below.

That's not to say vector databases are unsuitable as a memory backend — they are a core component of most memory systems. They're simple to deploy, scalable, and well supported. But on their own, they don't solve every memory capture and retrieval problem.

File-based memory

Another approach is to drop the database and store memory in plain files. The pattern became popular with coding agents: the agent maintains files like an AGENTS.md or a MEMORY.md that it reads and updates as it works. Claude Code and similar tools work this way.

The write path is just editing text. For example, after the agent extracts a lesson ("sales figures come from the billing export, not the CRM"), it updates the memory file accordingly — adding a new instruction or editing what's already there.

The read path has two flavors. The simplest is to load the whole file into context at the session start. Alternatively, reading memory files is exposed to the agent just like any other tool, so it can look things up on demand — open a topic file, grep the notes folder, and so on.

For example, when a user asks the agent to prepare a Q3 sales report, the system prompt may instruct it to check memory.md before starting. The agent then reads the file, finds notes from previous reporting tasks — such as "use EUR" or "break results down by country" — and incorporates them into the workflow.

If it discovers a new rule during the task, for example that "EU VAT now has to be included", it updates the memory file so the information is available in future sessions.

This approach has a lot of benefits:

• Simple and transparent. Memory is just a text file or several. You can easily inspect it or even add your corrections manually.
• Version-controlled by default. The files live in the code repository, so you can see the history of all changes and revert to a previous state if needed.
• No extra infrastructure. There's no database or embedding pipeline to run — memory is handled through the file system.

But, of course, it has matching cons:

• Retrieval quality is limited. If you load everything into context each time, you run into the same issues external memory was supposed to solve — context can get bloated, and relevant notes lost in the middle. On top of that, it increases token burn, often just to hold something unrelated in memory. On-demand retrieval helps avoid this, but then finding the right note will depend on the agent knowing which file to open and what to look for. And with only filenames and string matching to go on, it can miss things.
• Memory is usually local to a project. Multiple agents editing shared files can get messy fast, and there is often no effective way to share knowledge across independent projects except copy-paste.

To sum up, file-based memory is a good fit for bounded problems, like a coding agent working in one repo. But once the memory grows, gets shared between agents, or needs retrieval beyond filenames, a more structured store makes sense.

Graph-based memory

In this option, instead of keeping memories as isolated entries, you store them as a knowledge graph. This lets you organize the data by mapping relationships between entities.

Entities are nodes in the graph — these could be "sales report," "EU deals," "billing export." Relationships are edges between them, each one a stored fact: the sales report covers the EU deals; the EU deals are booked in euros; the figures come from the billing export. The useful property of this structure is that you can traverse it: start at a node and follow its edges to connected nodes. Facts may have been written at different times, in different sessions — but once they're in the graph, they're connected, and retrieval can follow the connection.

You may have come across GraphRAG — notably Microsoft's implementation (Edge et al., 2024), which applies the same idea to static document collections. Their observation was that plain RAG fails on global questions, like "what are the main themes here?" So they suggested an approach that has an LLM derive an entity graph from the document corpus and precompute summaries of related entity clusters.

A memory graph works the same way, but the agent writes to it as it goes — so it also has to absorb facts that update or contradict what's already stored.

On the write path, this requires an extra step that ingests the memory into the graph. An LLM extraction converts the new information into nodes and edges.

For example, when a new fact arrives — "EU VAT must be included in the sales report" — it's broken into two entities and a relationship: an "EU VAT" node, a "sales report" node, and an "included in" edge between them. If either node (like "sales report") already exists in the graph, the new edge attaches to it. That's how the graph grows connected instead of fragmented.

The hard part is often entity resolution: deciding whether "the report," "the EU sales report," and "q3_summary.xlsx" are one node or three. In practice, nodes usually carry types — document, person, project — often guided by a preset schema that constrains the graph structure.

On the read path, retrieval usually starts with a semantic search to find the entry point: which node is this query about? For example, if the agent looks up "EU sales data", it may land on the "EU deals" node — using plain semantic matching. But from there, the system follows the node's edges: to the sales report that covers these deals, to the fact that they're booked in euros, to the billing export behind the figures.

So while the request only mentioned the EU sales data, traversal also brought in other relevant memories. A simple vector store can't do this: it can only return entries that are semantically similar to the query itself (or to expanded variants of it, if query expansion is used). A graph can return entries that are logically related to it.

This brings more precise retrieval that is also explainable — every fact has a path you can trace through the graph. If you already work with structured data, like a CRM or an org chart, it also maps naturally into this representation because it already consists of entities and relationships.

But a graph-based memory system also comes with additional complexity:

• A more complex write pipeline. Every new memory has to be mapped to existing relationships, which means more LLM calls and latency.
• More challenging to implement. Building and operating a graph-based memory system requires more infrastructure than a simple vector store, including entity extraction, resolution, and graph traversal.
• The graph needs design and ongoing maintenance. Someone has to define and maintain the schema and keep the graph consistent as new information arrives.

To sum up, graph-based memory is a good fit when relationships between pieces of information are central to the task, but the scope of the task should justify the added effort.

Hybrid memory systems

So far, we've looked at each AI agent memory architecture separately. The table below summarizes each approach and how it solves a different part of the problem.

But the choice doesn't have to be exclusive: a single memory system can combine several stores in a single layer, with logic on top deciding what goes where.

An early example of this idea is MemGPT (Packer et al., 2023). It demonstrated an approach for a conversational agent where the memory splits into tiers. Inside the prompt: a small block of key user facts, plus a rolling queue of recent messages — essentially short-term memory managed through the context window. Outside: a full message log, and an archival memory backed by a vector database. Within a session, every message is logged automatically, and as the context window fills up, older messages are compressed into a running summary. In parallel, the LLM decides which facts deserve promotion into the always-available memory block and which should be archived for long-term retrieval.

That is one example — but you can imagine how this scales as the problem grows. Supporting a single conversational agent is very different from supporting a fleet of agents working over shared, constantly evolving company data. Here are some of the things that tend to complicate memory design in practice:

• Different types of data need different storage. Exact terms — IDs, names, project codes — are best suited for a structured table and keyword index. Relationships want a graph. Fuzzy matching wants vectors. So whenever you deal with different data sources, one database is usually an oversimplification.
• Time-aware queries need temporal metadata. "What did the customer prefer last March?" is unanswerable if stored facts carry no notion of when they were true.
• You may inherit existing data sources. Agents often need to work with documents, tickets, CRM, billing database, etc. in whatever state they're in. So memory needs to support ingestion of existing data rather than only a clean greenfield memory workflow.
• Everything gets harder at scale. If you have huge amounts of data, queries can slow down, ingestion backs up, and simple rules that worked at a thousand memories break at a million. So a memory system that operates on a company-wide basis has very different needs compared to a single chatbot.

Most importantly, agents don't operate in isolation. They exchange work with one another and connect to those existing systems — and no single agent's access pattern is the one to optimize for. For any of this to work, all participants need a shared representation of the entities they operate on and the relationships between them. A shared agentic memory layer becomes the common context that ties these systems together in a way that makes coherent sense.

Cognee is an open-source framework built for many of these problems: a memory layer that combines extraction, organization, storage, and retrieval across multiple data sources, rather than a single storage backend. Let's describe its architecture.

Cognee architecture

Cognee is built around two ideas: model the memory as a knowledge graph with structured representations on top, and treat memory as something that keeps learning — not a one-off snapshot.

Different stores. Following the point above, no single database handles all aspects of memory — so Cognee combines three complementary storage layers:

• a vector database (embeddings for chunks, nodes, and summaries)
• a graph database (entities and relationships),
• a relational database (metadata, permissions, pipeline state).

The layers stay linked: every node in the graph has a corresponding embedding, so retrieval can move between semantic similarity and graph traversal without losing track of what's what. This way, your data is both searchable by meaning and connected by relationships.

This architecture can also be deployed on existing logs and legacy data.

Multi-layer knowledge graphs. Ingestion runs a pipeline: incoming data is classified, chunked, and passed through LLM extraction that turns it into entities and relationships; summaries are generated, everything is embedded, and edges are committed to the graph. Only new or updated data is processed on re-runs.

On top of the raw entity layer, Cognee builds structured representations — typed nodes, optionally grounded in a domain ontology, so that something like "automobile maker" and "car manufacturer" collapse into one canonical node instead of fragmenting the graph. This is also what gives multiple agents and connected systems the shared representation we discussed: a common set of entities everyone can refer to. With temporal mode enabled, facts additionally carry time information for time-aware queries.

A memory-native API. The memory layer is exposed to agents as four functions. These are the operations an AI agent may need to invoke when working with memory:

import cognee

# store and connect new information
await cognee.remember("The sales report covers the EU deals.")

# session memory: fast cache, syncs to the graph in the background
await cognee.remember("User prefers EUR format.", session_id="chat_1")

# query: auto-routes to the best search strategy (vector, graph, or both)
results = await cognee.recall("What format should the sales report use?")

# memory re-weights itself from use
await cognee.improve()

# remove data that should no longer be used
await cognee.forget(dataset="main_dataset")

These four cover the full memory lifecycle:

• remember stores and connects new information — a document, a conversation, a correction, a tool result. Under the hood, it runs the whole write path: ingestion, extraction, and graph building.
• recall queries the memory with auto-routing to choose the best retrieval strategy: it checks session memory first, then falls through to the graph, choosing between semantic search and traversal depending on the query.
• improve updates memory from use, corrections, and feedback — reweighting what's stored, so the system gets better instead of only getting bigger.
• forget removes data that should no longer be used — a stale dataset, a correction, a deletion request.

Because the interface is this small and uniform, any agent can operate on the same memory the same way — whether it's called a library, a REST API, or an MCP tool. Different agents, and different sessions of the same agent, share one memory through one set of verbs.

AI memory lifecycle

What's important is that AI agent memory is not a static snapshot — its contents need to be maintained as new information arrives or existing facts become outdated. Otherwise, the system will degrade with more time and data. This maintenance involves several processes.

Consolidation and reflection. Raw entries tend to accumulate faster than they become useful, so memory systems need to compress them. You want to have fewer, more useful records rather than lots of disconnected small ones. The idea goes back to Generative Agents (Park et al., 2023), whose agents periodically reflect — draw higher-level conclusions from accumulated memories and write those back as new entries.

In Cognee, this runs on the graph: summaries and derived facts are generated on top of the raw entries and stored as their own layer. The system can also infer new connections from facts it already holds — and the connections it adds stay traceable to the facts behind them. Agents work from the compressed view without losing the underlying records.

Updating and forgetting. Things change, and not everything in memory should survive or receive the same degree of attention.

When a fact is updated, a simple decision is to overwrite it, but then you'd lose useful history, like what was believed to be true in the past, and why. The alternative is to keep both versions and mark which one is current, or otherwise modify its relevance. Early systems like MemoryBank (Zhong et al., 2023) modeled this on the Ebbinghaus forgetting curve: every memory carries a strength that fades with time and is reinforced on recall.

Cognee handles this through a set of mechanisms:

• Temporal mode stores facts with time information, so this metadata is preserved and accessible for future use.
• No hard overwrites by default. Changing memory doesn't have to mean rewriting the graph. Structural edits — deleting nodes, merging entities — are destructive: the old state is gone, and mistakes can propagate. Instead, memory refinement happens in continuous space, gradually re-weighting memories. Hard deletion (the "forget" function) is reserved for data the user decides must actually go, and is invoked explicitly.
• Continuous reweighting from a broader set of signals. To prioritize the most relevant memories, Cognee uses a broad set of signals — not just whether a specific memory got recalled, but also how useful it proved. When the agent's answers get evaluated, that signal propagates back to memory, and confirmed answers strengthen their sources. Similarly, corrections attach to the memories that produced a wrong answer, pushing misleading entries down rather than blindly overwriting them.

This continual self-improvement loop runs through the improve operation: agents can call it at the end of a session to apply feedback weights, fold session learnings into the permanent graph, and enrich it further. Under the hood, the enrichment pass is powered by Memify, Cognee's memory-refinement pipeline.

Evaluating AI agent memory

Choosing the AI agent memory architecture is ultimately an engineering trade-off. Different applications place different demands on agentic memory, so there is no single "best" design. A memory layer for per-user personalization, a shared context for multi-agent workflows, and a knowledge base over a document corpus all succeed on different terms.

In practice, AI memory quality can be evaluated in two ways: using external memory benchmarks and application-specific tests.

Memory benchmarks can provide a useful directional signal. Some of the most widely used today are LoCoMo, LongMemEval, and BEAM. BEAM is the most demanding of the three: it probes ten distinct memory abilities — fact tracking, information updates, contradiction resolution, temporal ordering, and more — over conversations up to 10M tokens.

In a recent comparison, Cognee reached 79% on BEAM's 100k-token setting, compared with the previously reported state of the art of 73.4%, and 67% at 10M tokens versus 64.1%, while token usage remained roughly flat as stored data grew. (Full methodology and reproduction code are in the BEAM deep dive).

However, we treat these numbers the way we'd advise you to treat any benchmark result: not as a definitive measure of quality, but as a directional signal that our approach of structured, graph-based memory holds up under a genuinely stressful test.

The reason is that all benchmarks have important blind spots. By design, they have to evaluate memory in a general way, independent of any particular application. That inevitably limits what they can measure. Most memory benchmarks focus on a narrow set of capabilities — often how well the system recalls the right information under controlled conditions.

However, scoring well on retrieval doesn't necessarily translate into a reliable production memory system. For example, many benchmarks say nothing about catastrophic forgetting: whether adding new information degrades or overwrites existing knowledge. But memory that cannot reliably preserve knowledge is difficult to trust, regardless of how well it retrieves it.

Ultimately, the most important metric is downstream task performance. In production, memory should be evaluated by the behavior it enables. Does the AI agent stay consistent across long conversations and provide reliable answers? Can a fleet of agents coordinate effectively using a shared state? Does the system avoid repeatedly reconstructing context, keeping both token usage and latency under control?

These behaviors rarely map directly to benchmark scores. That's why the most valuable evaluations are application-specific tests built around your own agentic workflows, with public benchmarks serving as a useful sanity check or starting point rather than the final verdict.

We'll take a closer look at memory benchmarks and evaluation methodologies in a separate in-depth guide.

Try Cognee

If you're building AI agents that need memory across sessions, users, or data sources, you can get started with open-source Cognee in just a few lines: pip install cognee.

Need a managed memory layer? Talk to us about Cognee Cloud.