How to Build a Memory Graph for AI Agents

TL;DR:

• A memory graph for AI agents is a persistent graph representation of agent memory that stores entities, events, decisions, observations, and sources as connected records with typed relationships.
• Let the agent's recurring questions drive the schema — they determine which identities need nodes, which relationships need edges, and how far retrieval should travel.
• Vector retrieval identifies semantically related records; graph retrieval follows explicit relationships such as dependency, ownership, sequence, causality, and provenance. Hybrid retrieval can combine both.
• Memory graphs need an ongoing update lifecycle for identity resolution, provenance, contradictions, revision, deletion, and bounded retrieval as the graph grows.

A memory graph for AI agents stores information as connected entities, events, decisions, observations, and sources, rather than as text or discrete records. Typed relationships encode dependency, ownership, sequence, causality, and provenance, giving the memory layer structure that plain text or chunk stores don't provide by default.

When stored records are encoded as vector embeddings, a query written in different language can be embedded in the same vector space to identify likely entry nodes. Graph retrieval then follows explicit relationships among those nodes and returns the relevant path along with its time, scope, and supporting evidence.

Recent research describes the graph-based agentic memory lifecycle through extraction, storage, retrieval, and evolution stages. In this guide, we'll turn that lifecycle into an implementation workflow for building and maintaining graph-backed long-term context.

What a Memory Graph Stores

A memory graph connects what an agent has learned, observed, decided, or done with the relationships and evidence that give those records their context. The information can come from conversations, documents, structured data, tool results, task outcomes, human corrections, and earlier memories that have been revised or consolidated.

Nodes: Persistent data points

Common node types include:

• entities, such as people, organizations, services, products, or code components;
• events, such as meetings, incidents, deployments, or transactions;
• decisions, including accepted choices and rejected alternatives;
• observations, such as a preference, constraint, or system state;
• sources, including documents, messages, tickets, commits, or API responses;
• tasks or episodes, which group records created during the same workflow or interaction.

The node model should follow the agent's recurring queries. An enterprise agent can need Customer, Commitment, Person, and Meeting nodes, while a coding agent can need Service, Interface, Decision, Test, and Commit.

Edges: How those data points relate

Typed edges describe what the nodes have to do with one another. For example, a service DEPENDS_ON an interface, a decision CONSTRAINS an implementation, and a claim can be SUPPORTED_BY a source.

Other relationships can capture ownership, sequence, causality, or supersession, giving retrieval more specific paths than a generic RELATED_TO edge.

Relationships can also be embedded with properties such as event time, validity interval, source reference, confidence, project or user scope, verification status, and revision timestamps. These properties support current-state, historical, and provenance queries from the same graph.

Lifecycle rules

Persistent graph memory changes as the agent receives new information: observations can create records, task outcomes can add relationships, later evidence can revise earlier claims, and retention policies can remove records.

A memory implementation needs rules for:

• which observations become durable;
• who can create or revise records;
• how memories are scoped;
• how time and provenance are retained;
• how contradictions and later corrections are represented;
• when records are archived or deleted.

Graph memory can be one part of a broader AI agent long-term memory architecture, alongside agent workflow state and other persistent context.

Some semantic housekeeping

Before we move on, let's distinguish some concepts that relate to different architectural roles in graph-based agent systems:

A memory graph can use a knowledge graph as its representation and GraphRAG as a retrieval method. It also needs admission, revision, deletion, scoping, and access policies because agent activity keeps changing the graph after the initial write.

When Should You Use a Memory Graph?

Vector retrieval works well for semantic lookup across largely independent records. But for agents that repeatedly need to follow explicit relationships, distinguish identities, reconstruct history, or verify provenance, graph memory provides a more direct representation.

Graph traversal can also be constrained by edge types, node classes, time ranges, access scope, hop depth, and returned paths, preventing retrieval from expanding into unrelated parts of the graph.

Here are some examples of which kinds of queries correspond to which retrieval mechanism:

Many natural-language queries don't name the exact graph entity, which makes a hybrid design that combines vector and graph retrieval the most viable option.

A recent multi-graph agentic memory architecture (MAGMA) proposes an example of semantic, temporal, causal, and entity graph relations combined with query-adaptive traversal.

Design the Memory Graph with Retrieval Requirements In Mind

It makes much more sense to start schema design from the questions the system consistently needs to answer, then to first build only the graph structure required for those retrieval paths.

Here are some examples of tracing a real question to the schema element that answers it:

Any proposed node or edge type should support a recurring retrieval path before it gets added.

Define persistent identities and specific relationships

Nodes should represent identities or records the agent needs to recognize across multiple interactions. A schema that turns every noun in the source text into a node quickly becomes harder to query and maintain.

Identity consistency is critical here. References such as NorthStar, NorthStar Inc., and NorthStar Corporation all need to resolve to a single node before new relationships are attached. Otherwise, the graph can fragment one customer's history across several identities.

Keep edge types specific enough to distinguish the connections the application actually queries, as in the table above. A controlled relationship vocabulary also prevents equivalent phrasing from fragmenting one concept across several edge labels.

Add time and provenance where retrieval needs them

Relationships that change over time should carry validity and source metadata. Alice ── WORKS_ON ──> Payments can also need a validity interval, source reference, project scope, confidence score, or verification state.

Temporal awareness lets the graph retain earlier states while identifying which relationship applies to the current query. Provenance gives later retrieval a route back to the evidence behind a claim.

Keep the first schema narrow

Start with the node and edge types required by a small set of recurring queries. Knowledge, temporal, hierarchical, and causal structures can be added later as retrieval requirements expand.

cognee also supports custom graph models when the default schema needs additional node or relationship types.

Building Out the Memory Graph Workflow

Once the schema is defined, the next job is turning incoming information into graph memory that can be retrieved and revised later without losing its source or scope.

Step 1: Define memory scope

Each record needs clear boundaries for where it can be reused. Depending on the application, scope can include:

• user;
• agent;
• organization;
• project or repository;
• task or workflow;
• time range;
• access level.

In cognee, multi-user mode can isolate datasets between users while tenant- and role-based permissions provide controlled shared access.

Apply scope before retrieval — a memory from another user, project, or restricted source should never enter the wrong context.

Step 2: Create the initial graph schema

Define the node types, relationship types, and properties the first retrieval scenarios explicitly need.

For example, a coding-agent graph can begin with:

Service
Interface
Decision
Test
Commit

Service ── USES ──> Interface
Decision ── CONSTRAINS ──> Service
Test ── COVERS ──> Interface
Commit ── CHANGED ──> Service

Step 3: Extract memories and resolve identities

Incoming conversations, documents, tool results, and task outcomes need to be converted into graph records by extracting entities and relationships, resolving references against existing nodes, attaching source information, and assigning scope and temporal metadata.

Duplicate identities fragment one entity's history across several nodes, while an incorrect merge connects records that belong to different entities, so getting this wrong comes with a cost.

Automate merges only under a defined confidence policy, ideally when identifiers and supporting evidence both agree. Leave ambiguous references separate until more evidence or human review resolves them.

Step 4: Retain time, provenance, and verification state

Every consequential claim should keep its source, observation time, validity period, verification state, and replacement status where one applies:

Decision_17
  ├─ CONSTRAINS → AuthService
  ├─ SUPPORTED_BY → ADR_042
  ├─ valid_from → 2026-02-14
  └─ verification_status → confirmed

  (illustrative — field names vary by system)

Separating event time from ingestion time lets the graph place late-arriving information in the correct historical context.

Relationships produced through extraction or reasoning should also retain metadata that distinguishes inferred connections from claims stated directly in a source.

Step 5: Build graph and semantic indexes

Graph traversal is easier to constrain once likely entry nodes have been identified. Natural-language queries usually don't include an exact node identifier, which is why a hybrid knowledge graph memory layer can combine:

• vector embeddings for semantic discovery;
• lexical indexes for exact names and identifiers;
• graph indexes for nodes and relationships;
• metadata filters for user, project, time, and permissions.

The vector index and graph store can be kept in separate systems or share a backend, depending on the wider memory architecture.

The vector search portion can be handled by a wide range of databases, from fully managed services such as Pinecone to self-hosted and open-source alternatives.

Step 6: Retrieve a bounded subgraph

Traversal should follow only the relationships the query needs: an ownership question can follow OWNED_BY, root-cause analysis can use CAUSED_BY and DEPENDS_ON, and historical retrieval can use SUPERSEDES, FOLLOWED_BY, and validity metadata.

A retrieval pass can:

1. identify candidate entry nodes;
2. apply scope, permission, and time filters;
3. traverse selected relationship types;
4. restrict hop depth;
5. cap returned nodes or paths;
6. rank the resulting subgraph.

Without these bounds, one highly connected node can pull the retrieval result far past the context the agent actually needs for the task in front of it.

Step 7: Convert the subgraph into model context

Serialize only the selected paths into compact model context.

The result can preserve entities, relationships, timestamps, source references, verification state, and unresolved conflicts:

Incident_204
  ├─ CAUSED_BY → DeploymentRollback
  └─ IMPACTED → BillingService

BillingService
  └─ COVERED_BY → BillingIntegrationTest

DeploymentRollback
  verified_at → 2026-05-02
  source → incident-reports/204.md

  (illustrative — field names vary by system)

This keeps the structure behind the response while controlling how much context reaches the model.

Step 8: Revise memory as new evidence arrives

New information can add a relationship, change a property, resolve an identity, narrow a validity interval, replace an earlier claim, or create a conflict that needs review.

A changed decision, for example, can stay in the graph as historical context while a SUPERSEDES relationship identifies which version applies now. Historical states can still support retrospective queries, while superseded claims can be excluded from current-state retrieval.

The 2026 graph-based agentic memory survey calls this stage memory evolution, including consolidation, graph reasoning, and structural reorganization.

Build and Query a Memory Graph With cognee

Let's do a walkthrough of the full lifecycle on one compact example: three short sources about a customer account, where two sources name the same customer differently and a third arrives later and changes an earlier relationship.

Everything below runs fully locally with cognee (graph, vector, and relational stores included), and each output shown is real output from the run.

import cognee
from cognee.api.v1.search import SearchType
from cognee.memify_pipelines.consolidate_entities import consolidate_entities_pipeline

DATASET = "northstar_demo"

SOURCE_1 = ("Sales notes, 2026-03-02: NorthStar Inc. committed to the Q3 rollout. "
            "The Q3 rollout commitment is owned by Dana.")
SOURCE_2 = ("Support ticket #88, 2026-03-10: NorthStar depends on the export "
            "feature for the Q3 rollout.")
SOURCE_3 = ("Email, 2026-04-21: Ownership update. The Q3 rollout is now owned by Priya.")

CUSTOM_PROMPT = """Keep the exact surface form of each entity name as written.
Entity types: Customer, Commitment, Feature, Person.
Use ONLY these relationship names: committed_to, depends_on, owned_by."""

# 1) INGEST two sources that name the same customer differently
await cognee.add([SOURCE_1], dataset_name=DATASET, node_set=["northstar_account"])
await cognee.add([SOURCE_2], dataset_name=DATASET, node_set=["northstar_account"])
await cognee.cognify(datasets=[DATASET], custom_prompt=CUSTOM_PROMPT)

# 2) RESOLVE IDENTITY — preview the merge plan, then apply
await consolidate_entities_pipeline(similarity_threshold=0.6, dry_run=True, dataset=DATASET)
await consolidate_entities_pipeline(similarity_threshold=0.6, dataset=DATASET)

# 3) RETRIEVE — vector entry node + bounded traversal, then answer with provenance
query = "Who owns the NorthStar rollout commitment, and what does it depend on?"
ctx = await cognee.search(query_text=query, query_type=SearchType.GRAPH_COMPLETION,
                          datasets=[DATASET], top_k=10, only_context=True)
ans = await cognee.search(query_text=query, query_type=SearchType.GRAPH_COMPLETION,
                          datasets=[DATASET], top_k=10, include_references=True)

# 4) REVISE — later evidence arrives; owned_by is declared single-valued
await cognee.add([SOURCE_3], dataset_name=DATASET, node_set=["northstar_account"])
await cognee.cognify(datasets=[DATASET], custom_prompt=CUSTOM_PROMPT,
                     functional_relationships={"owned_by"})

# 5) RE-QUERY — history preserved; current-state filtering is policy
ans2 = await cognee.search(query_text=query, query_type=SearchType.GRAPH_COMPLETION,
                           datasets=[DATASET], top_k=10)

After step 1, the graph honestly contains two customer identities — each source produced its own subgraph:

node [Entity] 'northstar'          node [Entity] 'northstar inc.'
edge northstar --[committed_to]--> q3 rollout
edge northstar inc. --[committed_to]--> q3 rollout commitment
edge q3 rollout commitment --[owned_by]--> dana

Step 2 previews the merge plan without touching the graph, then applies it. Entity names are embedded and clustered by similarity; the canonical node is chosen deterministically and every edge is re-pointed:

consolidate_entities plan: clusters=2 duplicates=2 edges_to_repoint=8 dry_run=True
merge report: [{"canonical_name": "northstar",  "merged_from": [{"name": "northstar inc."}]},
               {"canonical_name": "q3 rollout", "merged_from": [{"name": "q3 rollout commitment"}]}]

edge northstar --[committed_to]--> q3 rollout
edge q3 rollout --[owned_by]--> dana
edge q3 rollout --[depends_on]--> export feature

Step 3 retrieves through a vector entry node: the natural-language query is embedded, matching nodes seed a bounded graph traversal (top_k, neighborhood_depth), and only_context=True returns the serialized subgraph — with the source chunks structurally attached through contains and is_part_of edges. With include_references=True, the answer arrives with its provenance:

Dana owns the NorthStar Q3 rollout commitment, and it depends on the export feature.

Evidence:
- chunk 1 of document text_68da2f9c... : "Support ticket #88, 2026-03-10: NorthStar
  depends on the export feature for the Q3 rollout."
- chunk 1 of document text_95f0d0ae... : "Sales notes, 2026-03-02: ... The Q3 rollout
  commitment is owned by Dana."

Step 4 ingests the later email. Because owned_by was declared single-valued via functional_relationships, cognee detects the conflict and marks the older assertion — the fact is tagged, not deleted, so history and provenance survive:

Temporal resolution superseded 1 edge(s) across 1 relationship(s)

edge q3 rollout --[owned_by]--> dana   <-- superseded=True,
     superseded_by=3538c89f-..., reason="superseded by a more recent 'owned_by' assertion"
edge q3 rollout --[owned_by]--> priya

Step 5 re-runs the same query. Retrieval now reflects the revision while keeping the history visible, and a current-state view is a one-line filter on the superseded property:

- Owner: Priya (ownership updated by email on 2026-04-21; previously assigned to Dana)
- Depends on: the export feature (per Support Ticket #88, 2026-03-10)

[SUPERSEDED] q3 rollout --owned_by--> dana
[CURRENT]    q3 rollout --owned_by--> priya

One practical caveat: extraction is stochastic, so a later source using a new surface form can mint a fresh node and hide the conflict from temporal resolution. Therefore, it's advised to re-run identity consolidation after each ingest, before conflict resolution — identity resolution and revision are one maintenance loop, not two independent steps.

Maintain the Memory Graph's Accuracy

Long-lived graph memory needs maintenance rules because an incorrect identity or relationship can affect every later retrieval path that uses it.

Here are the most important ones:

Preserve unresolved contradictions

Conflicting sources should stay distinguishable until policy or human review resolves the discrepancy.

Store competing claims separately with their provenance intact:

Claim_A
  ├─ VALUE → September
  └─ SUPPORTED_BY → Source_A

Claim_B
  ├─ VALUE → October
  └─ SUPPORTED_BY → Source_B

Claim_B
  └─ CONTRADICTS → Claim_A

Each claim keeps its own timestamp, confidence, verification state, and validity period. Retrieval can rank competing claims by source authority, verification status, validity period, and recency, or pass the unresolved conflict to human review.

cognee does exactly this — with contradiction_detection enabled, conflicting facts get a contradicts edge carrying a reason and a confidence score, non-destructively.

The same distinction applies to relationships inferred by an extraction or reasoning layer and relationships stated directly in a source. Keeping those categories separate gives retrieval a clearer basis for weighting evidence.

Keep the schema in check

Reuse relationship types for new records rather than letting extraction create a new label whenever it encounters slightly different phrasing.

For example, if WORKS_FOR, EMPLOYED_BY, MEMBER_OF, and BELONGS_TO all represent the same relationship in an application, retrieval gets harder to reason about as those labels accumulate.

Label sprawl is the default behavior of LLM extraction, and a governed edge vocabulary is exactly what cognee's custom graph models provide — relationship names come from typed pydantic fields instead of the extractor's phrasing, which keeps traversal predictable.

Bound graph growth and traversal

Consolidate duplicates, remove or archive records under retention policies, keep indexes updated, and cap traversal by hop depth, relationship type, node count, and total context size to avoid pushing retrieval cost up as memory accumulates.

Preserve permissions through derived memory

A graph record built from restricted information should inherit that source's access controls automatically.

If an agent turns a private document into entities and relationships, those derived records should carry the same access restrictions as the source document. User, organization, project, and document-level permissions need to apply on both the write and read side.

This becomes more important once several agents share one knowledge graph memory layer, where organization, agent, and user memory need separate scopes.

Measure Whether Graph Memory Improves the Agent

Evaluation should answer three questions:

• Does retrieval find the right paths?
• Does the graph stay internally reliable?
• Does the resulting context improve the agent's work?

Retrieval and path quality

Standard retrieval metrics such as Precision@K, Recall@K, and MRR show whether relevant memories rank near the top. Graph retrieval also needs evaluation at the path level: does it start from the correct entity, follow the required relationship types, respect time and access scope, return supporting evidence, and stop before expanding into unrelated parts of the graph?

For a multi-hop question such as:

Which active customer commitment depends on the delayed feature?

The required path could be:

Customer
→ RECEIVED → Commitment
→ DEPENDS_ON → Feature
→ DELAYED_BY → Migration

Path evaluation confirms that retrieval recovers the required nodes and relationships in the correct sequence.

Graph quality

A correct final answer can still hide graph defects that affect later queries, so graph integrity needs its own evaluation.

Relevant metrics include:

Track operational measures such as update latency, extraction cost, graph-write cost, traversal latency, and returned context size separately.

Downstream agent performance

Task-level evaluation asks whether graph memory improves the work the agent does as well as the graph's own retrieval quality.

For a coding agent, evaluation can track task completion, root-cause accuracy, correct identification of affected components, exploration tokens and tool calls, and time to a validated result. For an enterprise agent, the focus can be entity resolution, historical accuracy, commitment tracking, multi-hop question answering, and correct attribution to source records.

LoCoMo and LongMemEval offer broader long-term-memory tests. LoCoMo evaluates reasoning over extended multi-session conversations, while LongMemEval targets information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. MAGMA evaluates a multi-graph memory architecture against both benchmarks.

Memory Graph Implementation Checklist

Here's a practical checklist to confirm the system can preserve identity, relationships, evidence, and access rules across the full memory lifecycle before graph memory goes into a production agent workflow:

Retrieval requirements

• ☐ Define the recurring questions the graph must answer.
• ☐ Identify which queries need semantic, structural, temporal, causal, or provenance retrieval.
• ☐ Confirm graph traversal returns information simpler retrieval would miss.

Schema and identity

• ☐ Keep the initial node and edge vocabulary small.
• ☐ Use controlled relationship types.
• ☐ Assign canonical identities where possible.
• ☐ Define how aliases, duplicate entities, and uncertain matches get handled.
• ☐ Add schema elements only when a recurring query requires them.

Time and provenance

• ☐ Preserve source references for consequential claims.
• ☐ Distinguish event time from ingestion time where needed.
• ☐ Record validity intervals for relationships that change.
• ☐ Represent superseded and contradictory records explicitly.
• ☐ Keep verification or approval state attached to important memory.

Retrieval

• ☐ Maintain semantic or lexical indexes for natural-language queries that need help finding entry nodes.
• ☐ Apply user, project, permission, and time filters before traversal.
• ☐ Restrict traversal to relevant relationship types.
• ☐ Cap hop depth, returned paths, and total context size.
• ☐ Preserve relationship structure and provenance when converting a subgraph into model context.

Updates and governance

• ☐ Define which agent observations can become durable memory.
• ☐ Set policies for merges, revisions, contradictions, and deletion.
• ☐ Preserve access restrictions in derived graph records.
• ☐ Require review for high-impact entity merges or relationship changes.
• ☐ Monitor graph growth, duplication, and unsupported relationships.

Evaluation

• ☐ Measure retrieval and path quality.
• ☐ Track entity-resolution and graph-integrity errors.
• ☐ Test temporal and provenance accuracy.
• ☐ Compare agent outcomes with and without graph memory.
• ☐ Include retrieval latency, token use, storage, extraction, and maintenance cost.

Every Retrieved Relationship Needs to Be Defensible

A memory graph is only as good as the path it can hand back later. Extracting more entities or accumulating more edges doesn't help if the agent can't tell which relationships still apply, where they came from, or what changed after they were written.

Provenance, identity, and revision history need to travel with the relationships retrieval returns, giving the agent enough evidence to verify them before acting.

Maintained this way, the graph becomes a running record of connected decisions, events, dependencies, and outcomes, with each consequential relationship traceable to its evidence.

🧠 Build traceable agent memory with cognee.

Use cognee to connect entities, relationships, history, and provenance in graph-backed memory that agents can retrieve across sessions.

Start building today with your free cognee Cloud key or explore our docs to learn more about how our memory layer works.

FAQ

Answers to the most common questions from this guide.

Does graph memory require a graph database?+

No. A memory graph describes the logical structure of the memory rather than a required storage engine.

Graph databases provide native traversal and relationship queries, while relational or document backends can support graph memory as long as they preserve typed relationships, metadata, provenance, and the retrieval operations the agent needs.

Can I add graph memory to an existing vector-based agent?+

Yes. The existing vector layer can keep handling semantic retrieval while graph structure is added for memories that benefit from explicit relationships.

A practical migration can begin with a small set of high-value entities and relationship types, then expand as new retrieval needs appear. The whole memory system doesn't need to be rebuilt at once.

Can a memory graph store both episodic and semantic memory?+

Yes. Events, interactions, and task outcomes can form episodic memory, while consolidated entities, facts, and relationships can form semantic memory.

The two can stay connected. A persistent fact can keep links back to the episodes or sources it was learned from, so later retrieval gets both the consolidated knowledge and the history behind it.

How do you migrate existing agent memory into a graph?+

Start with recurring queries that benefit from relationship-aware retrieval. Existing memories can stay in their current store while selected records are processed into entities, typed relationships, provenance, and temporal metadata.

From there, validate entity identity, connect existing records to the new graph, and migrate additional memory where relationship-aware retrieval or long-term maintenance improves the workflow.