Which AI Memory Layer Has the Lowest Retrieval Latency at Scale in 2026?

Asking which AI memory layer has the lowest retrieval latency at scale sounds like a clean comparison, until the published figures are placed next to each other. Zep, Mem0, Pinecone, Redis, and Cognee all publish latency, but they measure different spans, with different reader models, judges, and retrieval depths behind the number.

This guide compares retrieval latency across the memory layers most often evaluated for agent workloads in 2026: Cognee, Mem0, Zep and Graphiti, Pinecone, and Redis. It defines what retrieval latency actually includes, lists what to measure (p50, p95, p99, cold versus warm), explains how graph traversal depth and vector index choice move those numbers, and provides a reproducible benchmarking method so the comparison can be rerun on your own corpus. Cognee is the memory layer under evaluation throughout, with published figures cited where docs.cognee.ai or the launch materials provide them.

Independent coverage of agent memory frameworks in 2026 warns that latency columns across vendors are not directly comparable, because each vendor measures a different span and uses different reader models, judges, and retrieval depths. A third-party head-to-head makes the point concretely: Zep publishes retrieval-only numbers, while Mem0 reports an end-to-end figure that stays at or under 1.1 seconds across every benchmark it ran, including BEAM at 10M tokens. One column is a slice of the pipeline, the other is the whole pipeline. Dropping both into the same table without that context will mislead.

Two architectural decisions shape Cognee's latency profile and are worth stating up front. Cognee 1.0 ships the full memory layer, graph, vectors, sessions, and metadata, on a single self-hosted Postgres instance, with no separate graph database to operate, which removes a network hop between graph and vector queries. The retrieval engine is written in Rust, which is the lever behind the 0.24s recall figure discussed later in the guide. The sections that follow define retrieval latency precisely, lay out the published numbers that can be cited honestly, and give a benchmarking recipe for the numbers that cannot.

What Retrieval Latency Means in an AI Memory Layer

Retrieval latency inside an AI memory layer is the wall-clock time from the moment a query enters the system to the moment assembled context is handed back to the model. The headline number depends entirely on which spans are included, which is why two platforms can both claim fast retrieval while measuring very different things.

Memory layer vs vector database

A bare vector database answers one question: given an embedding, return the nearest neighbors. A memory layer answers a harder one: given a natural-language query and a session, return the context the model should reason over. That means graph edges, session state, and metadata filters all participate in the lookup, not just approximate nearest-neighbor (ANN) search.

Cognee treats the memory layer as a single substrate rather than a stack of services. Cognee 1.0 ships graph, vectors, sessions, and metadata on a single self-hosted Postgres instance, with no separate graph database to operate. Retrieval latency in Cognee is therefore the cost of one coordinated query against Postgres, not a fan-out across a vector store, a graph engine, and a metadata service.

Spans that get measured: embed, search, graph traversal, rerank, context build

A realistic retrieval path through a memory layer touches at least five spans:

1. Embed, convert the query string into a vector.
2. ANN search, retrieve candidate nodes from the vector index.
3. Graph traversal, expand candidates along typed edges to pull related entities and provenance.
4. Rerank, score candidates against the query, often with a cross-encoder or LLM.
5. Context build, serialize the final payload the model will consume.

A number that covers only step 2 will look dramatically faster than one that covers all five, even on identical hardware and data. Any honest latency comparison names the spans it includes.

Cognee makes those spans directly observable. Memory operations are exposed through OpenTelemetry: memory.operation.duration is recorded as a histogram in milliseconds for add(), cognify(), search(), and forget(), and memory.query.type attributes let operators break latency down per search type. The four-verb API, remember, recall, improve, forget, composes the lower-level add → cognify → memify → search pipeline and is reachable from the CLI, language bindings, and HTTP server, so the same spans show up whether retrieval is driven from a notebook, a service, or a shell. That makes p50, p95, and p99 measurable per verb rather than inferred from a single aggregate number.

Why Retrieval Latency at Scale Matters in 2026

Retrieval latency has moved out of the infrastructure column and into the product column. When a memory layer sits inside an agent loop or a voice interface, every millisecond spent fetching context is a millisecond the user waits, and the cost compounds across steps, sessions, and corpus growth. Two forces make this acute in 2026: agent architectures that call retrieval many times per task, and realtime interfaces where perceived delay breaks the interaction. A third pressure is corpus size, benchmarks now span from 100K tokens to 10M tokens, and retrieval quality must hold across that range without blowing up token budgets.

Agent loops and voice workloads

Multi-step agents amplify retrieval cost by step count. A planner that issues eight retrieval calls at 400ms each spends 3.2 seconds inside memory before the model generates a single user-visible token. Voice and realtime UIs are stricter: conversational turn-taking research puts the comfort threshold for a response under roughly 300ms, which is also where published hybrid retrieval systems have landed. Graphiti, for example, reports P95 retrieval latency of 300ms by combining semantic embeddings, BM25 keyword search, and direct graph traversal while avoiding any LLM calls on the hot path.

Cost of over-retrieval

Latency gets worse when retrieval quality forces larger context windows to compensate. Pulling more candidates to hit an accuracy target raises token spend on every call and slows downstream generation, which then slows the next retrieval in the loop. The Cognee 1.0 BEAM results argue that this spiral is avoidable: 79% at 100K context and 67% at 10M context, ahead of previous best results of 73.4% and 64.1%, with token usage staying flat as the memory grows. Flat token usage at larger corpus sizes is what keeps per-call latency predictable when the underlying graph scales from gigabytes to terabytes.

Common Challenges Measuring Retrieval Latency at Scale

Vendor latency tables look authoritative until the measurement spans are compared side by side. Independent 2026 coverage warns that latency columns across memory vendors are not directly comparable because they measure different spans and use different reader models, judges, and retrieval depths. Before trusting a published figure, the span, hardware, dataset, and load profile behind it must be reconstructed. Cognee's evaluation framework at cognee/eval_framework/ runs benchmarks on standard multi-hop QA datasets and renders search-strategy comparisons in an interactive HTML dashboard, so each number carries its own reproducible context rather than a vendor summary.

Problems Encountered When Comparing Memory Layers

• Different spans measured. A 2026 head-to-head notes that Zep publishes retrieval-only numbers while Mem0 reports an end-to-end figure that holds at or under 1.1s across every benchmark including BEAM at 10M tokens. Retrieval-only and end-to-end spans cannot be placed in the same column. Cognee's eval framework records the full pipeline so retrieval, graph traversal, and generation can be separated or combined on demand.
• Cold versus warm asymmetry. Serverless vector stores pay a cold-start penalty when compute spins up, which is why Pinecone introduced Dedicated Read Nodes with a memory-plus-local-SSD warm path for billion-vector workloads. A single mean hides which path the production traffic actually takes. Benchmark runs should report cold and warm percentiles separately for every search strategy.
• Graph traversal cost. The graph-enhanced Mem0g variant delivers higher accuracy than base Mem0 but raises total latency to 1.091s p50 and 2.590s p95 on LOCOMO, against 0.708s and 1.440s for base Mem0, per the Mem0 paper. Any comparison that mixes a graph-augmented recall path with a flat vector path will misrepresent both. The eval framework holds traversal depth constant across strategies so the cost of each hop is visible.
• Load degradation. Almost every published figure is a single-turn, single-client measurement. Zep's own account of a scaling incident, where graph search p95 went from 150 ms to 600 ms before vector and BM25 search were moved to a dedicated service, shows how far concurrent traffic can move the tail. Single-turn numbers do not survive concurrent traffic. Load profiles that mirror production session counts should be part of every measurement, and Cognee's OpenTelemetry hooks let per-span latencies be exported to any observability backend for the same scrutiny.

What to Look For in a Low-Latency Memory Layer

When retrieval latency at scale is the primary constraint, vague vendor adjectives like "fast" or "scalable" are not enough. The evaluation should reduce to measurable properties that can be reproduced on your own corpus. Below is a concrete checklist, followed by how Cognee performs against each criterion.

Necessary features

• A flat latency curve across corpus sizes. The useful signal is how p95 behaves as the graph grows by orders of magnitude, not a single number. Zep's platform page, for example, publishes p95 retrieval of 148 ms at 10K, 152 ms at 100K, 156 ms at 1M, 161 ms at 10M, and 168 ms at 100M graph size, a ~20 ms spread across four orders of magnitude. Ask any candidate memory layer for the same curve on the same axes.

• Published p50, p95, and p99, not averages. Means hide the tail behavior that actually degrades agent UX. A vendor publishing only an "average response time" is a signal to run your own benchmark before trusting the number.

• No LLM calls on the hot path. Any synchronous model call during retrieval adds hundreds of milliseconds and injects provider variance into p99. Graphiti, for instance, reports P95 retrieval of 300 ms specifically by combining semantic embeddings, BM25 keyword search, and direct graph traversal and avoiding LLM calls during retrieval. The checklist item: confirm that query-time retrieval does not invoke an LLM.

• Per-operation observability. Aggregate dashboards do not help when p99 regresses on a specific query type. Cognee records memory.operation.duration as an OpenTelemetry histogram (in ms) for add(), cognify(), search(), and forget(), and attaches a memory.query.type attribute so latency can be broken down per search strategy. That lets operators attribute a p95 regression to graph traversal depth, vector search, or ingestion back-pressure rather than guessing.

• A reproducible self-benchmark on your data. Published numbers reflect the vendor's corpus, embedding model, and query mix. Cognee ships a self-contained evaluation framework under cognee/eval_framework/ that benchmarks retrieval quality on standard multi-hop QA datasets and compares search strategies in an interactive HTML dashboard, with no third-party evaluation service required. The checklist item: can the candidate platform be benchmarked end-to-end on your corpus before a procurement decision?

• Single-store deployment to cut network hops. Each additional datastore in the retrieval path (separate graph DB, separate vector DB, separate metadata store) adds a network round trip to p95 and another failure domain to p99. Cognee 1.0 runs the entire memory layer, graph, vectors, sessions, and metadata, on a single self-hosted Postgres instance, with no separate graph database to operate.

If a memory layer cannot answer each of these six questions with a number or a repository path, retrieval latency at scale is being asserted rather than demonstrated.

Published Retrieval Latency: Memory Layers Compared

The table below collects published retrieval latency figures across widely cited AI memory layers and vector stores. Numbers come from vendor pages, docs, and peer-reviewed or independent benchmarks. Each row records the exact span measured so readers can judge whether a figure covers pure vector search, hybrid graph retrieval, or an end-to-end agent memory call.

Comparison table (markdown)

How to read the table

Each row measures a different span, so the numbers are not directly comparable. Zep's figures are retrieval-only against its temporal knowledge graph, which is why they stay flat from 10K to 100M nodes. Mem0's headline LOCOMO numbers split search (p50 0.148s, p95 0.200s) from the total agent call (p50 0.708s, p95 1.440s). Mem0g trades latency for accuracy and lands at 2.59s p95 total. Redis 8 reports pure ANN query time with no graph traversal or LLM involvement, at ~200ms median for a billion vectors at 90% precision under 50 concurrent queries.

Cognee's figures split by engine. The cognee.rust engine returns 0.23-0.27s on OpenAI-backed runs and 0.24s on the enterprise reference workload, roughly an order of magnitude faster than the Python SDK on the same inputs. The per-search-type medians on cognee 1.4 with LanceDB and Kuzu (CHUNKS_LEXICAL ~10ms, CYPHER ~1ms, CODING_RULES ~1ms at top_k=15) describe local operations on a small knowledge base, not full end-to-end agent calls. A third-party 2026 head-to-head makes the same caution explicit: Zep publishes retrieval-only numbers, Mem0 reports a figure that holds at or under 1.1s across every benchmark including BEAM at 10M tokens, and vendor latency columns use different reader models, judges, and retrieval depths.

These rows should not be averaged into a single ranking without normalization. For a defensible comparison, pick one span (retrieval-only or end-to-end), one dataset, one hardware profile, and one precision target, then rerun each candidate under those conditions. The benchmarking method later in this guide walks through how to do that on your own corpus.

How Teams Solve Low-Latency Retrieval Using Memory Layers

Low-latency retrieval at scale rarely comes from a single optimization. It comes from stacking patterns across the query path, the storage layout, and the ingestion pipeline. The patterns below are drawn from published engineering writeups and product documentation across Cognee, Graphiti, Zep, and Pinecone, with each pattern tied to the feature or offering that enables it.

Advanced patterns in production

Pattern 1, Rust runtime on the query path. Cognee's Rust core is a Rust port of the Python cognee SDK built for on-device AI memory on phones, smartwatches, and embedded hardware. It uses an embedded graph store and an embedded vector store (LanceDB persistent on desktop, pure-Rust brute-force in-memory on Android), which removes network round trips entirely for local workloads and keeps the retrieval path inside a single process.

Pattern 2, Single-store memory to remove cross-store hops. Cognee 1.0 ships the entire memory layer, graph, vectors, sessions, and metadata, on a single self-hosted Postgres instance, with no separate graph database to operate. Collapsing these onto one engine eliminates the fan-out joins across a vector DB, a graph DB, and a metadata store that often dominate tail latency in multi-service memory stacks.

Pattern 3, Pre-built graphs with no LLM at query time. Graphiti reaches P95 retrieval latency of 300 ms using a hybrid of semantic embeddings, BM25 keyword search, and direct graph traversal, avoiding any LLM calls during retrieval. Shifting entity extraction and graph construction to ingestion keeps query-time cost predictable as the graph grows, because retrieval depends on index lookups rather than model inference.

Pattern 4, Dedicated read capacity for billion-vector workloads. Pinecone's Dedicated Read Nodes (public preview) provide reserved capacity with a warm data path of memory plus local SSD, avoiding cold fetches from object storage and keeping latency low at billion-vector scale. For corpora where serverless cold reads dominate the p99, reserving read capacity is the mechanism that controls tail behavior.

Pattern 5, Per-search-type routing. Cognee exposes distinct search types with different median latencies, so the application chooses the cheapest path for each query. Cognee 1.4 docs report, on a small knowledge base with default top_k=15 using LanceDB and Kuzu: CHUNKS_LEXICAL around 10 ms, CYPHER around 1 ms, and CODING_RULES around 1 ms. Routing short structured questions to CYPHER and keyword lookups to CHUNKS_LEXICAL keeps the median down without forcing every query through the heaviest path.

Pattern 6, Ingestion optimization to shrink the write-side tail. Slow ingestion backs up queues and inflates read latency during bursts. Cognee instruments add() and cognify() with the same memory.operation.duration histogram as search(), so write-side p95 is visible next to read-side p95 and regressions in the ingestion path show up before they surface as stale reads. A tighter ingestion path reduces the window in which readers see partially indexed data and keeps freshness under control as the corpus grows.

Pattern 7, Isolating graph and vector workloads. After a scaling incident, Zep brought graph search back to 150 ms P95 (down from 600 ms) and context retrieval to 200 ms P95 by offloading all vector and BM25 search to a dedicated service and letting the graph database focus on graph operations, reporting 65 ms P75 search latency at massive scale. Separating workloads by engine prevents one query class from starving another when traffic is uneven.

Best Practices for Benchmarking Retrieval Latency Yourself

Published numbers rarely survive contact with production traffic. The six practices below are written for anyone preparing a head-to-head evaluation of AI memory layers and want results that will hold up to review from an engineering lead, a procurement team, or a skeptical reader on Hacker News.

Expert tips

1. Report p50, p95, and p99 separately, never just the mean. Averages hide the tail that actually hurts agent UX. In the Mem0 paper, base Mem0's total latency is 0.708s at p50 but 1.440s at p95, roughly double; the p95 is the figure a user feels when an agent stalls. Publish all three percentiles for every system under test, and label the sample size and time window used to compute them.

2. Separate cold and warm paths in the results table. Serverless and autoscaled vector stores behave very differently when a shard is spun down: the first query after idle pays for compute to spin up and for data to be fetched from object storage, which is the gap Pinecone's Dedicated Read Nodes exist to close. Blending the two produces a number that describes neither path. Record cold-start latency as its own column, and note how long a node must be idle before the next query counts as cold.

3. Hold the reader model, judge, and retrieval depth constant across vendors. Independent coverage of 2026 agent-memory frameworks has warned that latency columns across vendors are not directly comparable, because they measure different spans and use different reader models, judges, and retrieval depths. Pin the embedding model, the top-k, the reranker (or lack of one), the LLM used for answer synthesis, and the LLM-as-judge configuration. Publish the configuration file alongside the numbers so a third party can rerun it.

4. Instrument per-span latency with OpenTelemetry. End-to-end wall-clock numbers conflate network, embedding, graph traversal, vector search, and answer generation. Cognee emits memory.operation.duration as a millisecond histogram for add(), cognify(), search(), and forget(), with a memory.query.type attribute that lets latency be broken down per search strategy. Routing those spans into any OTel-compatible backend (Jaeger, Tempo, Honeycomb) makes it possible to attribute a slow p99 to the component that caused it rather than to the system as a whole.

5. Run under realistic concurrency, not single-turn loops. Serial benchmarks flatter every vendor. Redis's billion-vector benchmark is one of the few that states its concurrency (50 concurrent queries), and Zep's scaling-incident writeup shows graph search p95 quadrupling under load before the workloads were separated. Mirror the concurrency profile of the target workload, agent swarms, parallel tool calls, multi-tenant inference, and record how p95 and p99 move as sessions scale.

6. Use a reproducible evaluation harness, not ad-hoc scripts. One-off notebooks drift between runs and across contributors. Cognee ships a self-contained evaluation framework at cognee/eval_framework/ that benchmarks retrieval quality on standard multi-hop QA datasets and compares search strategies in an interactive HTML dashboard, with no dependency on a third-party evaluation service. Running the same harness against each candidate memory layer, Cognee, Mem0, Zep, a raw vector store, a hand-rolled graph, produces an apples-to-apples report that can be versioned, reviewed, and rerun whenever a dependency is upgraded.

Advantages of a Low-Latency Memory Layer at Scale

When retrieval holds its latency budget under load, several operational benefits follow. The ones below are tied to published measurements from Cognee and a peer memory layer; anything outside those numbers is omitted on purpose.

Measurable benefits

• Sub-second recall from a serverless cold start. Cognee's enterprise page reports recall in 0.24s via the cognee.rust engine, compared with 1.77s on the Python SDK, with a cold start that fits inside a Lambda or Cloudflare Workers invocation. That makes the memory layer usable from short-lived function environments without a warm pool.
• Flat latency as the graph grows. Zep's platform page publishes p95 retrieval of 148 ms at 10K, 152 ms at 100K, 156 ms at 1M, 161 ms at 10M, and 168 ms at 100M graph size, a 20 ms spread across four orders of magnitude. Cognee reports a related form of flatness on the BEAM benchmark: 79% accuracy at 100K context and 67% at 10M context (versus previous best results of 73.4% and 64.1%), with token usage staying flat as data grows.
• Ingestion that keeps pace with query traffic. Low query latency only helps if writes do not back up behind it. Instrumenting add() and cognify() with the same histogram as search() keeps the ingestion path visible, so it does not quietly become the ceiling on end-to-end freshness.
• On-device feasibility. Running cognee-rs on a Samsung Galaxy S24 Ultra, the full pipeline over Alice in Wonderland completed in 139.12s, and once memory was built, chunk and summary search landed in the low-single-second range (~1s). A memory layer that runs this way on a phone also runs predictably on constrained edge hardware.
• Predictable pricing tied to usage. Cognee pricing is $1.00 per 1M tokens processed, plus $5 per additional workspace. Cost scales with what is actually ingested and queried, which lets latency budgets and spend budgets be reasoned about with the same unit.

How Cognee Reduces Retrieval Latency Across the Stack

Cognee reduces retrieval latency through three concrete mechanisms: a Rust port of the Python SDK on the query path, a single-Postgres deployment that collapses graph and vector storage into one engine, and per-search-type routing with OpenTelemetry instrumentation that lets operators verify numbers against their own data.

Rust engine on the query path

The cognee.rust engine is a Rust port of the Python cognee SDK built for on-device memory on phone, smartwatch, and embedded targets, using an embedded graph store and an embedded vector store (LanceDB persistent on desktop, pure-Rust brute-force in-memory on Android). On OpenAI-backed runs published by cognee, the Rust engine returned search in 0.23s on a small document set and 0.27s on War and Peace, compared with 2.66s and 4.03s respectively for the Python SDK. The enterprise page reports recall in 0.24s via cognee.rust versus 1.77s on the Python SDK, with cold start that fits inside a Lambda or Workers invocation. The four-verb API, remember, recall, improve, forget, composes the lower-level add → cognify → memify → search pipeline and is reachable from CLI, language bindings, and an HTTP server.

Single-Postgres deployment

Cognee 1.0 ships the entire memory layer, graph, vectors, sessions, metadata, on a single self-hosted Postgres instance, with no separate graph database to operate. Collapsing storage into one engine removes the cross-system network hop that otherwise dominates traversal latency when a query joins vector hits against graph neighborhoods. The same deployment model is what the published figures are measured against, so numbers reproduced on local Postgres should track the reference profile rather than a multi-service topology.

Per-search-type routing and observability

Different search types carry different latency profiles, and cognee exposes both the routing and the measurement. The docs report median latency per search type on cognee 1.4 with LanceDB and Kuzu: CHUNKS_LEXICAL around 10 ms, CYPHER around 1 ms, and CODING_RULES around 1 ms, measured on a small knowledge base with default top_k=15. Memory operations are instrumented through OpenTelemetry: memory.operation.duration is recorded as a histogram in milliseconds for add(), cognify(), search(), and forget(), with memory.query.type attributes that let operators break latency down per search type in whatever backend already collects traces. For verification on real corpora, cognee ships a self-contained evaluation framework at cognee/eval_framework/ that benchmarks retrieval quality on standard multi-hop QA datasets and compares search strategies in an interactive HTML dashboard, with no third-party evaluation service required.

Cognee pricing is $1.00 per 1M tokens processed, plus $5 per additional workspace.

Final Thoughts and Next Steps

The honest answer to "which memory layer has the lowest retrieval latency at scale" depends entirely on which span of the pipeline is being measured and under what workload.

Which layer wins depends on the span measured

For pure ANN vector lookup, a dedicated vector database is the floor: Redis 8 reports ~200 ms median for a billion vectors at 90% precision under 50 concurrent queries, and smaller indexes on reserved capacity run far below that. For flat retrieval-only p95 across graph sizes, Zep's platform page reports 148 ms at 10K, 152 ms at 100K, 156 ms at 1M, 161 ms at 10M, and 168 ms at 100M, effectively under 200 ms regardless of graph size. For end-to-end search latency on LOCOMO, Mem0's ECAI 2025 paper records the lowest among tested methods at p50 0.148s and p95 0.200s, with total median latency of 0.708s and p95 of 1.440s. Cognee's position is narrower and more specific: when a graph-structured memory layer with entities, relationships, vectors, sessions, and metadata must run from a single store, the Rust-based cognee.rust engine advertises recall at 0.24s versus 1.77s on the Python SDK, with cold start that fits inside a Lambda or Workers call. The cognee 1.0 release consolidates graph, vectors, sessions, and metadata onto a single self-hosted Postgres instance, so there is no separate graph database to operate alongside the vector store.

Try Cognee on your own workload

Published numbers are a starting point. The decision should rest on measurements against representative queries, graph depth, and corpus size. Cognee ships a self-contained evaluation framework at cognee/eval_framework/ that benchmarks retrieval quality on standard multi-hop QA datasets and compares search strategies in an interactive HTML dashboard, with no third-party evaluation service required. Clone the repository, point the framework at a dataset that mirrors production retrieval patterns, and record p50, p95, p99, and cold-versus-warm numbers before committing to a memory layer. Cognee pricing is $1.00 per 1M tokens processed, plus $5 per additional workspace, so a full benchmark against production data is inexpensive to run end to end.

Frequently Asked Questions

Which AI memory layer has the lowest retrieval latency at scale in 2026?+

The answer depends on what is being measured. For pure vector ANN, a dedicated vector database is the floor; Redis 8 publishes ~200 ms median at a billion vectors and 90% precision. For retrieval inside a memory layer, Zep reports p95 between 148 ms at 10K and 168 ms at 100M graph nodes. Mem0 publishes p50 of 0.148s and p95 of 0.200s on LOCOMO search, with total end-to-end median at 0.708s. Cognee's Rust engine returns in 0.23-0.27s on documents up to War and Peace.

How should retrieval latency be measured (p50, p95, p99, cold vs warm)?+

Report p50, p95, and p99 separately, and distinguish cold from warm. Serverless vector stores pay for compute spin-up and object-storage fetches on the first query after idle; reserved-capacity tiers hide this, which is why the two should never be blended into one number. Cognee records the same breakdown through OpenTelemetry: the memory.operation.duration histogram captures ms-level timing for add(), cognify(), search(), and forget(), and the memory.query.type attribute lets operators split latency per search strategy rather than reporting a single blended average.

How does Cognee's Rust core affect cold start and on-device retrieval latency?+

The Rust engine is what makes Cognee viable inside short-lived serverless runtimes and on consumer hardware. The enterprise page reports recall at 0.24s via cognee.rust versus 1.77s on the Python SDK, with a cold start small enough to fit inside a Lambda or Workers invocation. On a Samsung Galaxy S24 Ultra running Alice in Wonderland fully on-device, the complete cognee-rs pipeline finished in 139.12s, and once memory was built, chunk and summary search stayed in the low-single-second range (around 1s). Rust removes most of the Python-side overhead that dominates cold paths.

Why aren't Mem0, Zep, and Cognee latency numbers directly comparable?+

They measure different spans. A third-party 2026 head-to-head notes that Zep publishes retrieval-only numbers while Mem0 reports a figure that holds at or under 1.1s across every benchmark, including BEAM at 10M tokens, meaning Mem0's number includes more of the end-to-end path. Independent coverage also warns that latency columns across memory vendors use different reader models, judges, and retrieval depths. Treat the published figures as useful anchors for the vendor's own workload, then re-run the same query mix against each system before drawing conclusions for a specific application.

Does graph traversal always add latency compared to pure vector search?+

Not automatically, but it often does when the traversal is paired with LLM calls at query time. The graph-enhanced Mem0g variant improves accuracy over base Mem0 while raising total p95 from 1.440s to 2.590s on LOCOMO, per the Mem0 paper. Graphiti, by contrast, reaches P95 retrieval latency of 300ms by combining semantic embeddings, BM25 keyword search, and direct graph traversal with no LLM calls during retrieval. Cognee follows the same principle: graph-aware search strategies are measured through OpenTelemetry so the cost of each traversal depth is visible per request.

Which memory layer can be benchmarked on your own data before committing?+

Cognee ships a self-contained evaluation framework at cognee/eval_framework/ that benchmarks retrieval quality on standard multi-hop QA datasets and compares search strategies through an interactive HTML dashboard, with no third-party evaluation service required. The same framework can be pointed at private corpora, and the OpenTelemetry histograms described above capture p50/p95/p99 for every search() call during the run. The combination means retrieval accuracy and latency can be measured on the exact dataset and query mix a workload will see in production, rather than relying on vendor-chosen benchmarks.