Full context memory costs approximately 26,000 tokens per query at production scale. That number, drawn from the Mem0 benchmark published at ECAI 2025 and a cost-performance analysis posted to arXiv in March 2026, defines the architectural problem every developer building persistent agents must resolve. Passing the entire conversation history sets the accuracy ceiling. It also sets a cost floor that makes the approach non-viable past short sessions. Every other memory architecture in production today is a structured tradeoff against that 26,000-token number.
The decision is no longer whether to build memory. It is which architecture, and what you are trading when you choose. The 2026 benchmarks make the tradeoffs measurable for the first time.
The Three Architectures the Benchmarks Compare
The Mem0 research team, led by Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav, published at ECAI 2025 (arXiv:2504.19413) an evaluation of ten distinct memory approaches across three dimensions: LLM score (binary correctness judged by a model), token consumption per query, and latency in seconds. The three-axis evaluation is the methodological contribution. A system that scores well on accuracy but consumes 26,000 tokens per query is not production-viable. A system with low latency but poor recall is not useful. Optimizing one axis at the cost of the others produces something that benchmarks well and deploys badly.
Three patterns dominate the decision space.
Full context. Passes the entire interaction history to the model on every query. Highest accuracy ceiling because the model has access to everything. Token cost: approximately 26,000 tokens per query at the measured scale. Latency: highest. The arXiv 2603.04814 analysis notes that effective context utilization is typically shorter than the nominal window size, meaning even 200K or 1M token windows do not fully close the accuracy gap on complex multi-hop queries. Makes sense for short sessions where accuracy is paramount and context is bounded. Not viable for agents running across weeks.
Flat fact extraction (vector-only memory). Distills conversation history into structured facts stored as vector embeddings. At query time, semantic similarity retrieval pulls the most relevant facts into context. The Mem0 benchmark places this approach at 66.9 percent LLM score with p95 latency of 1.44 seconds. The documented limitation: flat extraction loses relationships. A fact like “user’s manager is Sarah” and a fact like “user is planning to transfer teams” are stored independently. A query requiring reasoning across both depends on whether the retrieval step surfaces both simultaneously. Reranker layers, available in Mem0 v1.0.0 with support for Cohere, ZeroEntropy, Hugging Face, Sentence Transformers, and LLM-based rerankers, improve candidate precision but do not fix the structural relationship-loss problem.
Graph memory (vector plus relational retrieval). Structures memory as a knowledge graph where entities and their relationships are explicit nodes and edges. Graph traversal can follow relationship chains that flat vector retrieval misses. Mem0g, the graph-augmented variant, scores 68.4 percent LLM score versus 66.9 percent for vector-only. That 1.5 percentage point improvement is modest on average queries but concentrates on multi-hop questions. Latency p95 is 2.59 seconds versus 1.44 seconds, a 1.8x cost. Kuzu, added as a graph backend in September 2025, runs embedded without a separate server process, substantially lowering the operational cost of graph memory compared to Neo4j-dependent setups.
The Distribution Problem the Averages Hide
The benchmark averages understate what matters. Single-fact lookups (where does the user work, what is the user’s timezone) favor fact-based systems strongly. Multi-hop reasoning over relationship chains (how did the user’s role change after the reorg, what was decided in the meeting about the project the user mentioned last week) favors graph memory or full context. The 1.5 point spread between vector-only and graph memory on average compresses a much larger spread on relationship-heavy queries.
The full-context approach wins on accuracy for bounded sessions, but the 26,000-token cost per query scales linearly with history length. An agent running daily interactions for three months accumulates thousands of turns. Full context for that history is not a cost question. It is an architectural impossibility for most model APIs given rate limits and per-token pricing. The arXiv 2603.04814 analysis found additional input length can impair reasoning in some settings, not just increase cost. Longer context is not uniformly better even when it is affordable.
The Temporal Problem and What TSM, Zep, and A-MEM Actually Do
Static fact storage breaks when user states change. Three systems represent the next generation of memory architectures designed specifically for this failure mode, and each makes a different bet about where the cost of temporal reasoning should live.
Temporal Semantic Memory (TSM), proposed by Su and colleagues in 2026, distinguishes between the time a conversation is recorded and the real-world time events occur. It consolidates temporally continuous facts into durative summaries that capture persistent user states. The design assumes most user facts have continuity (employment status, location, relationship status) and should be summarized as enduring rather than stored as discrete events. Developers working on agents with slow-evolving user state should evaluate TSM-style consolidation before adding graph complexity.
Zep structures agent memory as a temporally-aware knowledge graph that tracks historical relationships between entities and maintains fact validity periods. It enables reasoning about how entity states evolve across sessions. Zep is the right choice when your application needs to answer “when did this fact become true” or “was this fact true six months ago.” It is not the right choice if all you need is current-state retrieval, because the temporal graph overhead does not pay back.
A-MEM takes an agentic approach inspired by the Zettelkasten method. It constructs memory notes with LLM-generated contextual attributes and autonomously establishes semantic links between related memories. New experiences can trigger updates to existing memory representations, creating a system that refines its own structure over time rather than accumulating isolated facts. A-MEM pays a per-memory LLM cost for structure generation that the other systems do not, but produces memory graphs that are interpretable and editable.
None of these are benchmarked against each other in a unified evaluation. The Mem0 ECAI 2025 paper covers ten architectures across the basic design space. TSM, Zep, and A-MEM are newer and lack equivalent comparative data. Developers choosing a memory architecture in April 2026 are choosing between systems with solid benchmarks and systems that claim better properties without benchmark validation.
Metadata Filtering: The Feature That Changes the Math
Metadata filtering, available in Mem0 open-source since v1.0.0, allows structured attributes to be stored alongside memories and filtered at query time. Before this, the only retrieval mechanism was semantic similarity. Metadata filtering opens scoped queries: retrieve only memories tagged with a specific project, from a specific time range, or where confidence exceeds a threshold.
This matters for multi-user deployments. A customer service agent handling hundreds of users cannot rely on semantic search alone to retrieve the right user’s memory. Metadata filtering makes user-scoped retrieval deterministic. It also enables time-bounded queries that pure semantic retrieval cannot express efficiently.
The combination of metadata filtering and reranking addresses two distinct failure modes: metadata filtering narrows the candidate set before retrieval, and reranking corrects wrong ordering within the retrieved set. Using both, in that order, produces meaningfully better precision than either alone. The state management architecture of production agents makes this directly consequential: the wrong facts in context produce wrong behavior, not just wrong answers.
Memory and Compaction Are the Same Problem at Different Timescales
The agent memory architecture question and the context compaction question are the same question at different timescales. Context compaction, documented in five layers inside Claude Code, manages memory within a single session. Persistent memory systems manage what carries across sessions. Both face the same core constraint: context windows are finite, retrieval is lossy, and information loss compounds.
The practical failure modes developers encounter in long Claude Code sessions map directly to the memory architecture problem. Instructions placed early in a session get compacted away. Rules that need to survive multiple sessions must move to CLAUDE.md, the system prompt layer that compaction cannot touch. This is the same design decision as choosing where in the memory hierarchy to store a given fact: in working context, in session-scoped vector memory, or in durable structured storage. The hierarchy is: ephemeral working context, session-scoped retrievable memory, and permanent system-prompt-level rules. Every fact gets placed somewhere on that hierarchy, and the placement determines both retrieval cost and survival probability.
Limitations of the Current Benchmark Data
The Mem0 ECAI 2025 benchmark uses a GPT-5-mini judge on a majority-vote protocol. Model-as-judge evaluations introduce systematic biases toward responses from models in the same family. The evaluation covers factual accuracy on persistent user-specific information across multi-session dialogues. It does not cover tasks requiring real-time knowledge, reasoning over structured data, or compositional multi-document evidence chains.
The arXiv 2603.04814 findings characterize the accuracy-cost tradeoff for flat-typed fact extraction specifically. Hierarchical or clustered extraction approaches are not evaluated. The comparison between fact-based memory and long-context LLMs uses prompt caching for the long-context baseline. Prompt caching materially reduces cost for repeated prefix queries, so deployments without caching would show a larger cost differential than the paper reports.
The 10-approach comparison represents point-in-time performance on specific benchmark tasks. As agent interaction patterns diverge from the benchmark’s multi-session dialogue focus, toward code agents, research agents, or domain-specific workflows, the relative performance rankings may shift.
The Practical Decision Framework
For developers choosing a memory architecture in April 2026, the benchmark data supports a concrete decision path.
Start with vector-only memory plus metadata filtering. This handles the majority of personalization use cases at 66.9 percent accuracy, sub-1.5 second p95 latency, and token costs an order of magnitude below full context. If your application is simple preference retrieval, user history, or FAQ-style interactions, this is where you stop.
Escalate to graph memory if your benchmark queries show measurable performance degradation on multi-hop reasoning. The 1.8x latency cost is real but the relationship-reasoning benefit is larger on queries that need it. The Kuzu embedded backend removes the operational overhead that previously made graph memory impractical for smaller deployments.
Add temporal structure (TSM-style consolidation or Zep’s validity periods) if your application reasons about how user states change over time. Skip it if you only need current state.
Reserve full context for bounded, high-stakes sessions where accuracy is the only constraint and history length is manageable. A legal review agent reading a specific document set. A debugging agent with a specific incident window. Not an ongoing assistant running for months.
The 26,000-token figure is not a ceiling to optimize against. It is a reference point for understanding what accuracy costs when you refuse to make tradeoffs. Every other architecture is a structured tradeoff against that number, and the 2026 benchmarks finally make those tradeoffs legible.
The Mem0 paper is available at arxiv.org/abs/2504.19413. The cost-performance analysis is at arxiv.org/abs/2603.04814.