Every production AI agent eventually hits the same wall. The agent was working. It understood the context. Then it was asked to do something that required information from earlier in the conversation, or from a previous session, or from a document it processed three hours ago, and it got it wrong. Not because the model forgot in any cognitive sense. Because the information was no longer in the active context window that the model was processing.
Memory architecture is how agent teams solve this problem. There is no single correct solution. There are four distinct patterns, each with different performance characteristics, cost profiles, failure modes, and tradeoffs. Choosing the wrong pattern for a given agent’s task profile is one of the most common causes of agents that fail in production after succeeding in development. This analysis covers each pattern in technical detail, where each performs well, and where each breaks.
Pattern 1: Full Context Window Memory
The simplest memory architecture is no architecture at all. The agent keeps the entire conversation history, all tool call results, and all document content in the active context window from the start of the session to the end. When the context window fills, the session ends and a new session begins with a fresh empty context.
Full context window memory works well for short, contained tasks. A developer asking Claude Code to explain a function, fix a bug, and then write a test for the fix is running a sequence of steps that fits comfortably in a 200,000-token context window. The model sees everything that happened before each new step and maintains coherent understanding of the full task. No external storage, no retrieval, no indexing. The model’s attention mechanism is the memory system.
The failure mode is context length. Transformer attention mechanisms do not scale linearly with context length. Computational cost scales quadratically with context length in standard attention, and linearly in efficient attention variants like FlashAttention, but even linear scaling means that a 200,000-token context costs ten times what a 20,000-token context costs in compute time and money. More importantly, model accuracy on information from early in the context window degrades as context length grows. The lost-in-the-middle phenomenon means that information injected at turn 2 of a 50-turn conversation may be effectively invisible to the model at turn 50.
Full context window memory is appropriate for single-session tasks with bounded scope. It is inappropriate for agents that run for hours, accumulate large amounts of tool call output, or need to reliably retrieve specific information from early in the conversation.
Pattern 2: Hierarchical Summarization Memory
Hierarchical summarization addresses the context length failure mode by periodically compressing older context into summaries. The agent maintains a rolling window of recent context in full detail, summarizes older context into progressively shorter representations, and keeps only the summaries once the detailed content has been compressed.
Claude Code’s compaction algorithm is an implementation of this pattern. As documented in the arXiv paper analyzing Claude Code’s architecture, the compaction system runs when the active context approaches a threshold, identifies content that is no longer likely to be immediately relevant to the current execution step, compresses that content into a summary, and removes the full content from the active context. The summary is shorter, so context length is managed. The essential information from the compressed content is preserved in the summary.
The critical design decision in hierarchical summarization is the compression policy: which content gets summarized when, and how aggressively. An aggressive policy that compresses early compresses information that might still be needed. A conservative policy that compresses late manages context less effectively and still hits the window limit under heavy use. The compression must be semantic, not mechanical: a character-count-based truncation throws away potentially critical information uniformly. A semantic summarization that identifies and preserves key facts while compressing supporting context is much harder to implement correctly.
The failure mode of hierarchical summarization is lossy compression. Information that seemed unimportant when it was summarized turns out to be critical at a later step. The model cannot retrieve the original detail because it was compressed. The agent either produces wrong output because the detail is gone or requests the information again, triggering redundant tool calls to retrieve context it already had.
Hierarchical summarization is appropriate for long single-session tasks where the task scope is known in advance and the compression policy can be tuned to preserve the information categories that are likely to be needed later. It is less appropriate for open-ended tasks where the information that will be needed later cannot be predicted at compression time.
Pattern 3: External Vector Store Memory
External vector store memory moves information out of the context window and into a persistent vector database, retrieving relevant information through semantic search when it is needed. When the agent processes a document, it chunks the document, generates embeddings for each chunk, and stores them in the vector database. When the agent later needs information that was in the document, it generates a query embedding, searches the vector store for semantically similar chunks, and retrieves the most relevant ones into the active context.
This pattern is the foundation of retrieval-augmented generation (RAG) architectures, and it has been deployed at scale in production AI applications for longer than any other memory pattern. The vector store is persistent across sessions by default: information indexed into the store is available in every subsequent session until explicitly deleted. An agent with vector store memory can retrieve information from documents it processed weeks ago as easily as from documents it processed five minutes ago.
The failure modes of vector store memory are retrieval failures and retrieval staleness. Retrieval failures occur when the information the agent needs is in the vector store but the query embedding does not match the stored chunk embeddings with sufficient similarity to surface it. This happens when the agent’s internal representation of its information need differs semantically from how the information was expressed in the original document. A developer who knows that a function uses a specific algorithm but queries the vector store for the algorithm’s name may not retrieve the chunk that describes the algorithm by a different name used in the documentation.
Retrieval staleness occurs when information in the vector store is outdated. A codebase indexed three weeks ago may contain function signatures that have since changed. An agent that retrieves an outdated signature and uses it to generate a call will generate a call to a function that no longer has that signature. Vector store memory requires explicit invalidation and re-indexing when the underlying information changes, which requires either continuous re-indexing (expensive) or awareness of which information has changed since the last index (complex).
Vector store memory is appropriate for agents that need cross-session access to large bodies of reference material, such as documentation, policies, codebase history, and knowledge bases. It is less appropriate for information that changes frequently or for precise numerical or factual queries where semantic similarity may return plausible but incorrect results.
Pattern 4: Episodic Log Memory
Episodic log memory records the agent’s action history: what the agent did, when it did it, what the result was, and what decision led to the action. The log is structured rather than vectorized, stored in a format optimized for retrieval by action type, timestamp, or outcome rather than by semantic similarity to a query.
AgentCore Memory’s episodic memory tier in Amazon Bedrock is an implementation of this pattern. Every action the agent takes through AgentCore’s Tool Execution layer is automatically logged to episodic memory with the action type, inputs, outputs, timestamp, and the task context that triggered the action. An agent that needs to know what it did during a previous session for audit or recovery purposes retrieves records from the episodic log by query rather than by semantic search.
The episodic log pattern solves problems that none of the other three patterns address. It provides the audit trail that regulated enterprise deployments require. It enables workflow recovery: an agent that fails midway through a 20-step workflow can resume from the last successful step by consulting its episodic log rather than restarting from the beginning. It enables behavioral analysis: teams debugging agent performance issues can examine the episodic log to trace exactly what the agent did and in what order, which is not possible with models that have no external action record.
The failure mode of episodic log memory is log volume and query precision. A busy agent that takes thousands of actions per day generates a log that grows rapidly. Retrieving relevant entries from a large log requires either expensive full-log scans or a well-designed query interface that can retrieve specific action types within specific time ranges or matching specific input patterns. Log design is an underinvested area in most agent implementations because the log seems like a compliance artifact rather than an operational one. The teams that discover its operational value are usually the ones debugging a production incident at 2 a.m. and finding that they cannot trace what the agent did.
Combining Patterns: The Production Memory Architecture
No production agent system uses a single memory pattern in isolation. The patterns are complementary, and the effective agent memory architecture uses different patterns for different data types and access requirements.
The standard production memory architecture combines all four patterns in a hierarchy. Full context window memory handles the immediate working set: the current task, the recent tool call results, and the immediate conversation history. Hierarchical summarization manages context length for longer sessions by compressing older conversation turns. External vector store memory provides persistent access to reference material, documentation, and prior session knowledge. Episodic log memory records every action for audit, recovery, and behavioral analysis.
AgentCore Memory’s four tiers, in-session memory, cross-session memory, semantic memory, and episodic memory, are exactly this production stack implemented as a managed service. Building equivalent infrastructure independently requires choosing a vector database, implementing embedding generation, designing the episodic log schema, building the retrieval interfaces, and managing the consistency between in-session state and the persistent stores. That is several weeks of engineering work that precedes building any actual agent logic. The managed stack trades configuration flexibility for implementation speed.
The Memory Security Dimension
Memory architecture decisions directly affect the security posture of an agent system in ways that are not always apparent during development.
Vector store memory is a persistent attack surface. Information stored in the vector store is available across sessions and cannot be automatically expired. An attacker who can inject malicious content into the vector store, either through a supply chain attack on the document ingestion pipeline or through a prompt injection that causes the agent to write malicious content to the store, creates persistent effects that outlast the current session. The malicious content remains in the store, potentially influencing every future session that retrieves it, until it is explicitly identified and deleted.
The MCP-SafetyBench analysis of context poisoning attacks describes this attack pattern at the protocol level. The vector store is the persistence layer that makes context poisoning effects cross-session rather than single-session. An agent that uses external vector store memory without content integrity controls for what gets written to the store extends the blast radius of a successful context poisoning attack from the current session to all future sessions.
Episodic log memory creates a different risk: the log contains a complete record of the agent’s actions, including the content of tool inputs and outputs. If the episodic log is stored in a system without adequate access controls, it becomes a detailed record of sensitive operations that is accessible to anyone with read access to the log. The compliance value of the episodic log depends on it being both complete (every action logged) and appropriately protected (only authorized parties can read it). These requirements pull in opposite directions for systems that combine broad agent access with restrictive log access policies.
Sizing Memory Architecture to Agent Task Profile
The correct memory architecture for an agent depends on the agent’s task profile: the typical task duration, the volume and variability of information the agent needs to access, the regulatory requirements for the domain, and the agent’s failure recovery requirements.
A coding agent that helps developers with discrete, short tasks in a single IDE session uses full context window memory plus episodic logging for debugging support. There is no need for cross-session vector memory because each task is self-contained, and the compaction overhead of hierarchical summarization is not justified by a typical task that fits in under 10,000 tokens.
A research agent that synthesizes information from large document collections over multi-hour sessions uses hierarchical summarization for active context management plus vector store memory for the document corpus. The episodic log captures which documents were retrieved and what conclusions were drawn, supporting both audit requirements and the ability to resume interrupted research sessions.
A financial services agent that initiates transactions in a regulated environment uses the full four-tier stack: active context for the current workflow, cross-session memory for customer context and preferences, semantic memory for product and policy documentation, and episodic memory for the complete audit trail that financial regulation requires. The KYA Framework’s Behaviour Monitoring pillar assumes the existence of a complete episodic log as its foundational data source. Without it, behavioral analysis and audit compliance are not possible.
The memory architecture decision is not a technical afterthought to be resolved after the agent logic is built. It is a constraint on what the agent can do, how much the agent costs to run, what security properties the agent system has, and whether the agent can meet the audit and compliance requirements of its deployment context. Teams that treat memory architecture as an implementation detail rather than a design decision consistently find themselves rebuilding it after the agent reaches production, which is the most expensive time to make architectural changes. The 31% of agent pilot failures attributable to context collapse are largely memory architecture failures. Addressing them before building the agent logic is the most efficient path to production reliability.