“Every agent that runs in production will eventually hit the context wall. What separates working systems from failing ones is how you build around it.”
Every AI agent that lives long enough hits the same wall: context loss. You build an agent, it works beautifully for the first few interactions, then somewhere around turn thirty or fifty it forgets what it was doing. Critical context evaporates. Your agent starts repeating questions. It makes decisions based on incomplete information.
I've watched this happen in dozens of production deployments. The problem isn't the model—it's the architecture.
As production agents handle more complex tasks and generate more tool results, they often exhaust their effective context windows—leaving developers stuck choosing between cutting agent transcripts or degrading performance.
The good news: this problem is solvable. You just need to treat memory as a first-class system component, not an afterthought.
Why Context Forgetting Happens
Let me be direct about what's actually happening.
While chat models operate with a relatively static context window—primarily the user's message and system instructions—agents face a far more complex challenge. Agents make tool calls in loops, and each tool's output becomes part of the context that the LLM must process in the next step.
This is the core problem. Every tool call, every API response, every intermediate result gets stacked into context. After dozens of interactions, you're processing hundreds of thousands of tokens just to maintain state. The model starts to struggle. It loses focus.
Without context engineering, you hit context rot: the model struggles to reason over extremely long histories, and performance degrades even when the information technically fits within the window.
It's not a capacity problem—it's an attention problem. The model can technically fit the data, but the quality of reasoning collapses.
The Architecture Pattern That Works
When you build AI agents that actually work in production, you need to separate three things:
- Working context - what the agent needs right now
- Session state - what's been done and what matters
- Persistent memory - what the agent should remember forever
Most teams try to keep everything in working context. That's why they fail.
Context is a compiled view over a richer stateful system. Sessions, memory, and artifacts (files) are the sources—the full, structured state of the interaction and its data. Flows and processors are the compiler pipeline—a sequence of passes that transform that state. The working context is the compiled view you ship to the LLM for this one invocation.
Think of it like a computer's memory hierarchy. Your CPU doesn't access main memory for everything—it uses cache. Your agent shouldn't load everything into context either.
Storage Strategies That Scale
I've implemented persistent memory systems in three main ways, depending on the use case:
File-Based Memory (Simplest)
The memory tool enables Claude to store and consult information outside the context window through a file-based system. Claude can create, read, update, and delete files in a dedicated memory directory stored in your infrastructure that persists across conversations. This allows agents to build up knowledge bases over time, maintain project state across sessions, and reference previous learnings without having to keep everything in context.
This is what I use for most agents. It's simple, debuggable, and works without external infrastructure. You write structured data to disk, the agent reads it back when needed.
The pattern looks like this:
// Agent writes to memory
const memory = {
projectState: {
completedTasks: ["auth setup", "database schema"],
currentPhase: "API implementation",
blockers: ["missing API key for payment service"],
decisions: {
"database choice": "PostgreSQL for ACID guarantees",
"auth strategy": "JWT with refresh tokens"
}
},
timestamp: new Date().toISOString()
};
// Agent reads memory on next session
const previousContext = await readMemoryFile('project-state.json');
// Load only relevant decisions into context
For Claude Code specifically, all memory files are automatically loaded into Claude Code's context when launched. Files higher in the hierarchy take precedence and are loaded first, providing a foundation that more specific memories build upon.
Semantic Search (For Scale)
When you have months of conversation history, file reads aren't enough. You need to search for relevant memories, not load everything.
Simple embedding search breaks down as memory grows. Teams evolved to a multi-technique approach combining semantic search, keyword matching and graph traversal. Each method handles different types of query.
This is where vector databases come in. You embed your memory, store it in something like Supabase with pgvector, and retrieve only what's relevant:
// Embed the query
const queryEmbedding = await generateEmbedding("authentication implementation");
// Search memory
const relevantMemories = await supabase
.rpc('match_memories', {
query_embedding: queryEmbedding,
match_threshold: 0.7,
match_count: 5
});
// Load only top-5 relevant memories into context
const contextMemories = relevantMemories.map(m => m.content).join('\n\n');
The key insight: at scale, the distinction between memory systems and RAG blurs. Both involve selecting relevant information and loading it into context. The difference lies in intent. RAG typically handles knowledge retrieval while memory systems manage agent state and learned patterns.
Summarization (For Long Runs)
Sometimes you need to compress history without losing critical information.
When a configurable threshold (such as the number of invocations) is reached, an asynchronous process is triggered. It uses an LLM to summarize older events over a sliding window—defined by compaction intervals and overlapping size—and writes the resulting summary back into the Session as a new event with a "compaction" action. Crucially, this allows the system to prune or de-prioritize the raw events that were summarized.
This is expensive (you're calling the model to compress), but it's worth it for long-running agents. A 10-hour agent session becomes a 2-3 page summary that still captures all the decisions and context.
What to Store (And What to Discard)
Not everything deserves to be remembered. Store:
- Decisions made - why you chose option A over B
- Learned patterns - what worked, what didn't
- User preferences - how they like things done
- System state - what's been completed, what's blocked
- Discovered constraints - API limits, data quirks
Don't store:
- Raw tool outputs - keep summaries instead
- Failed attempts - unless there's a pattern
- Verbose logs - compress to key events
- Redundant information - one source of truth
The principle is simple: you should be striving for the minimal set of information that fully outlines your expected behavior. (Note that minimal does not necessarily mean short; you still need to give the agent sufficient information up front to ensure it adheres to the desired behavior.) It's best to start by testing a minimal prompt with the best model available to see how it performs on your task, and then add clear instructions and examples to improve performance based on failure modes found during initial testing.
Real-World Implementation
Here's how I structure persistent memory for a typical agent:
interface AgentMemory {
// Session metadata
sessionId: string;
createdAt: string;
lastUpdated: string;
// What the agent learned
learnings: {
patterns: string[];
constraints: Record<string, string>;
successFactors: string[];
};
// What matters for decisions
context: {
userPreferences: Record<string, any>;
projectState: Record<string, any>;
decisions: Record<string, { choice: string; reasoning: string }>;
};
// What to do next
nextSteps: {
blockers: string[];
recommendations: string[];
followUp: string[];
};
}
// Write memory at checkpoints
async function updateMemory(agent: Agent, updates: Partial<AgentMemory>) {
const current = await readMemory(agent.id);
const merged = {
...current,
...updates,
lastUpdated: new Date().toISOString()
};
await writeMemory(agent.id, merged);
}
// Inject memory at session start
async function loadMemoryToContext(agent: Agent): Promise<string> {
const memory = await readMemory(agent.id);
return `
# Agent Memory
## Previous Learnings
${memory.learnings.patterns.map(p => `- ${p}`).join('\n')}
## Key Decisions
${Object.entries(memory.context.decisions)
.map(([key, val]) => `- ${key}: ${val.choice} (because: ${val.reasoning})`)
.join('\n')}
## Current Blockers
${memory.nextSteps.blockers.map(b => `- ${b}`).join('\n')}
`.trim();
}
This structure gives you the best of both worlds: minimal context window usage and maximum information retention.
Measuring Success
You'll know your memory system is working when:
- Agents stop repeating themselves - they remember what they already tried
- Cost stabilizes - token usage doesn't explode with conversation length
- Quality improves over time - agents make better decisions as they accumulate context
- Debugging gets easier - you can inspect what the agent knows
Monitor three metrics:
- Context window utilization - what percentage of your token budget are you using?
- Memory hit rate - how often does the agent find relevant memories?
- Decision quality - are agents making better choices after accessing memory?
The Bigger Picture
Memory management isn't just about solving context limits. It's about building agents that learn. When you structure persistent memory correctly, your agent becomes smarter over time. It remembers what worked last time. It avoids repeated mistakes. It accumulates knowledge.
This is the difference between a stateless tool and a true collaborative partner.
If you're building production AI agents, this is non-negotiable. Start simple with file-based memory. Graduate to semantic search when you need scale. Add summarization when sessions get long. But start treating memory as a first-class system component from day one.
For deeper patterns on building reliable agent systems, see Building Reliable AI Tools and The Architecture of Reliable AI Systems. And if you're working with Claude specifically, Building Production-Ready AI Agents with Claude covers how to build AI agents that actually work in real deployments. You might also find The Claude Code Memory Crisis helpful for understanding how to handle persistent context in long-running development workflows.
The agents that win in production aren't the ones with the biggest context windows. They're the ones with the best memory systems.
Ready to build agents that remember? Get in touch and let's talk about what you're building.