Agent Memory in Production
Production agent memory is usually backed by a database or vector store, with explicit rules for what gets kept, summarized, or dropped.
Prerequisites
Overview
Memory that persists across sessions needs somewhere to live — usually a database for structured facts and a vector store for semantic recall — plus a policy for what’s worth remembering at all, since keeping everything isn’t free or even useful.
Where It Fits
Agent Interaction
What’s Worth Keeping?
Structured Facts (DB)
Semantic Recall (Vector Store)
Key Points
- Structured vs. semantic memory
- Discrete facts (a user’s preference) suit a database; fuzzy recall (“what did we discuss last time”) suits a vector store.
- Retention policy
- Not every interaction is worth remembering — a policy decides what gets summarized, stored, or discarded.
- Staleness
- Stored memory can become outdated — production systems need a way to update or expire it, not just append forever.
Interview Question
How would you decide what an agent should actually remember across sessions?
Not everything is worth persisting — I’d distinguish discrete, structured facts worth storing explicitly (like a stated preference) from general conversational context, which is better handled as summarized or vector-searchable recall. Either way, there needs to be a policy for staleness, since stored memory that’s never updated or expired eventually becomes wrong.
Explain It in 30 Seconds
Production agent memory is usually backed by a database for structured facts and a vector store for semantic recall, with an explicit retention policy for what’s worth keeping and a plan for staleness, rather than storing every interaction forever.
Real-World Stack
Technologies commonly used to implement this in production.