Agent Memory Components (Extractor / Store / Retriever)

Any agent "memory" system, whatever its cognitive-science branding, is three components in a line: an extractor that turns transcripts into statements, a store that holds them, and a retriever that pulls the relevant ones back into context. The value of the decomposition is that each layer has exactly one decision that's actually hard, and the marketing usually talks about a different layer than the one that's failing. This page pulls that structure out of agent-memory-anatomy so other pages can point at it directly.

Extractor — the timing decision

The extractor is an LLM reading a raw transcript and emitting abstracted statements: the durable claims worth keeping, stripped of conversational noise. The hard decision is when it runs.

Eager extraction fires per message. It stays current and never has a huge backlog to summarize, but it pays LLM cost on every turn and commits before a thread resolves — it may store a hypothesis the next message overturns. Lazy extraction runs at end of session. It's cheaper, and it sees the whole arc so it can extract conclusions instead of guesses. Its exposure is length: a long transcript triggers lost-in-the-middle degradation, where the model attends to the start and end and loses the middle. Neither is free, and the choice is a real tradeoff, not a solved question.

Store — the contradiction decision

The store is a vector index, a relational table, a knowledge graph, or a combination. Picking the substrate is the easy part; every option can hold statements. The hard part is what happens when a new statement contradicts one already there.

There are three moves. Overwrite replaces the old fact — clean, but it destroys the history that lets the system answer "what did I believe last month?" Append keeps both — lossless, but retrieval now returns stale and current facts side by side unless something marks which is which. Supersede is the middle path: keep the old statement, flag it stale, point it at the replacement. Supersede is the only one that preserves an audit trail without poisoning retrieval, and it's also the most work to implement. This is the memory-conflict-detection problem, and it's the layer most libraries under-invest in.

Retriever — quality as the store grows

The retriever is RAG over the accumulated statements: vector similarity, keyword match, a reranker on top, plus time filters and presupposition checks (does the query silently assume a fact the store now contradicts?). The thing to understand is that retrieval — not storage — is what degrades as the store grows. Ten statements are trivial to search; a hundred thousand are not. The relevant ones are still in there; finding them among the near-duplicates and superseded versions is the problem. This is why "just keep everything, disk is cheap" is correct about storage and incomplete about the system: keeping everything is fine only if the retriever and the store's supersede logic can still surface the current answer. See ai-cannot-forget-forgive for why the answer is better retrieval, not deletion, and agent-memory-decay for the decay-signal approach that folds freshness into the ranking.

The thesis: what you actually built is autobiographical semantic memory

The three components together produce one kind of memory, regardless of the taxonomy on the box. The extractor compresses episodes into facts, dropping the "when and where" that would make them episodic. Procedural memory — evolving how the agent acts — needs a mechanism the store-and-retrieve pipeline doesn't have; a memory_type="procedural" tag on a vector row is a label, not procedural memory. Prospective memory (do Y when X next appears) isn't in the pipeline at all. Subtract episodic, procedural, and prospective and what's left is semantic memory about the agent's own past. That's autobiographical semantic memory, and calling it that is more honest than the four-part cognitive taxonomy the libraries advertise. agent-memory-strategy-decision-tree is the practical companion: given a category of information, which of these layers should own it.

One store-layer answer worth naming separately: several 2026 products keep memory as plain markdown pages rather than vector rows, so the contents can be read, hand-corrected and diffed. llm-wiki-as-agent-memory covers the tradeoff — auditability and portability in exchange for a recurring gardening pass, since a markdown store contradicts itself visibly where a vector store just retrieves worse.

Related

• agent-memory-anatomy — the source essay this concept is extracted from
• llm-wiki-as-agent-memory — the markdown-wiki answer to the store question
• agent-memory-decay — freshness and consolidation as retrieval signals
• agent-memory-strategy-decision-tree — deciding which layer each fact belongs in
• memory-conflict-detection — the store's contradiction problem in depth
• ai-cannot-forget-forgive — why forgetting is the wrong lever