Agentic memory: what it is, what it costs and when to skip it

The short version

• Agentic memory is data an AI agent saves and retrieves across sessions, not a change to the model itself.
• Agentic memory differs from RAG because the agent writes the content itself, and its writes can be wrong.
• Published LoCoMo benchmark scores for agent memory systems are not like-for-like, so they cannot settle which one to choose.
• A wrong or poisoned memory returns in every later session that retrieves it, while a bad answer ends with its session.
• An agent doing one-off tasks, or one that can query a system of record, usually does not need long-term memory.

What is agentic memory?

Agentic memory is the data an AI agent saves and retrieves so that useful information survives beyond a single session. The model does not change. A language model sees only what is in the current prompt, so memory is a separate system that decides what to store and what to put back into that prompt later.

That makes it an engineering problem, not a property of the model. Somebody has to decide what is worth saving, how it is found again, who may read it and when it expires. When a product says an agent 'remembers', those decisions have been made somewhere. Ask where.

What types of memory does an AI agent use?

Most designs separate short-term memory, which holds the current session, from long-term memory, which persists across sessions. Long-term memory is usually split again into facts, past events and procedures, often called semantic, episodic and procedural memory.

The labels matter less than the difference in behaviour. A mistake in short-term memory disappears when the session ends. A mistake in long-term memory keeps coming back. Most of the hard work sits in long-term memory.

What does each type of agent memory hold?

How is agentic memory different from RAG?

RAG reads from a body of content that people curated. Agentic memory is written by the agent itself, during its own work. Retrieval can look identical in both. The write path is the difference, and it is where the new risks come from.

A document in a knowledge base usually has an owner, a source and some form of review. A memory has none of those unless you add them. It is the agent's own interpretation of something it saw, and the agent may have got it wrong. So a memory system needs things a plain retrieval system can do without: a record of where each item came from, a way to correct it and a rule for when it expires.

Many agents need both. Company policy belongs in a curated index. What this customer said last week belongs in memory.

Does agentic memory improve an agent's results?

It can, but the public evidence is narrow. The best example we have is GitHub's own A/B test of Copilot coding agent sessions with and without memories. GitHub reports a gap of 7 percentage points in favour of sessions with memories, about 8.4% in relative terms, at p<0.00001.

Read that result with its limits. GitHub has not published the sample size or the test dates. The test covers software development, where an agent returns to the same repository and the same conventions, so memory has an obvious job to do. It may not carry over to other kinds of work.

We have no measurements of our own to add. The practical conclusion is that memory deserves a test where work repeats, and that the test should be yours, on your tasks.

Can benchmarks tell you which agent memory system to choose?

Not on their own. Benchmarks can help you build a shortlist, but published scores for agent memory systems are often vendor-run and are not like-for-like. LoCoMo, a public benchmark built on conversation data, is the common reference point, and the reports that use it differ in ways that matter.

Mem0's preprint tests its own product against other memory systems, including a full-context baseline, and it is not independent. Letta ran its own LoCoMo test using GPT-4o mini, with the conversation data in a file the agent could search. It did not re-run Mem0 and used Mem0's reported score, and its page gives no scorer, run count or variance. Mem0 has since reported a score for a newer algorithm on its managed platform, which is a different system again.

A university-authored preprint is closer to neutral ground. It benchmarks six memory architectures on LoCoMo using gpt-4o-mini with default or recommended settings. It is not shown as peer reviewed, and default settings may not reflect how a system would be tuned for your workload.

A conversation benchmark also tells you little about your own data. If your agent works through support tickets or contracts rather than long chats, build a small test set from your own records. Ask each vendor for a full-context baseline as well, because that shows what the memory layer adds over sending everything.

What does agentic memory cost to run?

The cost is usually tokens and upkeep rather than storage. If a model extracts what to remember, every write costs tokens. Every item retrieved goes into the prompt, so every read adds tokens to that call and to any later call that carries it forward. Someone also has to correct and clear out what the agent stored.

The design you choose can change that bill a long way. The university preprint reports costs in tokens and hours rather than dollars, and one comparison in its table is 7,044k tokens against 1,308k, about 5.4 times. Those numbers come from a benchmark on default or recommended settings, so treat them as a sign of how far systems can differ, not a forecast. The paper did not measure maintenance cost, which is the part that keeps running after launch.

Some managed services price memory as its own line. AWS's AgentCore pricing page includes a worked example for the memory component, using rates effective 6 October 2026. It is an illustrative calculation, not measured customer spend. It leaves out the rest of the cost of running an agent, and it assumes each short-term event is retrieved once. If your agent reads memory more often than that, redo the sum with your own numbers.

How does agentic memory go wrong?

It goes wrong by making a mistake last. A bad answer in a single session ends with that session. A bad memory comes back in every later session that retrieves it, and by then nobody knows where it came from.

The common failures are ordinary: a fact that was true when saved and is now stale, one event treated as a general rule, two memories that contradict each other with no way to choose. None of these raises an error. The agent starts behaving slightly wrongly, and the cause sits in a store most teams never read.

Memory can also be attacked. Microsoft security researchers describe what they call AI Recommendation Poisoning, the manipulation of AI memory for profit, based on a 60-day review of AI-related URLs in email traffic, along with public web patterns and Defender signals. Their figures are a count of observed cases, not a prevalence rate, and they concern AI assistants rather than custom-built agents. The mechanism carries over to any agent that writes to memory from content it reads, because an instruction in that content can outlive the session.

Shared memory adds a privacy failure. If one store serves many users without strict scoping, a fact learned from one can surface for another. And 'the agent just remembers' is a poor answer when someone asks what you hold about them and asks you to delete it.

When is agentic memory not worth building?

It is not worth building when the agent does one-off tasks, or when the information already lives in a system you can query. Reading a customer's current address from the CRM beats remembering it, because the CRM gets corrected and the agent's copy does not.

A copy in memory is a second source of truth, and it will drift. Before adding a memory layer, check whether a read-only tool call would do the job.

When you do need something, the simplest version is often a good start. Letta's benchmark post asks whether a filesystem is all you need. One vendor's run cannot settle that, but it is the right question to ask first. A searchable file or a plain table is easy to read, correct and delete.

What are the signs an agent does not need memory yet?

• •Each task is self-contained and nothing from the last one helps the next.
• •The facts already live in a system of record the agent can query.
• •You cannot yet say what is stored, for whom, or how it would be deleted.
• •You have no way to measure whether results improve with memory switched on.
• •A short instruction file kept by a person would cover what the agent needs to know.

How should you add agentic memory to an agent?

Add it narrowly and measure it. Start with one kind of memory for one repeated task, record where each item came from, and compare results with memory on and off before widening it.

What are the steps for adding memory to an agent?

1. Write down what the agent may remember and what it must never store.
2. Scope every item to a user, an account or a project, and enforce that scope at retrieval.
3. Store the source and date with each memory, so a wrong item can be traced and removed.
4. Give memories an expiry rule, and give a newer fact a way to replace an older one.
5. Make the store readable and editable by a person, and build deletion before launch.
6. Run the same tasks with memory on and off, and compare both the outcome and the token cost.