Companion code: 09-memory

TL;DR

• Agents can improve across tasks without changing model weights, by writing lessons into a memory that later tasks read. Since 2023 the research has converged on what to write: short, causal, transferable lessons, distilled from failures as well as successes.
• It’s useful to separate facts (what’s true about the user or world) from lessons (how to work). Most “memory” products focus on the first; this post is mostly about the second.
• Memory is a loop: write, manage, read. The manage step (deduplicate, update, expire, check) is what keeps memory from going stale, bloated or poisoned.

The agent that never learns

Here’s a failure you can reproduce in a few minutes. The CRM from earlier posts gets one realistic quirk: get_orders returns amounts in cents, in a field called amount that doesn’t say so. That’s how many payment APIs work. Ask an agent “how much has Riverside Legal spent?”, and it may confidently report $197,200 instead of $1,972.

Fixing the prompt or the tool description is the right fix once you know. The deeper problem is that an agent without memory makes this mistake on every money question, forever, until a person notices and intervenes. Every task starts from zero. Post #3 was about managing memory within one task. This post is about memory across tasks: how an agent can get better with experience.

Two kinds of memory

Facts answer “what do I know?” The 2023 Generative Agents paper (Park et al.) gave its simulated townspeople a memory stream: a record of everything they experienced in natural language, periodically summarised into higher-level reflections and retrieved by relevance and recency. MemGPT framed fact memory as an operating system paging between fast and slow storage. Mem0 (2025) is a production-focused version: extract salient facts from conversations, consolidate them with what’s already known, and retrieve them later.

Lessons answer “what have I learned about doing this?” That’s where the cents bug lives, and it’s what the rest of this post is about.

From reflection to transferable lessons

2023: reflect, then retry. Reflexion had an agent write a short verbal self-critique after a failed attempt and try again with that critique in context. It worked, but the critique was about that attempt at that task.

2023: causal abstractions. CLIN (Majumder et al.) asked what kind of memory transfers. Its answer: not general “helpful hints”, but causal abstractions, statements of the form “doing X is necessary for Y” or “X does not contribute to Y”, updated after every trial. In the ScienceWorld environment, CLIN improved steadily over repeated trials, beat Reflexion by 23 absolute points, and carried what it had learned into new environments and new tasks, all without changing any model weights.

2025: learn from failures too, and distil strategies. ReasoningBank (Ouyang et al.) targeted agents in long-running roles that face a continuous stream of tasks. It distils generalizable reasoning strategies from both the agent’s successes and its failures, as judged by the agent itself, retrieves the relevant ones for each new task, and folds new lessons back in. On web-browsing and software-engineering benchmarks, it beat memory designs that store raw trajectories or only successful routines. The contrast is the point: a log of what happened is not a lesson.

2025–26: playbooks and benchmarks. The ACE paper (covered in post #3) treats an agent’s context as an evolving playbook, updated incrementally rather than rewritten, to avoid wearing knowledge down. Evo-Memory (late 2025) turned the question into a benchmark: datasets arranged as streams of tasks, so you can measure whether an agent’s memory actually helps on task 50 because of tasks 1–49. And the 2026 survey Memory for Autonomous LLM Agents (Du et al.) organizes the whole field around one loop: write, manage, read.

The loop

The companion code (09-memory) implements this loop in about 120 lines. Each step is small.

Read: lessons go into the system prompt at the start of each task. With a handful of lessons, “retrieve everything” is fine. With hundreds, you’d retrieve by relevance, as the Generative Agents and ReasoningBank designs do.

if lessons:  # read: lessons go into the (stable) system prompt for this task
    system += "\n\nLessons from earlier tasks:\n" + "\n".join(f"- {l}" for l in lessons)

Feedback: after each task, the agent learns whether it was right. In the demo, a grader supplies it (“Incorrect: the expected answer was 1972”). In production it might be a test result, a user’s correction, or the agent’s own judgment, which is what ReasoningBank relies on.

Reflect and write: the important design decision. The reflection prompt asks for the CLIN and ReasoningBank kind of lesson, and explicitly rules out the useless kind:

"Write at most 2 lessons that would help on FUTURE, DIFFERENT tasks with these tools. Phrase each as a "
"cause and an action, e.g. 'Tool X returns Y, so do Z'. Never record this task's answer. If nothing "
"generalizable was learned, return an empty list."

“Never record the answer” matters more than it looks. A memory full of answers is a cache, and it overfits. A memory full of causes transfers: “get_orders returns amounts in cents, so divide by 100” fixes every future money question, not just the one that failed.

Manage: the demo does the minimum: skip exact duplicates and keep the most recent twenty lessons. Production memory needs more.

Managing memory: the part that’s easy to skip

Writing lessons is the exciting part. Managing them is what keeps a learning agent useful.

• Deduplicate and merge. The same lesson phrased three ways wastes context and adds no weight. Merge near-duplicates periodically, or have a model consolidate them.
• Update and expire. Tools change. When the payments API starts returning dollars, the cents lesson becomes actively harmful. Attach provenance (which task, which tool version, when), and drop or re-check lessons that keep coinciding with failures.
• Guard against poisoning. If an agent reads untrusted content, such as web pages, emails or tickets, and writes “lessons” from it, an attacker can plant instructions that persist across every future task. Lessons should come from verified feedback, be reviewable by people, and never be written directly from untrusted text. Try it in the demo: add one wrong lesson to memory.json and watch the agent faithfully apply it.
• Keep it small. Memory read into the prompt is context (post #3). A few dozen sharp lessons beat a few thousand vague ones.

Try it

cd examples/09-memory
python learn.py --memory off
python learn.py --memory on --reset

With memory on, you’ll see the agent fail, write a lesson, and apply it on the next money question. Results vary: a strong model sometimes spots that 4900 looks like cents and needs no help. Run both modes a few times. The measure of memory isn’t whether the agent ever makes the mistake; it’s whether it makes it again.

What we measured

We ran the six-task stream against Claude Opus 5.5 with memory off and on. Each answer is graded only on a committed final line, so a hedged answer (“197,200, or maybe $1,972”) counts as wrong. More on why below.

Mode	Correct	What happened
memory off	4/6	Reported cents as dollars on task 1 and again on task 4; caught the unit on two other money questions
memory on	5/6	Reported cents as dollars on task 1, wrote a lesson, and got every later question right

Three observations from reading the transcripts, all of which echo the sections above:

• The failure is inconsistent without memory. The model sometimes noticed that 4900 looks like cents, sometimes didn’t. That’s exactly why memory matters: you can’t rely on the model happening to notice this time.
• The lessons were good, and redundant. After six tasks the store held twelve lessons, and several said nearly the same thing (“amounts are integer cents, divide by 100” in five different phrasings). Nothing deduplicated them beyond exact matches. Run this for a few hundred tasks and the memory would be mostly repetition: the manage step isn’t optional.
• Hedging fools lenient graders. Our first grader passed any answer that contained the right number, and the model’s hedges (“197,200 — or $1,972 if these are cents”) sailed through. Requiring a single committed answer fixed it. If your feedback signal can be fooled, your memory will learn the wrong lessons. Post #12 has more on graders.

Trade-offs

• Feedback is the bottleneck. Lessons are only as good as the signal behind them. With ground truth, as in the demo or a test suite, learning is reliable. With self-judgment, an agent can learn the wrong lesson confidently.
• Memory makes behaviour history-dependent. The same task can behave differently today than last week because of what the agent has learned in between. That’s the point, and it’s also a debugging and evaluation challenge. Version your memory store like code.
• Sometimes the right fix is upstream. If every run learns “amounts are in cents”, fix the tool description (post #4). Memory is for the lessons you couldn’t anticipate, not a substitute for good tools.

Next in the series: The lethal trifecta and six patterns to defuse it.

Sources & further reading

• Park et al., Generative Agents: Interactive Simulacra of Human Behavior (2023).
• Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning (2023).
• Majumder et al., CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization (2023).
• Packer et al., MemGPT: Towards LLMs as Operating Systems (2023).
• Chhikara et al., Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory (2025).
• Ouyang et al., ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory (2025).
• Wei et al., Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory (2025).
• Du et al., Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers (2026).

#memory #learning #context-engineering