Mem0 — Long-Term Memory for Assistants¶

What problem does this solve?¶

Conversational assistants need to remember things across sessions: user preferences, prior decisions, ongoing projects. Naive approaches — stuffing the whole conversation history into the prompt — break down past a few turns. The model can't pick out what matters; the prompt balloons; tokens get expensive; and the same facts get repeated turn after turn. Mem0 (and similar systems like Letta/MemGPT, Zep) treats memory as a small retrievable knowledge base. After each turn, an extractor pulls durable facts from the exchange. Facts are deduplicated, scored, and stored. Before each turn, relevant facts are retrieved and prepended. The model sees a tight prompt with only what matters, regardless of how long the conversation has been going on.

Where it came from¶

Mem0 was released as a hosted product and open-source library in early 2024. The conceptual roots — extract-store-recall memory for agents — go back to MemGPT (Stanford, 2023) and earlier work on external memory for language models. The practical contribution was packaging it as a clean API: add(turn), search(query), with deduplication and scoring built in. By 2026 long-term memory for assistants is table stakes. Mem0 has competitors (Letta, Zep, Cognee) but the pattern is largely the same across all of them: extract durable facts, embed for retrieval, recall by relevance. The cookbook implementation matches that shape directly so the pattern is visible without any framework magic.

Where it fits in the RAG landscape¶

Memory layers complement the rest of the RAG stack and live alongside retrieval rather than replacing it:

Mem0 (this recipe). Persistent per-user fact store with extract-and-recall.
Letta / MemGPT. Memory + tool-use unified agent runtime.
Zep. Memory store with stronger temporal reasoning and time-windowed recall.
Cognee. Knowledge-graph-based memory; closer to GraphRAG (Recipe 31).
Native conversation history. Cheapest, breaks past a few turns.

Memory composes with retrieval (RAG): the memory layer holds who the user is; the retrieval layer holds what's in the corpus. Production assistants use both.

When to use it (and when not to)¶

Use Mem0-style memory in conversational assistants where users have ongoing relationships with the system. Coding assistants, customer-support bots, personal assistants — anywhere a user comes back and expects the system to know things about them. Skip it for stateless Q&A systems. There's no relationship to remember. Skip it when privacy regimes prohibit persistent user data. GDPR/CCPA require careful design and deletion mechanisms.

The intuition¶

Four intuitions:

Extract is the bottleneck. What you extract determines what you can recall. Bad extraction gives bad recall.

Recall is a retrieval. Standard cosine search over stored facts. The same machinery as RAG.

Deduplication is non-trivial. Two facts that say the same thing in different words should merge. The cookbook implementation is naive; production systems use embedding-similarity deduplication.

Forgetting is a feature. Not every fact is durable. Score by recency and reuse, prune the long tail.

Architecture¶

flowchart LR
  T[Conversation turn] --> EX[LLM:
extract durable facts]
  EX --> D[Dedupe vs
existing memory]
  D --> M[(Memory store)]
  Q[New question] --> RC[Recall:
retrieve relevant facts]
  M --> RC
  RC --> AN[LLM answers
with memory context]

References¶

📚 Mem0 documentation — Hosted product docs.
💻 mem0ai/mem0 repository — Open-source implementation.
📄 MemGPT — Towards LLMs as Operating Systems (Packer et al., 2023) — The earlier paper that framed the problem.
📚 Letta (formerly MemGPT) framework — Memory + agent runtime.
📚 Zep memory store — Alternative with strong temporal reasoning.
📚 LlamaIndex chat memory abstractions — The framework-level abstraction.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Build a memory store¶

Memory is just a Qdrant collection of fact strings. Each fact has metadata (category, recency, score).

In [2]:

Copied!

from cookbook.stores import QdrantBackend

MEM_DIM = len(client.embed(['probe'])[0])
mem = QdrantBackend('mem0', dim=MEM_DIM)
print(f'Memory store ready (dim={MEM_DIM}).')
from cookbook.stores import QdrantBackend

MEM_DIM = len(client.embed(['probe'])[0])
mem = QdrantBackend('mem0', dim=MEM_DIM)
print(f'Memory store ready (dim={MEM_DIM}).')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Memory store ready (dim=4096).

Step 2 — The fact extractor¶

Given a conversation turn, extract durable facts about the user worth remembering. We ask the LLM for JSON.

In [3]:

Copied!





import json
import re

EXTRACT = (
    'Extract durable facts about the user worth remembering from this turn. '
    'Respond as JSON {{"facts": [{{"text": str, "category": str}}]}}.\n\n'
    'Turn: {turn}'
)

def remember(turn: str):
    raw = client.chat(EXTRACT.format(turn=turn))
    m = re.search(r'\{.*\}', raw, flags=re.DOTALL)
    if not m:
        return []
    try:
        facts = json.loads(m.group()).get('facts', [])
    except json.JSONDecodeError:
        return []
    texts = [f['text'] for f in facts if isinstance(f, dict) and f.get('text')]
    if not texts:
        return []
    vecs = client.embed(texts)
    mem.add(
        texts, vecs,
        metadatas=[{'category': f.get('category', 'general')} for f in facts]
    )
    return texts

stored = remember('I prefer concise, one-paragraph answers and I work in Rust most of the time.')
print('Stored:')
for t in stored:
    print(f'  - {t}')
import json
import re

EXTRACT = (
    'Extract durable facts about the user worth remembering from this turn. '
    'Respond as JSON {{"facts": [{{"text": str, "category": str}}]}}.\n\n'
    'Turn: {turn}'
)

def remember(turn: str):
    raw = client.chat(EXTRACT.format(turn=turn))
    m = re.search(r'\{.*\}', raw, flags=re.DOTALL)
    if not m:
        return []
    try:
        facts = json.loads(m.group()).get('facts', [])
    except json.JSONDecodeError:
        return []
    texts = [f['text'] for f in facts if isinstance(f, dict) and f.get('text')]
    if not texts:
        return []
    vecs = client.embed(texts)
    mem.add(
        texts, vecs,
        metadatas=[{'category': f.get('category', 'general')} for f in facts]
    )
    return texts

stored = remember('I prefer concise, one-paragraph answers and I work in Rust most of the time.')
print('Stored:')
for t in stored:
    print(f'  - {t}')

Stored:
  - prefers concise answers
  - works in Rust

Step 3 — Simulate a few conversation turns¶

Store facts across turns. Each turn's extraction adds to the memory.

In [4]:

Copied!





turns = [
    'I prefer terse one-paragraph answers.',
    'I work primarily in Rust and avoid Python unless required.',
    'My current research project is on high-temperature superconductors.',
    'I find verbose explanations annoying — stick to the point.',
]
for t in turns:
    facts = remember(t)
    print(f'Turn: {t[:60]}')
    for f in facts:
        print(f'  +  {f}')
turns = [
    'I prefer terse one-paragraph answers.',
    'I work primarily in Rust and avoid Python unless required.',
    'My current research project is on high-temperature superconductors.',
    'I find verbose explanations annoying — stick to the point.',
]
for t in turns:
    facts = remember(t)
    print(f'Turn: {t[:60]}')
    for f in facts:
        print(f'  +  {f}')

Turn: I prefer terse one-paragraph answers.
  +  prefers terse one-paragraph answers
Turn: I work primarily in Rust and avoid Python unless required.
  +  works primarily in Rust
  +  avoids Python unless required
Turn: My current research project is on high-temperature supercond
  +  high-temperature superconductors
Turn: I find verbose explanations annoying — stick to the point.
  +  prefers concise explanations

Step 4 — Recall relevant memories¶

Standard cosine retrieval over the memory store.

In [5]:

Copied!





def recall(question: str, top_k: int = 5):
    qv = client.embed([question])[0]
    return mem.search(qv, top_k=top_k)

for h in recall('Suggest a weekend reading list for me.'):
    print(f'  {h.score:.3f}  {h.text}')
def recall(question: str, top_k: int = 5):
    qv = client.embed([question])[0]
    return mem.search(qv, top_k=top_k)

for h in recall('Suggest a weekend reading list for me.'):
    print(f'  {h.score:.3f}  {h.text}')

  0.463  prefers concise answers
  0.449  prefers concise explanations
  0.445  prefers terse one-paragraph answers
  0.394  works in Rust
  0.371  works primarily in Rust

Step 5 — Wrap as `answer_question`¶

Standard contract. We recall first, then answer with memory in context.

In [6]:

Copied!





def answer_question(question: str) -> tuple[str, list[str]]:
    facts = [h.text for h in recall(question)]
    answer = client.chat(
        'Known facts about user:\n' + '\n'.join(facts)
        + f'\n\nQ: {question}\nA:'
    )
    return answer, facts

ans, _ = answer_question('What kind of programming book should I read this weekend?')
print(ans)
def answer_question(question: str) -> tuple[str, list[str]]:
    facts = [h.text for h in recall(question)]
    answer = client.chat(
        'Known facts about user:\n' + '\n'.join(facts)
        + f'\n\nQ: {question}\nA:'
    )
    return answer, facts

ans, _ = answer_question('What kind of programming book should I read this weekend?')
print(ans)

"Rust in Action" or "The Rust Programming Language".

Look Inside¶

Inspect — what's in the memory store?¶

Recall everything to see the full state.

In [7]:

Copied!





from cookbook import _cache
print(f'Memory entries: {_cache.stats()["entries"]} (cache; not memory directly)')
print()
print('Recall against "user preferences":')
for h in recall('user preferences', top_k=10):
    print(f'  {h.score:.3f}  {h.text}')
from cookbook import _cache
print(f'Memory entries: {_cache.stats()["entries"]} (cache; not memory directly)')
print()
print('Recall against "user preferences":')
for h in recall('user preferences', top_k=10):
    print(f'  {h.score:.3f}  {h.text}')

Memory entries: 3513 (cache; not memory directly)

Recall against "user preferences":
  0.644  prefers concise answers
  0.628  prefers concise explanations
  0.608  prefers terse one-paragraph answers
  0.470  works in Rust
  0.467  works primarily in Rust
  0.430  avoids Python unless required
  0.426  high-temperature superconductors

Inspect — recall on different questions¶

Different queries should recall different facts. Verify that retrieval is selective.

In [8]:

Copied!





for q in [
    'What programming language should I focus on?',
    'What research topic am I interested in?',
    'How verbose should my replies be?',
]:
    facts = [h.text for h in recall(q, top_k=2)]
    print(f'Q: {q}')
    for f in facts:
        print(f'  +  {f}')
    print()
for q in [
    'What programming language should I focus on?',
    'What research topic am I interested in?',
    'How verbose should my replies be?',
]:
    facts = [h.text for h in recall(q, top_k=2)]
    print(f'Q: {q}')
    for f in facts:
        print(f'  +  {f}')
    print()

Q: What programming language should I focus on?
  +  works primarily in Rust
  +  avoids Python unless required

Q: What research topic am I interested in?
  +  prefers concise answers
  +  prefers terse one-paragraph answers

Q: How verbose should my replies be?
  +  prefers concise answers
  +  prefers terse one-paragraph answers

Inspect — duplicate detection¶

Add a near-duplicate fact and see what happens. Naive Mem0 doesn't dedupe; production Mem0 does.

In [9]:

Copied!





before_count = mem.client.count(mem.collection).count
remember('I dislike verbose answers and prefer concise ones.')
after_count = mem.client.count(mem.collection).count
print(f'Memory size: {before_count} -> {after_count}')
print('Naive Mem0 stores duplicates; production Mem0 detects and merges them.')
before_count = mem.client.count(mem.collection).count
remember('I dislike verbose answers and prefer concise ones.')
after_count = mem.client.count(mem.collection).count
print(f'Memory size: {before_count} -> {after_count}')
print('Naive Mem0 stores duplicates; production Mem0 detects and merges them.')

Memory size: 7 -> 8
Naive Mem0 stores duplicates; production Mem0 detects and merges them.

Inspect — cost¶

Each turn: 1 LLM call (extraction) + N embeddings (one per fact). Each query: 1 embedding + 1 LLM call.

In [10]:

Copied!

print('Per-turn cost: 1 extraction LLM call + N embeddings.')
print('Per-query cost: 1 query embedding + 1 answer LLM call.')
print('Memory stays cheap until you scale to thousands of users.')
print('Per-turn cost: 1 extraction LLM call + N embeddings.')
print('Per-query cost: 1 query embedding + 1 answer LLM call.')
print('Memory stays cheap until you scale to thousands of users.')

Per-turn cost: 1 extraction LLM call + N embeddings.
Per-query cost: 1 query embedding + 1 answer LLM call.
Memory stays cheap until you scale to thousands of users.

Run It¶

End-to-end with memory.

In [11]:

Copied!

ans, _ = answer_question('Recommend a reading list aligned with my interests.')
print('=== Memory-grounded answer ===')
print(ans)
ans, _ = answer_question('Recommend a reading list aligned with my interests.')
print('=== Memory-grounded answer ===')
print(ans)

=== Memory-grounded answer ===
For a Rust developer, I recommend "The Rust Programming Language" by Steve Klabnik and Carol Nichols, "Rust by Example", and "Rustonomicon" for in-depth knowledge of Rust. Additionally, "Clean Code" by Robert C. Martin and "The Pragmatic Programmer" by Andrew Hunt and David Thomas provide valuable insights on software development best practices.

Side by Side: Vanilla Baseline vs This Technique¶

Vanilla (no memory) vs Mem0 (with memory). The vanilla call lacks the user context Mem0 retrieves; the answer should be visibly less personalised.

In [12]:

Copied!





q = 'Recommend a reading list aligned with my interests.'
base_ans = client.chat(q)
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'no-memory', 'preview': base_ans[:160]},
    {'pipeline': 'mem0', 'preview': ours_a[:160]},
])
q = 'Recommend a reading list aligned with my interests.'
base_ans = client.chat(q)
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'no-memory', 'preview': base_ans[:160]},
    {'pipeline': 'mem0', 'preview': ours_a[:160]},
])

Out[12]:

	pipeline	preview
0	no-memory	I'd be happy to recommend a reading list tailo...
1	mem0	For a Rust developer, I recommend "The Rust Pr...

Knobs to Turn¶

Six knobs in priority order:

Extraction prompt. Quality of stored facts dominates everything downstream. Tune carefully and review what gets stored.
Top-k recall. Cookbook uses 5. Higher catches more memories at the cost of prompt length.
Deduplication. Production systems dedupe by embedding similarity. The cookbook doesn't to keep the implementation small.
Decay / forgetting. Score recency and reuse; prune low-score facts periodically to keep the store relevant.
Multi-user isolation. Use Qdrant payload filters to isolate per-user memories in a shared collection.
Categorisation. Stored facts can carry a category (preferences, projects, history). Recall can filter by category.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [13]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[13]:

	question	expected	actual	contexts_retrieved
0	What problem do state-space models aim to solv...	State-space models target the quadratic time a...	State-space models aim to solve the problem of...	5
1	Describe the selective scan mechanism introduc...	Selective scan makes the SSM parameters input-...	Mamba's selective scan: skips unnecessary pack...	5
2	How does Mamba achieve hardware efficiency on ...	Mamba uses a parallel scan implementation with...	Mamba achieves hardware efficiency on modern G...	5
3	Which earlier model family does Mamba descend ...	Mamba builds on the structured state-space seq...	Mamba descends from the Black Mamba and Copper...	5
4	Name two domains beyond text where SSM-style b...	Audio modeling and genomics have both seen suc...	SSM-style backbones have been applied to compu...	5

Closing Thoughts¶

Four failure modes you'll meet in production:

Fact drift. The extractor invents facts from ambiguous turns. Validate before storing — at minimum, check that the fact is grounded in the turn text.
Stale memories. "I'm working on X" recorded today is wrong six months later. Decay; categorise as time-bounded.
Privacy. Persistent per-user storage triggers regulatory questions. Plan for deletion (GDPR right-to-be-forgotten) from day one.
Cross-user contamination. A shared memory store leaks data between users. Use strict per-user isolation.

Compose with retrieval (RAG): memory holds who the user is; RAG holds what's in the corpus. Compose with LangGraph (Recipe 28): a memory node and a retrieval node, both feeding the answer node.