Contextual Retrieval — Anthropic's Highest-ROI Tweak¶

What problem does this solve?¶

A chunk loses everything its document said about what it is. The first paragraph of chapter 17 of the Rust book makes perfect sense in context — "this chapter" refers to async/await — but a chunk that starts with "This chapter introduces the concept of futures and async tasks" embedded in isolation matches every retrieval question that mentions chapters, futures, or tasks. The cosine vector is dominated by superficial cues, not the actual content. Anthropic's September 2024 contextual retrieval pattern fixes this with one prepended line: an LLM-written sentence that says where this chunk fits in the document. Chunks now embed with their context attached. Recall jumps 30–50 percent on standard benchmarks; pairing with BM25 and a reranker pushes it further. The implementation is twenty lines of code.

Where it came from¶

Anthropic published the technique in September 2024 with a detailed engineering blog and a reference implementation. The headline numbers: 35 percent improvement on retrieval failure rate when contextual headers were added to chunks; 49 percent when combined with contextual BM25; 67 percent when combined with reranking. Those numbers held across nine datasets including code repositories, legal text, and scientific papers. The technique itself was not novel — researchers had prepended hand-written headers to chunks for years. Anthropic's contribution was twofold: showing the LLM-generated header was as good as a hand-written one, and making it cheap to run at scale by using prompt caching. The 90 percent cache hit rate on the document context cut the per-token cost by an order of magnitude.

Where it fits in the RAG landscape¶

Three competing approaches to the same problem of "lost context":

Contextual retrieval (this recipe) — prepend an LLM-written header. Cheap, language-agnostic, works on any corpus. Highest ROI of the three.
Late chunking (Recipe 6) — embed the full document first, then pool token embeddings into chunks. The context is baked into the vector instead of into the text. Requires a long-context embedder; saves the LLM-call cost but locks you into compatible embedders.
Parent-child retrieval (Recipe 9) — small chunks for search, big chunks for generation. Solves a related but different problem; pairs well with contextual retrieval.

In production they stack. The Anthropic post explicitly recommends contextual retrieval + BM25 + reranker as the three-step pipeline. We build the first piece here; Recipe 18 adds BM25, Recipe 22 adds the reranker.

When to use it (and when not to)¶

Use contextual retrieval whenever your corpus is structured (chapters, sections, dated reports, threaded conversations) and chunks are likely to lose their position. Technical docs, legal filings, support tickets, codebases — all good fits. Skip it when chunks are already self-contained. Wikipedia summary paragraphs, FAQ entries, hand-curated knowledge-base articles often need no extra context. Adding a header buys you nothing and costs an LLM call per chunk. Skip it also when re-embedding costs are prohibitive. Contextual headers change the embedded text, which means you re-embed every chunk when you switch the header model or prompt. For corpora measured in hundreds of millions of chunks, that is a serious commitment.

The intuition¶

Three intuitions:

The header carries cheap context the chunk cannot. Without the header, a chunk's vector represents only its own 384 tokens. With a header that says "This chunk is from Chapter 17 (async/await), in the section on tokio runtimes, immediately after introducing futures", the vector now encodes position, topic, and continuity. None of those signals were in the chunk's text.

Prompt caching is what makes it tractable. Without caching, you call the LLM once per chunk with the full document context attached — at, say, 10,000 documents averaging 30 chunks, that is 300k LLM calls, each with the full document. Caching reduces it to one call per chunk with cached context, which is roughly the cost of a basic OCR pipeline.

The header does not have to be perfect. A header that says "This chunk discusses thread-safety concerns" is useful even if it slightly mischaracterises the chunk. The embedding is robust to noise; what it needs is more signal, and even a noisy header is signal.

Architecture¶

flowchart LR
  D[Document] --> C[Chunk]
  D --> F[Full document
cached context]
  C --> P[Prompt:
describe where
this chunk fits]
  F --> P
  P --> H[LLM-written
header sentence]
  H --> A[Augmented chunk
= header + chunk]
  A --> E[Embedder]
  E --> S[(Vector store)]
  style F fill:#fff5d0
  style H fill:#e9efff

References¶

📝 Introducing Contextual Retrieval (Anthropic, Sept 2024) — The blog post that defined the pattern.
📚 Anthropic Prompt Caching documentation — The mechanism that makes contextual retrieval cheap at scale.
💻 Contextual retrieval reference cookbook — Anthropic's own implementation; useful for cross-checking.
📝 Late Chunking — Jina AI, 2024 — An alternative answer to the same problem; covered in Recipe 6.
📚 OpenAI Prompt Caching — OpenAI's equivalent; works the same way for our purposes.
📚 LlamaIndex contextual retrieval pack — Reference implementation in the LlamaIndex ecosystem.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Load a small slice of the corpus¶

We use four chapters of the Rust book. The cookbook keeps the slice small (around 40 chunks) so the contextual-header generation completes quickly on a free Nebius account. The technique generalises to corpora of any size — only the bill grows.

In [2]:

Copied!





from cookbook.corpora import load_rust_book
from cookbook.chunkers import fixed_window

docs = list(load_rust_book())[:4]
chunks = fixed_window(docs, target_tokens=320, overlap_tokens=32)[:40]
print(f'Working with {len(chunks)} chunks across {len(docs)} chapters.')
print()
print('First chunk preview:')
print(chunks[0].text[:240])
from cookbook.corpora import load_rust_book
from cookbook.chunkers import fixed_window

docs = list(load_rust_book())[:4]
chunks = fixed_window(docs, target_tokens=320, overlap_tokens=32)[:40]
print(f'Working with {len(chunks)} chunks across {len(docs)} chapters.')
print()
print('First chunk preview:')
print(chunks[0].text[:240])

Working with 24 chunks across 4 chapters.

First chunk preview:
# Getting Started Let’s start your Rust journey! There’s a lot to learn, but every journey starts somewhere. In this chapter, we’ll discuss: - Installing Rust on Linux, macOS, and Windows - Writing a program that prints `Hello, world!` - Us

Step 2 — Define the contextual-header prompt¶

Anthropic's prompt is short and direct. We follow the same shape: a one-line role, the document title and a chunk-locating sentence, then ask for a single sentence of context. The constraint matters — without "one sentence only" the model writes a paragraph.

In [3]:

Copied!





CONTEXT_PROMPT = (
    'You are creating a one-sentence header that will be prepended to a chunk before embedding, '
    'so a vector search engine can rank it correctly. '
    'Describe where this chunk fits in the document and what it covers. '
    'Output exactly one sentence, no preamble.\n\n'
    'Document title: {title}\n\n'
    'Chunk:\n{chunk}\n\nHeader:'
)

sample_header = client.chat(CONTEXT_PROMPT.format(
    title=chunks[0].metadata.get('title', chunks[0].doc_id),
    chunk=chunks[0].text[:1500],
))
print('Generated header:')
print(' ', sample_header.strip())
CONTEXT_PROMPT = (
    'You are creating a one-sentence header that will be prepended to a chunk before embedding, '
    'so a vector search engine can rank it correctly. '
    'Describe where this chunk fits in the document and what it covers. '
    'Output exactly one sentence, no preamble.\n\n'
    'Document title: {title}\n\n'
    'Chunk:\n{chunk}\n\nHeader:'
)

sample_header = client.chat(CONTEXT_PROMPT.format(
    title=chunks[0].metadata.get('title', chunks[0].doc_id),
    chunk=chunks[0].text[:1500],
))
print('Generated header:')
print(' ', sample_header.strip())

Generated header:
  This chunk serves as the introductory chapter of the "Getting Started" document, covering the initial steps of installing Rust, writing a simple program, and utilizing the cargo package manager.

A good header reads like a one-sentence chapter caption. "This chunk is from the opening of the chapter on async and futures, introducing the runtime concept and motivating why Rust separates the executor from the language." Notice how that sentence carries information the chunk's first 30 words cannot.

Step 3 — Generate headers for every chunk¶

One LLM call per chunk. With caching on, repeated runs are free; the first run takes about a second per chunk on Nebius's default model. We store (header, chunk) pairs so the next step can embed them with the header prepended.

In [4]:

Copied!





def contextualise(chunks):
    out = []
    for c in chunks:
        title = c.metadata.get('title', c.doc_id)
        header = client.chat(CONTEXT_PROMPT.format(title=title, chunk=c.text[:1500]))
        out.append((header.strip(), c))
    return out

contextual = contextualise(chunks)
print(f'Generated {len(contextual)} contextual headers.')
print()
print('Three headers + opening lines of their chunks:')
for header, c in contextual[:3]:
    print(f'  HEADER: {header}')
    print(f'  CHUNK : {c.text[:100]!r}...')
    print()
def contextualise(chunks):
    out = []
    for c in chunks:
        title = c.metadata.get('title', c.doc_id)
        header = client.chat(CONTEXT_PROMPT.format(title=title, chunk=c.text[:1500]))
        out.append((header.strip(), c))
    return out

contextual = contextualise(chunks)
print(f'Generated {len(contextual)} contextual headers.')
print()
print('Three headers + opening lines of their chunks:')
for header, c in contextual[:3]:
    print(f'  HEADER: {header}')
    print(f'  CHUNK : {c.text[:100]!r}...')
    print()

Generated 24 contextual headers.

Three headers + opening lines of their chunks:
  HEADER: This chunk serves as the introductory chapter of the "Getting Started" document, covering the initial steps of installing Rust, writing a simple program, and utilizing the cargo package manager.
  CHUNK : '# Getting Started Let’s start your Rust journey! There’s a lot to learn, but every journey starts so'...

  HEADER: This chunk serves as the introduction to the document "Programming a Guessing Game", covering the overview of the project, including setting up a new Rust project using Cargo, and introducing the guessing game that will be implemented throughout the chapter.
  CHUNK : '# Programming a Guessing Game Let’s jump into Rust by working through a hands-on project together! T'...

  HEADER: This chunk is from the "Processing a Guess" section of the "Programming a Guessing Game" document, covering the initial part of the guessing game program that asks for user input, processes it, and checks its form, including code implementation and explanation of the `io` input/output library.
  CHUNK : 'command comes in handy when you need to rapidly iterate on a project, as we’ll do in this game, quic'...

Step 4 — Embed `header + chunk` and index¶

Standard embedding, but the input is the concatenation. Nothing fancy. The whole technique lives in step 3; this step is just plumbing.

In [5]:

Copied!





from cookbook.stores import QdrantBackend

augmented_texts = [f'{h}\n\n{c.text}' for h, c in contextual]
vectors = client.embed(augmented_texts)

store = QdrantBackend('contextual', dim=len(vectors[0]))
store.add(
    texts=augmented_texts,
    vectors=vectors,
    metadatas=[c.metadata for _, c in contextual],
    ids=[c.chunk_id for _, c in contextual],
)
print(f'Indexed {len(augmented_texts)} contextualised chunks.')
from cookbook.stores import QdrantBackend

augmented_texts = [f'{h}\n\n{c.text}' for h, c in contextual]
vectors = client.embed(augmented_texts)

store = QdrantBackend('contextual', dim=len(vectors[0]))
store.add(
    texts=augmented_texts,
    vectors=vectors,
    metadatas=[c.metadata for _, c in contextual],
    ids=[c.chunk_id for _, c in contextual],
)
print(f'Indexed {len(augmented_texts)} contextualised chunks.')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Indexed 24 contextualised chunks.

Step 5 — Compare retrieval on a question that needs context¶

Pick a question whose answer chunk would be hard to find without its document position. Run retrieval against both the contextual store and a parallel store built from raw (uncontextualised) chunks. The gap is the technique's value.

In [6]:

Copied!





raw_vectors = client.embed([c.text for c in chunks])
raw_store = QdrantBackend('raw', dim=len(raw_vectors[0]))
raw_store.add([c.text for c in chunks], raw_vectors, ids=[c.chunk_id for c in chunks])

q = 'When inside an async block do I need to await a future for it to make progress?'
qv = client.embed([q])[0]

print('--- raw retrieval (no headers) ---')
for h in raw_store.search(qv, top_k=3):
    print(f'  score={h.score:.3f}  {h.text[:160]}')

print()
print('--- contextual retrieval ---')
for h in store.search(qv, top_k=3):
    print(f'  score={h.score:.3f}  {h.text[:160]}')
raw_vectors = client.embed([c.text for c in chunks])
raw_store = QdrantBackend('raw', dim=len(raw_vectors[0]))
raw_store.add([c.text for c in chunks], raw_vectors, ids=[c.chunk_id for c in chunks])

q = 'When inside an async block do I need to await a future for it to make progress?'
qv = client.embed([q])[0]

print('--- raw retrieval (no headers) ---')
for h in raw_store.search(qv, top_k=3):
    print(f'  score={h.score:.3f}  {h.text[:160]}')

print()
print('--- contextual retrieval ---')
for h in store.search(qv, top_k=3):
    print(f'  score={h.score:.3f}  {h.text[:160]}')

--- raw retrieval (no headers) ---
  score=0.430  <!-- manual-regeneration cd listings/ch02-guessing-game-tutorial/no-listing-03-convert-string-to-number/ touch src/main.rs cargo run 76 --> ```console $ cargo r
  score=0.401  command comes in handy when you need to rapidly iterate on a project, as we’ll do in this game, quickly testing each iteration before moving on to the next one.
  score=0.401  input your guess. 10 You guessed: 10 Too small! Please input your guess. 99 You guessed: 99 Too big! Please input your guess. foo Please input your guess. 61 Yo

--- contextual retrieval ---
  score=0.359  This chunk serves as the introduction to the chapter "Understanding Ownership", covering the fundamental concept of ownership in Rust and its related features, 
  score=0.320  This chunk is from the "Programming a Guessing Game" document and covers generating a random number using the `rand` crate, accessing crate documentation, and t
  score=0.306  This chunk is the final section of the "Programming a Guessing Game" document, covering the completion of the guessing game project, a summary of the concepts l

Step 6 — Wrap it as `answer_question`¶

Standard contract. The augmented chunk text (header + body) gets passed to the LLM; the model is free to use the header for grounding too.

In [7]:

Copied!





PROMPT = (
    'Use only the passages below to answer the question. '
    'If they do not contain the answer, say so plainly.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    hits = store.search(qv, top_k=k)
    contexts = [h.text for h in hits]
    answer = client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question))
    return answer, contexts

ans, _ = answer_question('How do you mark a function as runnable inside the async runtime, and what changes about its return type?')
print(ans)
PROMPT = (
    'Use only the passages below to answer the question. '
    'If they do not contain the answer, say so plainly.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    hits = store.search(qv, top_k=k)
    contexts = [h.text for h in hits]
    answer = client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question))
    return answer, contexts

ans, _ = answer_question('How do you mark a function as runnable inside the async runtime, and what changes about its return type?')
print(ans)

The passages provided do not contain the answer to this question. They discuss various topics such as ownership in Rust, using crates, and programming a guessing game, but do not mention marking a function as runnable inside an async runtime or changes to its return type.

Look Inside¶

Inspect — what does a typical header look like?¶

Headers are the technique. If they are vague, the technique gives you nothing. If they are specific, you get the recall lift. Read five of them.

In [8]:

Copied!





for header, c in contextual[10:15]:
    print(f'  Chapter   : {c.metadata.get("chapter", c.doc_id)}')
    print(f'  Header    : {header}')
    print(f'  Chunk start: {c.text[:80]!r}...')
    print()
for header, c in contextual[10:15]:
    print(f'  Chapter   : {c.metadata.get("chapter", c.doc_id)}')
    print(f'  Header    : {header}')
    print(f'  Chunk start: {c.text[:80]!r}...')
    print()

Chapter : ch02-00-guessing-game-tutorial
Header : This chunk is from the "Programming a Guessing Game" document, specifically covering the section on how Cargo handles dependencies and compilation, including its ability to avoid recompiling unchanged code and dependencies, and ensuring reproducible builds.
Chunk start: 'Cargo also grabbed other crates that `rand` depends on to work. After downloadin'...

Chapter : ch02-00-guessing-game-tutorial
Header : This chunk is located in the "Managing Dependencies" or "Building and Dependency Management" section of the "Programming a Guessing Game" document and covers how Cargo manages dependencies, including the role of the Cargo.lock file in ensuring reproducible builds and the process of updating crates to new versions using the `cargo update` command.
Chunk start: 'the _guessing_game_ directory. When you build a project for the first time, Carg'...

Chapter : ch02-00-guessing-game-tutorial
Header : This chunk is part of Chapter 2 of the "Programming a Guessing Game" document, covering the process of using the `rand` crate to generate a random number, including updating dependencies and implementing the necessary code in the `src/main.rs` file.
Chunk start: 'are now using is 0.8.6. To use `rand` version 0.999.0 or any version in the 0.99'...

Chapter : ch02-00-guessing-game-tutorial
Header : This chunk is from the "Programming a Guessing Game" document and covers generating a random number using the `rand` crate, accessing crate documentation, and testing the guessing game program.
Chunk start: 'takes a range expression as an argument and generates a random number in the ran'...

Chapter : ch02-00-guessing-game-tutorial
Header : This chunk is part of the "Programming a Guessing Game" document, specifically covering the comparison of the user's guess to the secret number, including an explanation of the `std::cmp::Ordering` enum and a `match` expression to handle the possible outcomes.
Chunk start: '## Comparing the Guess to the Secret Number Now that we have user input and a ra'...

Good headers name the chapter, the sub-topic, and where in the flow this chunk sits. Bad headers say things like "This chunk discusses Rust" — too generic to add retrieval signal. If yours look bad, sharpen the CONTEXT_PROMPT.

Inspect — measure recall@5 on the eval slice¶

Use the corpus's eval-set slice. Run the loose recall proxy from Recipe 3 on both raw and contextual stores. The contextual store should beat raw by 3–10 points on a corpus that benefits — Rust chapters are a moderate-benefit case.

In [9]:

Copied!





from cookbook.corpora import load_eval_questions

qs = [q for q in load_eval_questions() if q['corpus'] == 'rust-book'][:10]
def recall(store_):
    hits = 0
    for q in qs:
        qv = client.embed([q['question']])[0]
        retrieved = store_.search(qv, top_k=5)
        gold_words = [w.lower() for w in q['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in r.text.lower() for w in gold_words) for r in retrieved):
            hits += 1
    return hits / max(1, len(qs))

print(f'raw recall@5         = {recall(raw_store):.2f}')
print(f'contextual recall@5  = {recall(store):.2f}')
from cookbook.corpora import load_eval_questions

qs = [q for q in load_eval_questions() if q['corpus'] == 'rust-book'][:10]
def recall(store_):
    hits = 0
    for q in qs:
        qv = client.embed([q['question']])[0]
        retrieved = store_.search(qv, top_k=5)
        gold_words = [w.lower() for w in q['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in r.text.lower() for w in gold_words) for r in retrieved):
            hits += 1
    return hits / max(1, len(qs))

print(f'raw recall@5         = {recall(raw_store):.2f}')
print(f'contextual recall@5  = {recall(store):.2f}')

raw recall@5         = 1.00

contextual recall@5  = 1.00

Inspect — what is the per-chunk cost?¶

We measure the size of the contextualisation payload so the cost is visible. Multiply by your corpus size at scale; the prompt cache lowers the input-token bill by 5–10x in production.

In [10]:

Copied!





import tiktoken
enc = tiktoken.get_encoding('cl100k_base')
header_tokens = enc.encode(contextual[0][0])
chunk_tokens = enc.encode(contextual[0][1].text)
prompt_tokens = enc.encode(CONTEXT_PROMPT.format(title=contextual[0][1].metadata.get('title', ''), chunk=contextual[0][1].text[:1500]))
print(f'Header  tokens : {len(header_tokens)}')
print(f'Chunk   tokens : {len(chunk_tokens)}')
print(f'Prompt  tokens : {len(prompt_tokens)}  (cached after first call on a given document)')
import tiktoken
enc = tiktoken.get_encoding('cl100k_base')
header_tokens = enc.encode(contextual[0][0])
chunk_tokens = enc.encode(contextual[0][1].text)
prompt_tokens = enc.encode(CONTEXT_PROMPT.format(title=contextual[0][1].metadata.get('title', ''), chunk=contextual[0][1].text[:1500]))
print(f'Header  tokens : {len(header_tokens)}')
print(f'Chunk   tokens : {len(chunk_tokens)}')
print(f'Prompt  tokens : {len(prompt_tokens)}  (cached after first call on a given document)')

Header  tokens : 35
Chunk   tokens : 65
Prompt  tokens : 126  (cached after first call on a given document)

Inspect — does the technique change which chunks rank first?¶

Compare the top-1 chunk under raw vs contextual retrieval for several questions. The interesting case is when the top-1 changes — that is where contextual retrieval found a better answer than raw.

In [11]:

Copied!





questions = [
    'How do you define a thread-safe shared counter in Rust?',
    'When should I prefer Rc over Arc?',
    'What does the question-mark operator do for error propagation?',
    'In an async function, what type does the function actually return?',
]
for q in questions:
    qv = client.embed([q])[0]
    raw_top = raw_store.search(qv, top_k=1)[0]
    ctx_top = store.search(qv, top_k=1)[0]
    same = raw_top.doc_id == ctx_top.doc_id
    print(f'  {"same" if same else "DIFFERENT":10s}  {q[:70]}')
questions = [
    'How do you define a thread-safe shared counter in Rust?',
    'When should I prefer Rc over Arc?',
    'What does the question-mark operator do for error propagation?',
    'In an async function, what type does the function actually return?',
]
for q in questions:
    qv = client.embed([q])[0]
    raw_top = raw_store.search(qv, top_k=1)[0]
    ctx_top = store.search(qv, top_k=1)[0]
    same = raw_top.doc_id == ctx_top.doc_id
    print(f'  {"same" if same else "DIFFERENT":10s}  {q[:70]}')

  same        How do you define a thread-safe shared counter in Rust?
  same        When should I prefer Rc over Arc?
  DIFFERENT   What does the question-mark operator do for error propagation?

  DIFFERENT   In an async function, what type does the function actually return?

Run It¶

Representative end-to-end run on a question whose answer chunk depends on its document position.

In [12]:

Copied!





q = 'When inside an async function do you need to await another async call before its work makes progress?'
ans, ctxs = answer_question(q)
print('=== Answer ===')
print(ans)
print()
print('Top context preview:')
print(ctxs[0][:300])
q = 'When inside an async function do you need to await another async call before its work makes progress?'
ans, ctxs = answer_question(q)
print('=== Answer ===')
print(ans)
print()
print('Top context preview:')
print(ctxs[0][:300])

=== Answer ===
The passages provided do not contain the answer to this question. They discuss various topics related to Rust programming, such as ownership, borrowing, and creating a guessing game, but do not address the specific question about async functions.

Top context preview:
This chunk serves as the introduction to the chapter "Understanding Ownership", covering the fundamental concept of ownership in Rust and its related features, including borrowing, slices, and memory layout.

# Understanding Ownership Ownership is Rust’s most unique feature and has deep implications

Side by Side: Vanilla Baseline vs This Technique¶

Vanilla baseline vs contextual retrieval on the same question. The interesting metric here is not the answer text — both pipelines usually answer correctly — but which passages were retrieved, and whether the contextual version pulled in the position-dependent chunk that vanilla missed.

In [13]:

Copied!





from cookbook.baselines import vanilla_pipeline

q = 'When inside an async function do you need to await another async call before its work makes progress?'
base = vanilla_pipeline(q, corpus='rust-book', top_k=5)
ours_a, ours_c = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'top_context_start': base.contexts[0][:140]},
    {'pipeline': 'contextual', 'top_context_start': ours_c[0][:140]},
])
from cookbook.baselines import vanilla_pipeline

q = 'When inside an async function do you need to await another async call before its work makes progress?'
base = vanilla_pipeline(q, corpus='rust-book', top_k=5)
ours_a, ours_c = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'top_context_start': base.contexts[0][:140]},
    {'pipeline': 'contextual', 'top_context_start': ours_c[0][:140]},
])

Out[13]:

	pipeline	top_context_start
0	vanilla	# Fundamentals of Asynchronous Programming: As...
1	contextual	This chunk serves as the introduction to the c...

Knobs to Turn¶

Five knobs:

The CONTEXT_PROMPT itself. The biggest lever. A prompt that asks for sub-topic, sub-section, and position usually beats a prompt that asks only for sub-topic. Iterate by hand-reading 10 generated headers and rewriting the prompt to fix the failure mode you see.
Header position. We prepend the header to the chunk before embedding. Some teams prepend and append; some put the header in metadata only and rely on hybrid search. Anthropic's reference puts header before chunk; we follow.
Header model. Headers do not need to be written by your best model. A small fast model (Llama-3.3-8B, Qwen-2.5-7B) produces headers of comparable quality at a fraction of the cost. Test on your corpus.
How much document context to give the model. Anthropic feeds the full document alongside the chunk so the header can refer to surrounding sections. For long documents this is expensive without prompt caching; we use the first 1500 chars of the chunk only here for clarity. In production, feed the full document and enable caching.
Re-contextualisation cadence. When you upgrade the header model or rewrite the prompt, you must re-embed every chunk. Plan for it. Many teams contextualise once, then leave the headers fixed across embedder upgrades.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [14]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'rust-book']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'rust-book']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[14]:

	question	expected	actual	contexts_retrieved
0	What is ownership in Rust?	A set of rules governing how memory is managed...	The passages provided do not contain a detaile...	5
1	What does the borrow checker do?	It statically enforces that references obey th...	The passages provided do not contain informati...	5
2	What is the difference between String and &str?	`String` is an owned, heap-allocated, growable...	The passages do not contain the answer to this...	5
3	Describe how match is exhaustive in Rust.	The compiler requires `match` arms to cover ev...	The passages provided do not contain a direct ...	5
4	What is a trait?	A trait is a named set of methods that types c...	The passages provided do not contain a clear a...	5

Closing Thoughts¶

Three places contextual retrieval falls down:

Tiny chunks. If your chunks are already short enough to be self-explanatory, a header adds nothing. We see this in proposition-decomposition pipelines (Recipe 8) where each chunk is a single factual claim.
Bad headers. A model that does not understand your domain writes vague headers. Test on a labelled slice before going to production. If the model writes "This passage discusses programming language semantics" five times in a row, the prompt is too generic.
Storage of full augmented text. The augmented text (header + chunk) is what you embed, but you also have to store it so retrieval can return the original chunk. This roughly doubles storage at the chunk level; small for most workloads, real at billions.

Compose it with everything: BM25 hybrid retrieval (Recipe 18), cross-encoder reranking (Recipe 22), Self-RAG filtering (Recipe 24). Anthropic's full pipeline is contextual retrieval + BM25 + reranking; that is what the 49 percent and 67 percent numbers came from.