A Tour of the Cookbook's Primitives¶
What problem does this solve?¶
Every later recipe leans on four primitives: a provider client that hides the chat / embed / rerank vendor SDKs, a set of corpus loaders that yield uniform Document records, a thin vector-store wrapper with a common .add/.search API, and a tracing toggle. If you skip this notebook, the first real recipe will throw four unfamiliar imports at you in cell three and you will be reading API docs instead of learning RAG.
We are not building RAG yet. We are checking that every primitive works in the environment you actually run code in, before we try to combine them. Five minutes here saves an hour of unrelated debugging later.
Where it came from¶
The four-primitive split mirrors what production RAG infrastructure has converged on since 2024: an LLM gateway (LiteLLM, Portkey, OpenRouter), a typed document model (LlamaIndex Document, LangChain Document), a backend-agnostic vector interface, and OpenTelemetry-grade tracing. The naming in this cookbook is intentionally distinct from LangChain and LlamaIndex so the conceptual gap stays visible — you should be able to read the code and tell exactly what each layer does.
We do not invent any of the primitives we use. cookbook.providers is a thin wrapper around LiteLLM; cookbook.stores wraps Qdrant, LanceDB, and Chroma; cookbook.tracing wraps Arize Phoenix or LangSmith. The wrappers exist to keep the recipes readable and provider-agnostic, not to hide capability. When you outgrow a wrapper, drop down to the underlying library — every wrapper exposes the raw client as a property.
Where it fits in the RAG landscape¶
Three boundaries you should understand before going further:
- Provider ↔ recipe. The
LLMClientis provider-agnostic by design. SetPROVIDER=nebiusin.envand your notebook runs on Nebius. SetPROVIDER=openaiand it runs on OpenAI with the same code. Recipes never importlitellm,openai, oranthropicdirectly. That boundary is the entire reason this cookbook does not break the day a vendor renames a model. - Corpus ↔ chunker. Loaders yield
Documentobjects with stable IDs and text content. Chunkers consumeDocumentlists. If you want a custom corpus, write a loader; you do not have to touch chunkers, embeddings, or stores. - Store ↔ retriever. Every vector store wrapped here (Qdrant, LanceDB, Chroma) exposes
add()andsearch()with identical signatures. Switching backends in a recipe is one line.
When to use it (and when not to)¶
Read this notebook once when you set up the repo, and again if you are debugging an environment issue. Skip it if you already have everything green — the actual RAG starts in the vanilla pipeline (Recipe 2). If any cell here fails, fix it before going further. A broken provider means every recipe's chat/embed call fails; a missing corpus means every retrieval gets zero results; a missing tracing dependency means you can read but not debug.
The intuition¶
Think of the cookbook as four layers, bottom-up:
- Data layer —
cookbook.corporayields uniform documents. - Vector layer —
cookbook.chunkerscuts them up,cookbook.providers.embed()vectorises,cookbook.storesindexes. - Retrieval layer —
cookbook.retrieversandcookbook.rerankersfind and reorder candidates. - Generation layer —
cookbook.providers.chat()answers from retrieved context, withcookbook.tracingwatching.
Every later recipe is a vertical slice through these four layers, swapping out one or two pieces per slice. If you keep the mental model clean, the cookbook reads like variations on a single theme.
Architecture¶
flowchart TB
subgraph Data
C1[load_arxiv_mamba]
C2[load_wikipedia_superconductors]
C3[load_sec_10k]
C4[load_rust_book]
end
subgraph Vectors
K[chunkers] --> EM[providers.embed]
EM --> ST[stores: Qdrant / LanceDB / Chroma]
end
subgraph Retrieval
RT[retrievers: hybrid / MMR / RRF]
RR[rerankers: cross-encoder / listwise]
end
subgraph Generation
CH[providers.chat]
end
subgraph Observability
TR[tracing.init_tracing]
EV[eval suite]
end
Data --> K
ST --> RT --> RR --> CH
TR -.-> EM
TR -.-> CH
EV -.-> CH
References¶
- 📚 LiteLLM — One SDK for 100+ LLMs — Hides vendor differences across chat and embedding APIs.
- 📚 Nebius AI Studio — Inference API reference — OpenAI-compatible endpoint we default to.
- 📚 Qdrant Python client documentation — The default vector store in this cookbook.
- 📚 Arize Phoenix — open-source LLM observability — OpenTelemetry-native tracing used by
cookbook.tracing. - 📚 OpenTelemetry GenAI semantic conventions — Why every LLM call ends up with the same span shape.
- 📚 RAGAS — reference-free evaluation for RAG systems — Powers the evaluation slice every recipe ends with.
Setup¶
Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')
from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing
client = LLMClient()
print(f'Provider: {client.provider} | Chat model: {client.chat_model}')
print(init_tracing())
Provider: nebius | Chat model: meta-llama/Llama-3.3-70B-Instruct Tracing disabled.
Build the Pipeline, Step by Step¶
Step 1 — Confirm the provider client wakes up¶
We will instantiate LLMClient with no arguments. It reads PROVIDER from the environment (defaults to nebius), picks the corresponding chat and embedding models, and is now ready for client.chat() and client.embed() calls. We will print the resolved settings so you can sanity-check them.
from cookbook.providers import LLMClient, list_providers
print('All supported providers:')
for name in list_providers():
print(f' - {name}')
print()
print(f'Current PROVIDER env var: {os.environ.get("PROVIDER", "(unset)")}')
print(f'Resolved client:')
print(f' provider : {client.provider}')
print(f' chat model : {client.chat_model}')
print(f' embed model: {client.embed_model}')
All supported providers: - nebius - openai - anthropic - groq - openrouter - together Current PROVIDER env var: nebius Resolved client: provider : nebius chat model : meta-llama/Llama-3.3-70B-Instruct embed model: Qwen/Qwen3-Embedding-8B
Switching providers later is a one-line change: client = LLMClient(provider='openai') (or set PROVIDER=openai in .env). Every line of recipe code from there on stays identical.
Step 2 — Round-trip a chat and an embedding¶
The single best check that everything is wired up. One short chat call returns a string; one embed call returns a vector. If either throws, fix it before going further — every later recipe will hit one of these in the first three cells.
import numpy as np
answer = client.chat('In one short sentence, what does RAG stand for and why is it useful?')
print('Chat round-trip:')
print(f' {answer}')
print()
vec = client.embed(['Retrieval-Augmented Generation grounds LLM answers in retrieved documents.'])[0]
v = np.asarray(vec)
print(f'Embedding round-trip:')
print(f' dim = {v.shape[0]}')
print(f' L2 norm = {np.linalg.norm(v):.4f}')
print(f' first 6 = {[round(float(x), 3) for x in v[:6]]}')
Chat round-trip: RAG stands for Red, Amber, Green, a color-coded system used to indicate status or risk levels, making it a useful visual tool for quick decision-making and prioritization. Embedding round-trip: dim = 4096 L2 norm = 1.0000 first 6 = [0.014, -0.002, 0.001, -0.052, 0.035, -0.021]
Both worked. The embedding norm should be close to 1 (Nebius normalises by default); a far-from-1 norm sometimes means you accidentally pointed at a model that does not normalise, in which case cosine code needs an extra divide.
Step 3 — Load each of the four corpora¶
Every recipe in the cookbook draws from one of four hand-picked public-domain corpora. Today we just confirm each loads, see how many documents it yields, and look at one sample document so the structure is concrete in your head.
from cookbook.corpora import (
load_arxiv_mamba,
load_wikipedia_superconductors,
load_sec_10k,
load_rust_book,
)
loaders = [
('arxiv-mamba', load_arxiv_mamba),
('wikipedia-superconductors', load_wikipedia_superconductors),
('sec-10k-pltr', load_sec_10k),
('rust-book', load_rust_book),
]
for name, loader in loaders:
docs = list(loader())
sample = docs[0]
avg_chars = sum(len(d.text) for d in docs) // max(1, len(docs))
print(f'{name:30s} docs={len(docs):4d} avg_chars={avg_chars:5d} sample_id={sample.doc_id}')
arxiv-mamba docs= 5 avg_chars= 5550 sample_id=arxiv:2403-mamba-survey#p1 wikipedia-superconductors docs= 42 avg_chars= 571 sample_id=wiki:BCS_theory sec-10k-pltr docs= 1 avg_chars=586419 sample_id=sec:PLTR-2024-10K#all rust-book docs= 18 avg_chars= 4337 sample_id=rust-book:ch01-00-getting-started
Different corpora hit different failure modes — the SEC filing is long and structured, the Wikipedia subset is many short entries, the Rust book chapters are medium-length prose with code, and the arXiv Mamba survey is one long technical document split across pages. Later recipes deliberately exercise different corpora so you see how each technique behaves across shapes.
Step 4 — Build a tiny Qdrant index and search it¶
We chunk one corpus, embed the chunks, push them into an in-memory Qdrant collection, and run one search. The point is not the answer — it is that every backend touches Nebius (for embedding) and Qdrant (for indexing) without any manual configuration.
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend
docs = list(load_rust_book())
chunks = sentence_window(docs, sentences_per_chunk=5, overlap=1)[:120]
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('tour-rust', dim=len(vectors[0]))
store.add(
texts=[c.text for c in chunks],
vectors=vectors,
metadatas=[c.metadata for c in chunks],
ids=[c.chunk_id for c in chunks],
)
q_vec = client.embed(['What does the Rust compiler do when two threads share mutable state?'])[0]
hits = store.search(q_vec, top_k=3)
for i, h in enumerate(hits, 1):
print(f'{i}. score={h.score:.3f}')
print(f' {h.text[:200]}')
print()
C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
1. score=0.707 Therefore, Rust offers a variety of tools for modeling problems in whatever way is appropriate for your situation and requirements. Here are the topics we’ll cover in this chapter: - How to create th 2. score=0.680 # Fearless Concurrency Handling concurrent programming safely and efficiently is another of Rust’s major goals. _Concurrent programming_, in which different parts of a program execute independently, 3. score=0.656 We’ve nicknamed this aspect of Rust _fearless concurrency_. Fearless concurrency allows you to write code that is free of subtle bugs and is easy to refactor without introducing new bugs. > Note: For
If you got three reasonably on-topic chunks, the entire vector path is healthy: chunker → embedder → store → searcher. That is the spine of every recipe in this cookbook.
Step 5 — Generate an answer using retrieved chunks¶
One LLM call against the retrieved context. This is the tiniest possible end-to-end RAG, on purpose. Anything fancier lives in Recipe 2 onward.
context = '\n\n'.join(h.text for h in hits)
prompt = (
'Use only the passages below. If they do not contain the answer, say so plainly.\n\n'
f'Passages:\n{context}\n\nQuestion: How does Rust prevent data races when two threads share mutable state?\nAnswer:'
)
answer = client.chat(prompt)
print(answer)
The passages do not contain the answer. They mention shared-state concurrency and the goal of preventing concurrency problems, but they do not specifically explain how Rust prevents data races when two threads share mutable state.
The answer should mention Send/Sync, Mutex, or borrow rules — whatever is in those three chunks. If it generalises beyond the passages, the model leaked prior knowledge; recipe 40 (Lynx guardrails) catches that in production.
Step 6 — Confirm the disk cache is wired¶
Every chat() and embed() call this cookbook makes is fingerprinted (SHA256 of model + payload) and cached under .cache/providers/. The second time you run the same notebook, the cache pays for itself many times over. We just check that the directory exists and has entries from the calls we already made.
from cookbook import _cache
stats = _cache.stats()
print(f'Cache entries: {stats["entries"]}')
print('Disable for one notebook with: os.environ["COOKBOOK_CACHE"] = "0"')
Cache entries: 3508 Disable for one notebook with: os.environ["COOKBOOK_CACHE"] = "0"
Cache files live at .cache/providers/<sha256>.json. To wipe and force a fresh pass, delete the directory. The cache is gitignored so it never leaks tokens or model outputs into version control.
Look Inside¶
Inspect — the corpus eval set¶
Every recipe ends with a small evaluation slice. The eval set is 80 hand-curated question/answer pairs across the four corpora. Quick look at the shape so the final cell of every other recipe makes sense.
from cookbook.corpora import load_eval_questions
eval_rows = load_eval_questions()
print(f'Total eval questions: {len(eval_rows)}')
print()
from collections import Counter
for corpus, count in Counter(r['corpus'] for r in eval_rows).items():
print(f' {corpus:30s} {count} questions')
print()
print('First eval row:')
for k, v in eval_rows[0].items():
print(f' {k:10s} = {v!r}')
Total eval questions: 80 arxiv-mamba 20 questions wikipedia-superconductors 20 questions sec-10k-pltr 20 questions rust-book 20 questions First eval row: corpus = 'arxiv-mamba' question = 'What problem do state-space models aim to solve compared to attention-based transformers?' answer = 'State-space models target the quadratic time and memory complexity of self-attention, providing linear-time sequence modeling with selective recurrence.' difficulty = 'easy'
Twenty questions per corpus, split across easy, medium, hard difficulty. Recipes 37–40 exercise this set with proper RAGAS and DeepEval metrics; other recipes just print a quick spot-check table.
Inspect — what tracing would look like¶
We keep tracing off in published notebooks so the outputs are clean. In your own runs, flip COOKBOOK_TRACING=phoenix and a local Phoenix UI launches at http://localhost:6006 with one span per chat/embed call. Recipe 39 walks the trace tree for a debugging session.
from cookbook.tracing import init_tracing
print('Tracing init result:', init_tracing(backend='off'))
print('To enable: set COOKBOOK_TRACING=phoenix in .env before launching Jupyter.')
Tracing init result: Tracing disabled. To enable: set COOKBOOK_TRACING=phoenix in .env before launching Jupyter.
Inspect — measure cache effectiveness¶
We re-run the embed call from Step 2 and measure how long it takes. With the cache hit, this should be milliseconds; without, it would be a network round trip.
import time
t0 = time.perf_counter()
_ = client.embed(['Retrieval-Augmented Generation grounds LLM answers in retrieved documents.'])[0]
dt = (time.perf_counter() - t0) * 1000
print(f'Cached embed call took {dt:.1f} ms')
if dt < 50:
print('Confirmed: cache served the request (no network round trip).')
else:
print('Looks like a live network call — check COOKBOOK_CACHE env var.')
Cached embed call took 7.5 ms Confirmed: cache served the request (no network round trip).
Single-digit milliseconds is the cache; hundreds of milliseconds is a live call. This is the difference between a free re-run and one that costs Nebius tokens.
Inspect — what files exist on disk¶
A quick map of the repo so you know where things live. If something in this list is missing on your machine, run python scripts/fetch_corpus.py from the repo root.
from pathlib import Path
ROOT = Path('..').resolve().parent if Path.cwd().name == '01-foundations' else Path.cwd()
if not (ROOT / 'pyproject.toml').exists():
ROOT = Path.cwd().parents[1]
for sub in ['corpus', 'cookbook', 'recipes', 'scripts', '.cache']:
p = ROOT / sub
exists = '✓' if p.exists() else '✗'
print(f' {exists} {sub:10s} {p}')
✓ corpus C:\Users\faree\Desktop\rag\rag-cookbook-2026\corpus ✓ cookbook C:\Users\faree\Desktop\rag\rag-cookbook-2026\cookbook ✓ recipes C:\Users\faree\Desktop\rag\rag-cookbook-2026\recipes ✓ scripts C:\Users\faree\Desktop\rag\rag-cookbook-2026\scripts ✓ .cache C:\Users\faree\Desktop\rag\rag-cookbook-2026\.cache
Run It¶
We have already exercised every primitive above. The 'run' cell here is just a final, slightly larger end-to-end check using the same machinery — chunk, embed, store, search, generate.
from cookbook.corpora import load_wikipedia_superconductors
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend
docs = list(load_wikipedia_superconductors())
chunks = sentence_window(docs, sentences_per_chunk=4)[:120]
vectors = client.embed([c.text for c in chunks])
store2 = QdrantBackend('tour-superconductors', dim=len(vectors[0]))
store2.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
def answer_question(question: str, k: int = 4) -> tuple[str, list[str]]:
qv = client.embed([question])[0]
hits = store2.search(qv, top_k=k)
contexts = [h.text for h in hits]
prompt = ('Use only these passages.\n\n' + '\n\n'.join(contexts) +
f'\n\nQuestion: {question}\nAnswer:')
return client.chat(prompt), contexts
ans, ctxs = answer_question('In one paragraph, what is the Meissner effect?')
print(ans)
The Meissner effect is a phenomenon in condensed-matter physics where a magnetic field is expelled from a superconductor during its transition to the superconducting state when it is cooled below the critical temperature. This expulsion causes the superconductor to repel a nearby magnet. The effect shields the superconductor from all magnetic fields, resulting in repulsion, and is closely related to the phenomenon of a superconductor levitating when flux tubes are pinned in place at lower temperatures.
Knobs to Turn¶
There are no knobs to tune in this notebook — its only job is to confirm the four primitives wake up correctly. But the choices you make here apply across the whole cookbook:
- Pick your provider carefully. Nebius is the default because it is cheap and OpenAI-compatible. If you have an OpenAI key, set
PROVIDER=openaiand the same code runs againsttext-embedding-3-largeandgpt-4o-mini. For local-only runs, setPROVIDER=localto push embeddings throughsentence-transformers; you will not get chat without a separate hosted call. - Pick your vector store carefully. Qdrant in-memory is the default for notebook reproducibility. For real workloads point
QDRANT_URLat a Qdrant cluster; the code is identical. LanceDB is the choice for laptop-scale persistence; Chroma is the choice for the smallest possible footprint. - Pick your tracing posture early. During development, turn Phoenix on (
COOKBOOK_TRACING=phoenix) and never turn it off — the cost is a few megabytes of disk and the payoff is a complete history of every call you made. - Decide on caching policy. The disk cache is on by default. For benchmarking, set
COOKBOOK_CACHE=0to force real network calls. For authoring, leave it on so re-running a notebook is free.
Evaluate on a Slice¶
Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample
qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'rust-book']
samples = []
for row in qs[:5]:
answer, contexts = answer_question(row['question'])
samples.append({
'question': row['question'],
'expected': row['answer'],
'actual': answer[:200],
'contexts_retrieved': len(list(contexts)),
})
import pandas as pd
pd.DataFrame(samples)
| question | expected | actual | contexts_retrieved | |
|---|---|---|---|---|
| 0 | What is ownership in Rust? | A set of rules governing how memory is managed... | There is no information about ownership in Rus... | 4 |
| 1 | What does the borrow checker do? | It statically enforces that references obey th... | There is no information about a "borrow checke... | 4 |
| 2 | What is the difference between String and &str? | `String` is an owned, heap-allocated, growable... | There is no information in the provided passag... | 4 |
| 3 | Describe how match is exhaustive in Rust. | The compiler requires `match` arms to cover ev... | There is no information about Rust or the conc... | 4 |
| 4 | What is a trait? | A trait is a named set of methods that types c... | A trait of a transmon is reduced sensitivity t... | 4 |
Closing Thoughts¶
If every cell above ran clean, you are good to go. The vanilla pipeline (Recipe 2) builds on what you just used; every later recipe varies one or two of these primitives at a time.
Common ways this notebook fails on first run, and what to do:
- Provider auth error —
NEBIUS_API_KEYis missing from.env. Copy.env.exampleto.envand paste your key. - Corpus loader is empty — the corpus has not been downloaded yet. Run
python scripts/fetch_corpus.pyfrom the repo root. - Qdrant client error about UUIDs — older
qdrant-clientversions; runpip install -U qdrant-client. ModuleNotFoundError: cookbook— the package is not installed in your environment. From the repo root:pip install -e ..
If everything is green, jump to recipes/01-foundations/vanilla-pipeline.ipynb and start building.