Step-Back Abstraction — Retrieve Principles, Then Specifics¶

What problem does this solve?¶

A specific reasoning question ("Given selective scan, why is HiPPO initialization still useful?") often retrieves narrow passages and misses the background that makes the answer make sense. The model needs both: the specific passage that mentions HiPPO and the conceptual passage that explains why initialization matters at all. Step-back prompting (DeepMind, 2023) abstracts the specific question into a more general one ("What role does initialization play in state-space models?"). Retrieving the abstracted query pulls in principles. Combining principles with the specific query gives the model the reasoning ingredients it lacked.

Where it came from¶

Step-back was published by DeepMind in late 2023 ("Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models"). The paper showed 4–7 point improvements on MMLU-Phys and TimeQA when the abstracted query's retrievals were combined with the original. Most subsequent work extended the technique to multi-turn or multi-hop retrieval, but the core abstraction step is unchanged. By 2026 step-back is a standard branch inside multi-query fusion. The cookbook implements it standalone here for clarity; in production it composes with HyDE and N paraphrases inside reciprocal_rank_fusion. The composition is what makes the pattern feel polished — each query-transformation flavour catches different failure modes.

Where it fits in the RAG landscape¶

Among query-transformation cousins in this cookbook:

HyDE (Recipe 13). Concrete fake answer in answer-register.
Multi-query (Recipe 14). N paraphrases that emphasise different vocabulary.
Step-back (this recipe). One more-abstract query that retrieves principles.
Sub-question decomposition (Recipe 16). Break the question into parts and run RAG per part.

Step-back wins on reasoning questions over textbooks because it pulls in the background concepts the specific query alone wouldn't retrieve. Sub-question decomposition wins on multi-part questions. They compose well — sub-questions can each be step-back-expanded before retrieval.

When to use it (and when not to)¶

Use step-back on reasoning questions over textbooks, surveys, or didactic content. The technique adds background that the specific query alone wouldn't retrieve, which the model needs to construct a complete answer. Skip it on factoid questions. "In what year was YBCO discovered?" doesn't need abstraction; the specific entity is already what retrieval needs. Skip it when the corpus has no abstract layer to retrieve. If your documents are pure facts (FAQ entries, product specs), the abstracted query retrieves nothing useful and just costs you an extra LLM call. Skip it when the question is genuinely multi-part rather than reasoning-deep — sub-question decomposition (Recipe 16) is the right tool there.

The intuition¶

Four intuitions to keep in mind:

Reasoning questions hide a principle. "Given X, why Y?" assumes both X-specific knowledge and Y-principle knowledge. Standard retrieval finds X. Step-back retrieves the Y-principle so the model has both ingredients.

Abstracted queries match background sections. Survey papers and textbooks have introduction sections that explain principles. Abstracted queries retrieve those introductions; specific queries retrieve the body. The pair gives the model both.

Fusion is essential. Step-back alone usually under-performs the raw query because it retrieves only principles and may miss the specific entity. The combination is what wins; the cookbook's step_back_retrieve does the fusion under the hood.

Abstraction is cheap. The abstractor only writes one short rewrite. The cost is one LLM call plus one extra retrieval — well under double the vanilla pipeline cost.

Architecture¶

flowchart TB
  Q[Specific question] --> A[LLM: abstract
to a parent question]
  A --> AP[Parent question]
  Q --> R1[Retrieve specific]
  AP --> R2[Retrieve principles]
  R1 --> F[RRF fuse]
  R2 --> F
  F --> G[LLM answers]

References¶

📄 Take a Step Back — DeepMind, 2023 — The original step-back paper.
📚 LangChain step-back retriever tutorial — Reference implementation pattern.
📄 HyDE (Recipe 13) — Companion query-transformation; composes well.
📝 Multi-query fusion (Recipe 14) — Step-back is often one branch inside multi-query fusion.
📄 Self-Ask prompting — Adjacent reasoning-via-decomposition technique.
📄 Chain-of-Verification (CoVe) — A different abstraction-then-verification pattern.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Index the Mamba survey¶

Standard setup. The survey has both background sections (principles) and core sections (specifics), so step-back has somewhere to land both queries.

In [2]:

Copied!





from cookbook.corpora import load_arxiv_mamba
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_arxiv_mamba())
chunks = sentence_window(docs, sentences_per_chunk=4)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('stepback', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} chunks.')
from cookbook.corpora import load_arxiv_mamba
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_arxiv_mamba())
chunks = sentence_window(docs, sentences_per_chunk=4)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('stepback', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} chunks.')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Indexed 81 chunks.

Step 2 — The step-back prompt¶

Ask the LLM for a more general version of the question. The cookbook's cookbook.retrievers.step_back_retrieve uses this prompt under the hood; we replicate it here for clarity.

In [3]:

Copied!





ABSTRACT_PROMPT = (
    "Rewrite the user's question into a more general, higher-level question "
    'that asks about the underlying principle or category. Output only the rewritten question.\n\n'
    'Question: {q}'
)

q = 'Given that selective scan needs input-dependent transitions, why is HiPPO initialization still useful?'
parent = client.chat(ABSTRACT_PROMPT.format(q=q)).strip()
print(f'Specific: {q}')
print(f'Parent  : {parent}')
ABSTRACT_PROMPT = (
    "Rewrite the user's question into a more general, higher-level question "
    'that asks about the underlying principle or category. Output only the rewritten question.\n\n'
    'Question: {q}'
)

q = 'Given that selective scan needs input-dependent transitions, why is HiPPO initialization still useful?'
parent = client.chat(ABSTRACT_PROMPT.format(q=q)).strip()
print(f'Specific: {q}')
print(f'Parent  : {parent}')

Specific: Given that selective scan needs input-dependent transitions, why is HiPPO initialization still useful?
Parent  : What are the benefits of using initialization methods that don't rely on input data in neural network architectures?

A good abstracted query removes the specific entity ("selective scan", "HiPPO") and asks about the underlying principle ("initialization in state-space models"). If the parent question still names the specific entity, the prompt is too lenient.

Step 3 — Retrieve with both, fuse¶

Embed both queries, retrieve top-k for each, fuse with RRF. cookbook.retrievers.step_back_retrieve does this; we use it directly.

In [4]:

Copied!

from cookbook.retrievers import step_back_retrieve

hits = step_back_retrieve(q, store, client.chat, client.embed, top_k=6)
for h in hits:
    print(f'  {h.score:.4f}  {h.text[:160]}')
from cookbook.retrievers import step_back_retrieve

hits = step_back_retrieve(q, store, client.chat, client.embed, top_k=6)
for h in hits:
    print(f'  {h.score:.4f}  {h.text[:160]}')

  0.0328  Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
(a)
(b)
Figu
  0.0325  2024. Vision mamba: Efficient visual representation learning
with bidirectional state space model. arXiv preprint arXiv:2401.09417 (2024).
  0.0164  Association for Computing Machinery, New York, NY,
USA, 3953–3957. https://doi.org/10.1145/3511808.3557624
[4] Albert Gu and Tri Dao. 2023. Mamba: Linear-time s
  0.0164  Session-based Recommendations with Recurrent Neural Networks. In ICLR
(Poster).
[9] Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, and Domonkos
Tikk. 
  0.0161  Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
Trovato and Tobin, et al. Table 2: Overall performance comparison of different methods on KuaiRand and L
  0.0159  Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
interesting

Step 4 — Inspect what each branch contributes to the fused result¶

RRF fuses both rankings. Some passages come from the specific branch, some from the abstract branch, and some appear in both. We mark the source for transparency.

In [5]:

Copied!





specific_hits = {h.doc_id for h in store.search(client.embed([q])[0], top_k=10)}
abstract_hits = {h.doc_id for h in store.search(client.embed([parent])[0], top_k=10)}
for h in hits:
    source = []
    if h.doc_id in specific_hits: source.append('specific')
    if h.doc_id in abstract_hits: source.append('abstract')
    print(f'  rrf={h.score:.4f}  source={",".join(source) or "unknown"}  {h.text[:120]}')
specific_hits = {h.doc_id for h in store.search(client.embed([q])[0], top_k=10)}
abstract_hits = {h.doc_id for h in store.search(client.embed([parent])[0], top_k=10)}
for h in hits:
    source = []
    if h.doc_id in specific_hits: source.append('specific')
    if h.doc_id in abstract_hits: source.append('abstract')
    print(f'  rrf={h.score:.4f}  source={",".join(source) or "unknown"}  {h.text[:120]}')

  rrf=0.0328  source=specific,abstract  Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June
  rrf=0.0325  source=specific,abstract  2024. Vision mamba: Efficient visual representation learning
with bidirectional state space model. arXiv preprint arXiv:
  rrf=0.0164  source=specific  Association for Computing Machinery, New York, NY,
USA, 3953–3957. https://doi.org/10.1145/3511808.3557624
[4] Albert Gu
  rrf=0.0164  source=abstract  Session-based Recommendations with Recurrent Neural Networks. In ICLR
(Poster).
[9] Balázs Hidasi, Massimo Quadrana, Ale
  rrf=0.0161  source=abstract  Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
Trovato and Tobin, et al. Table 2: Overall performance compariso
  rrf=0.0159  source=specific  Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June

Step 5 — Wrap as `answer_question`¶

Standard contract.

In [6]:

Copied!





PROMPT = (
    'Use only the passages below to answer the question. '
    'If they do not contain the answer, say so plainly.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 6) -> tuple[str, list[str]]:
    hits = step_back_retrieve(question, store, client.chat, client.embed, top_k=k)
    contexts = [h.text for h in hits]
    return client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question)), contexts

ans, _ = answer_question(q)
print(ans)
PROMPT = (
    'Use only the passages below to answer the question. '
    'If they do not contain the answer, say so plainly.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 6) -> tuple[str, list[str]]:
    hits = step_back_retrieve(question, store, client.chat, client.embed, top_k=k)
    contexts = [h.text for h in hits]
    return client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question)), contexts

ans, _ = answer_question(q)
print(ans)

The passages do not contain the answer to the question. They discuss various topics related to sequential recommendation and state space models, but do not address the specific question about HiPPO initialization and selective scan.

Look Inside¶

Inspect — three abstractions¶

Read what the abstractor produces. The technique only works if abstractions are good.

In [7]:

Copied!





for question in [
    'Given selective scan, why is HiPPO still useful?',
    'In Mamba, why does parallel scan matter on modern GPUs?',
    'When would I prefer S4 over Mamba?',
]:
    parent = client.chat(ABSTRACT_PROMPT.format(q=question)).strip()
    print(f'Q: {question}')
    print(f'  parent: {parent}')
    print()
for question in [
    'Given selective scan, why is HiPPO still useful?',
    'In Mamba, why does parallel scan matter on modern GPUs?',
    'When would I prefer S4 over Mamba?',
]:
    parent = client.chat(ABSTRACT_PROMPT.format(q=question)).strip()
    print(f'Q: {question}')
    print(f'  parent: {parent}')
    print()

Q: Given selective scan, why is HiPPO still useful?
  parent: What is the value of heuristic methods in optimization problems, even when more efficient algorithms are available?

Q: In Mamba, why does parallel scan matter on modern GPUs?
  parent: What is the significance of parallel scan in accelerating computations on modern massively parallel architectures?

Q: When would I prefer S4 over Mamba?
  parent: What are the key factors that influence the choice between different build automation tools?

Inspect — top hits per branch¶

Specific query retrieves specific passages. Abstract query retrieves principle passages. Look at both.

In [8]:

Copied!





q = 'Given that selective scan needs input-dependent transitions, why is HiPPO initialization still useful?'
parent = client.chat(ABSTRACT_PROMPT.format(q=q)).strip()
print('--- Specific branch top-3 ---')
for h in store.search(client.embed([q])[0], top_k=3):
    print(f'  {h.score:.3f}  {h.text[:140]}')
print()
print('--- Abstract branch top-3 ---')
for h in store.search(client.embed([parent])[0], top_k=3):
    print(f'  {h.score:.3f}  {h.text[:140]}')
q = 'Given that selective scan needs input-dependent transitions, why is HiPPO initialization still useful?'
parent = client.chat(ABSTRACT_PROMPT.format(q=q)).strip()
print('--- Specific branch top-3 ---')
for h in store.search(client.embed([q])[0], top_k=3):
    print(f'  {h.score:.3f}  {h.text[:140]}')
print()
print('--- Abstract branch top-3 ---')
for h in store.search(client.embed([parent])[0], top_k=3):
    print(f'  {h.score:.3f}  {h.text[:140]}')

--- Specific branch top-3 ---
  0.585  2024. Vision mamba: Efficient visual representation learning
with bidirectional state space model. arXiv preprint arXiv:2401.09417 (2024).
  0.567  Association for Computing Machinery, New York, NY,
USA, 3953–3957. https://doi.org/10.1145/3511808.3557624
[4] Albert Gu and Tri Dao. 2023. 
  0.565  Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June 03–05, 2018, Woodst

--- Abstract branch top-3 ---

  0.443  Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June 03–05, 2018, Woodst
  0.414  Session-based Recommendations with Recurrent Neural Networks. In ICLR
(Poster).
[9] Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou,
  0.411  Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
Trovato and Tobin, et al. Table 2: Overall performance comparison of different metho

Inspect — step-back vs raw on factoid query¶

Step-back should not hurt factoid queries even when it doesn't help them. We verify.

In [9]:

Copied!





q = 'In what year was Mamba published?'
raw_top = store.search(client.embed([q])[0], top_k=1)[0]
sb_hits = step_back_retrieve(q, store, client.chat, client.embed, top_k=1)
print(f'Raw  top-1: {raw_top.text[:120]}')
print(f'SB   top-1: {sb_hits[0].text[:120]}')
print(f'Same chunk? {raw_top.doc_id == sb_hits[0].doc_id}')
q = 'In what year was Mamba published?'
raw_top = store.search(client.embed([q])[0], top_k=1)[0]
sb_hits = step_back_retrieve(q, store, client.chat, client.embed, top_k=1)
print(f'Raw  top-1: {raw_top.text[:120]}')
print(f'SB   top-1: {sb_hits[0].text[:120]}')
print(f'Same chunk? {raw_top.doc_id == sb_hits[0].doc_id}')

Raw  top-1: Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June

SB   top-1: Uncovering Selective State Space Model’s Capabilities in Lifelong Sequential Recommendation
Conference acronym ’XX, June
Same chunk? True

Inspect — recall@5 on the eval slice¶

Loose recall proxy. Step-back should match or beat raw on reasoning questions; factoid questions tend to tie.

In [10]:

Copied!





from cookbook.corpora import load_eval_questions
qs = [q for q in load_eval_questions() if q['corpus'] == 'arxiv-mamba'][:8]
def recall_fn(fn):
    hits = 0
    for q in qs:
        retr = fn(q['question'])
        gold = [w.lower() for w in q['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in r.text.lower() for w in gold) for r in retr):
            hits += 1
    return hits / max(1, len(qs))
raw = lambda q: store.search(client.embed([q])[0], top_k=5)
sb = lambda q: step_back_retrieve(q, store, client.chat, client.embed, top_k=5)
print(f'raw       recall@5 = {recall_fn(raw):.2f}')
print(f'step-back recall@5 = {recall_fn(sb):.2f}')
from cookbook.corpora import load_eval_questions
qs = [q for q in load_eval_questions() if q['corpus'] == 'arxiv-mamba'][:8]
def recall_fn(fn):
    hits = 0
    for q in qs:
        retr = fn(q['question'])
        gold = [w.lower() for w in q['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in r.text.lower() for w in gold) for r in retr):
            hits += 1
    return hits / max(1, len(qs))
raw = lambda q: store.search(client.embed([q])[0], top_k=5)
sb = lambda q: step_back_retrieve(q, store, client.chat, client.embed, top_k=5)
print(f'raw       recall@5 = {recall_fn(raw):.2f}')
print(f'step-back recall@5 = {recall_fn(sb):.2f}')

raw       recall@5 = 1.00

step-back recall@5 = 1.00

Run It¶

End-to-end on a reasoning question.

In [11]:

Copied!

ans, _ = answer_question('Given selective scan, why would I still want a good initialization for the state matrix?')
print('=== Step-back answer ===')
print(ans)
ans, _ = answer_question('Given selective scan, why would I still want a good initialization for the state matrix?')
print('=== Step-back answer ===')
print(ans)

=== Step-back answer ===
The passages do not contain the answer to the question. They discuss selective state space models and their applications in sequential recommendation, but do not address the specific question of why a good initialization for the state matrix is still desirable given selective scan.

Side by Side: Vanilla Baseline vs This Technique¶

Vanilla vs step-back.

In [12]:

Copied!





from cookbook.baselines import vanilla_pipeline

q = 'Given selective scan, why would I still want a good initialization for the state matrix?'
base = vanilla_pipeline(q, corpus='arxiv-mamba', top_k=5)
ours_a, ours_c = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'step-back', 'preview': ours_a[:160]},
])
from cookbook.baselines import vanilla_pipeline

q = 'Given selective scan, why would I still want a good initialization for the state matrix?'
base = vanilla_pipeline(q, corpus='arxiv-mamba', top_k=5)
ours_a, ours_c = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'step-back', 'preview': ours_a[:160]},
])

Out[12]:

	pipeline	preview
0	vanilla	The passages do not contain the answer to this...
1	step-back	The passages do not contain the answer to the ...

Knobs to Turn¶

Five knobs in priority order:

Abstraction prompt. The whole technique pivots on this. "Ask about the underlying principle" works on textbooks; "ask about the broader category" works on encyclopedias. Test different phrasings on a labelled slice.
Abstractor model. A small fast model is fine. Abstraction is easy and doesn't benefit from frontier-tier reasoning.
Top-k per branch. We use top-k from each. Tune higher (8 each) if RRF is dropping useful chunks; lower if the fused list is too long.
Fuse or replace. Cookbook fuses both branches with RRF. Some implementations replace the raw query with the abstracted one — worse on factoid queries, sometimes better on pure reasoning queries.
Compose inside multi-query (Recipe 14). Step-back is one paraphrase among many. Production stacks often blend HyDE, step-back, and three vanilla paraphrases inside one RRF pool.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [13]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'arxiv-mamba']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'arxiv-mamba']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[13]:

	question	expected	actual	contexts_retrieved
0	What problem do state-space models aim to solv...	State-space models target the quadratic time a...	The passages do not contain a direct compariso...	6
1	Describe the selective scan mechanism introduc...	Selective scan makes the SSM parameters input-...	The passages do not contain a description of a...	6
2	How does Mamba achieve hardware efficiency on ...	Mamba uses a parallel scan implementation with...	The passages do not contain a clear explanatio...	6
3	Which earlier model family does Mamba descend ...	Mamba builds on the structured state-space seq...	The passages do not contain the answer. They m...	6
4	Name two domains beyond text where SSM-style b...	Audio modeling and genomics have both seen suc...	The passages do not contain the answer to the ...	6

Closing Thoughts¶

Three failure modes:

Over-abstraction. "What is electricity?" is too general; it retrieves background that doesn't help. Tighten the prompt to keep the parent question on-topic.
No principle layer in the corpus. If your corpus is all facts, the abstracted query retrieves nothing useful. Step-back wastes a call.
Factoid contamination. Adding principle context to a factoid query sometimes confuses the model. Self-RAG (Recipe 24) filters this out.

Compose with HyDE (Recipe 13) and multi-query (Recipe 14) under one RRF. Compose with sub-question decomposition (Recipe 16) when the question is both abstract and multi-part.