Multi-Query RAG-Fusion — N Rewrites + RRF¶

What problem does this solve?¶

A single query has one perspective. The Meissner-effect chunk that uses the phrase "magnetic flux expulsion" doesn't rank well for a query phrased "how do superconductors push out magnetic fields". They mean the same thing, but embedding cosine doesn't quite agree. Multi-query rewriting asks the LLM to write N paraphrases. Each paraphrase retrieves a slightly different chunk set. Fusing the N rankings with Reciprocal Rank Fusion gives a final ranking that benefits from every paraphrase's strengths. The technique is sometimes called RAG-Fusion; the implementation is twenty lines.

Where it came from¶

Multi-query rewriting predates RAG by years (it's how IR researchers handle query variance) but became a standard RAG technique in early 2024 after Adrian Raudaschl's RAG-Fusion blog post went viral. LlamaIndex and LangChain both shipped multi-query retrievers in that window. The RRF fusion step comes from Cormack et al. 2009 and is what makes the combination robust without per-corpus tuning — sum 1/(k+rank) across rankings, no per-corpus weight learning required. By 2026 multi-query is a default in many production systems, with N=3 to N=5 as common settings. The cost is N+1 retrievals plus one LLM call to produce the paraphrases; the win is a few points of recall across the eval set. Modern implementations parallelise the N retrievals, which makes the pattern essentially free in latency even at N=5.

Where it fits in the RAG landscape¶

Three query-transformation strategies (Recipes 13–17):

HyDE (Recipe 13). One hallucinated answer, embed that.
Multi-query fusion (this recipe). N paraphrases, fuse rankings.
Step-back (Recipe 15). One more-abstract query, retrieve principles.

Composition: multi-query fusion is the chassis; HyDE and step-back can be branches inside it. Many systems run [raw_query, hyde_query, step_back_query] and fuse all three under one RRF. The cookbook factors them as separate recipes for clarity; production stacks blend.

When to use it (and when not to)¶

Use multi-query whenever a single query phrasing isn't enough. Open-domain Q&A, exploratory search, anywhere users phrase questions inconsistently with the corpus's vocabulary. The technique is robust — it rarely hurts. Skip it when your queries are short, structured, and the corpus matches their vocabulary. Type-ahead search over product catalogues doesn't benefit much. Skip it when latency is critical and you cannot parallelise. N+1 retrievals are sequential by default; parallelising helps but adds engineering. RRF needs all rankings before fusing, so it's a barrier in the latency budget.

The intuition¶

Four intuitions to carry with you:

Each paraphrase finds different chunks. A paraphrase that emphasises one keyword retrieves chunks where that keyword dominates. Three paraphrases emphasising different angles cover more of the answer space than the original query alone could.

RRF is parameter-free. Cormack's 1/(k+rank) with k=60 works on every fusion you'll ever do. Don't fiddle with it; the algorithm is robust to k choice in a wide band.

Diminishing returns past N=5. Three paraphrases catch most of the gain. Five is generous. Ten is wasteful — you're paying for retrievals that contribute nothing new because the additional paraphrases overlap heavily.

The rewriter is doing the work. If your rewriter writes lazy paraphrases that just reorder words, fusion gains nothing. Tune the rewriter prompt before tuning N.

Architecture¶

flowchart TB
  Q[Original query] --> RW[LLM:
write N paraphrases]
  RW --> P1[Paraphrase 1]
  RW --> P2[Paraphrase 2]
  RW --> P3[Paraphrase 3]
  Q --> R0[Retrieve top-k]
  P1 --> R1[Retrieve top-k]
  P2 --> R2[Retrieve top-k]
  P3 --> R3[Retrieve top-k]
  R0 --> F[Reciprocal
Rank Fusion]
  R1 --> F
  R2 --> F
  R3 --> F
  F --> G[Top-k fused → LLM]

References¶

📝 RAG-Fusion — A New Approach (Adrian Raudaschl, 2024) — The blog post that popularised the name.
📚 LangChain Multi Query Retriever — Reference implementation.
📚 LlamaIndex Query Transform Cookbook — Multi-query plus several cousins.
📄 Reciprocal Rank Fusion (Cormack et al., 2009) — The fusion technique we use.
📄 HyDE (Recipe 13) — A cousin technique; often composed inside fusion.
📄 Step-back prompting (Recipe 15) — Another query-transformation flavour worth composing.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Build the index¶

Standard Wikipedia superconductors setup.

In [2]:

Copied!





from cookbook.corpora import load_wikipedia_superconductors
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_wikipedia_superconductors())
chunks = sentence_window(docs, sentences_per_chunk=5, overlap=1)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('fusion', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} chunks.')
from cookbook.corpora import load_wikipedia_superconductors
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_wikipedia_superconductors())
chunks = sentence_window(docs, sentences_per_chunk=5, overlap=1)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('fusion', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} chunks.')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Indexed 58 chunks.

Step 2 — Generate N paraphrases¶

One LLM call; ask for N numbered paraphrases. The prompt explicitly asks for different angles so the paraphrases retrieve different chunks.

In [3]:

Copied!





REWRITE_PROMPT = (
    'Write {n} different paraphrases of the question, each phrased to surface a different aspect or use different vocabulary. '
    'Number them, one per line. Do not include any preamble.\n\n'
    'Question: {q}'
)

def fan_out(question: str, n: int = 4) -> list[str]:
    raw = client.chat(REWRITE_PROMPT.format(n=n, q=question))
    rewrites = []
    for line in raw.splitlines():
        line = line.strip()
        if '.' in line[:4]:
            line = line.split('.', 1)[1].strip()
        if line:
            rewrites.append(line)
    return [question] + rewrites[:n]

rewrites = fan_out('How does the Meissner effect distinguish a superconductor from a perfect conductor?', n=4)
for i, r in enumerate(rewrites):
    print(f'  {i}: {r}')
REWRITE_PROMPT = (
    'Write {n} different paraphrases of the question, each phrased to surface a different aspect or use different vocabulary. '
    'Number them, one per line. Do not include any preamble.\n\n'
    'Question: {q}'
)

def fan_out(question: str, n: int = 4) -> list[str]:
    raw = client.chat(REWRITE_PROMPT.format(n=n, q=question))
    rewrites = []
    for line in raw.splitlines():
        line = line.strip()
        if '.' in line[:4]:
            line = line.split('.', 1)[1].strip()
        if line:
            rewrites.append(line)
    return [question] + rewrites[:n]

rewrites = fan_out('How does the Meissner effect distinguish a superconductor from a perfect conductor?', n=4)
for i, r in enumerate(rewrites):
    print(f'  {i}: {r}')

  0: How does the Meissner effect distinguish a superconductor from a perfect conductor?
  1: What sets a superconductor apart from an ideal conductor in terms of the Meissner effect's manifestation?
  2: In what way does the Meissner effect serve as a distinguishing characteristic between superconductors and perfect electrical conductors?
  3: How do the responses of superconducting materials and perfect conductors to magnetic fields differ, as illustrated by the Meissner effect?
  4: What role does the Meissner effect play in differentiating between the properties of a superconductor and those of a theoretically perfect conductor of electricity?

Step 3 — Retrieve for each paraphrase¶

Standard top-10 retrieval, one per paraphrase. We over-retrieve relative to what we'll keep, because RRF promotes consensus and we want enough candidates for the fusion to be meaningful.

In [4]:

Copied!





rankings = [store.search(client.embed([q])[0], top_k=10) for q in rewrites]
print(f'Built {len(rankings)} rankings.')
print('Top-1 chunk for each paraphrase:')
for i, ranking in enumerate(rankings):
    print(f'  {i}: {ranking[0].doc_id}  score={ranking[0].score:.3f}')
rankings = [store.search(client.embed([q])[0], top_k=10) for q in rewrites]
print(f'Built {len(rankings)} rankings.')
print('Top-1 chunk for each paraphrase:')
for i, ranking in enumerate(rankings):
    print(f'  {i}: {ranking[0].doc_id}  score={ranking[0].score:.3f}')

Built 5 rankings.
Top-1 chunk for each paraphrase:
  0: wiki:Meissner_effect#sw0000  score=0.738
  1: wiki:Meissner_effect#sw0000  score=0.687
  2: wiki:Meissner_effect#sw0000  score=0.708
  3: wiki:Meissner_effect#sw0000  score=0.724
  4: wiki:Meissner_effect#sw0000  score=0.677

Step 4 — Reciprocal Rank Fusion¶

We use cookbook.retrievers.reciprocal_rank_fusion. Sums 1/(k+rank) across rankings; the highest-scoring chunks are those that ranked well across multiple paraphrases.

In [5]:

Copied!





from cookbook.retrievers import reciprocal_rank_fusion

fused = reciprocal_rank_fusion(rankings, top_k=5)
print('Fused top-5:')
for h in fused:
    print(f'  score={h.score:.4f}  {h.text[:160]}')
from cookbook.retrievers import reciprocal_rank_fusion

fused = reciprocal_rank_fusion(rankings, top_k=5)
print('Fused top-5:')
for h in fused:
    print(f'  score={h.score:.4f}  {h.text[:160]}')

Fused top-5:
  score=0.0833  # Meissner effect

_Source: Wikipedia, CC BY-SA 4.0_

In condensed-matter physics, the Meissner effect is the expulsion of a magnetic field from a superconducto
  score=0.0820  # Superconductivity

_Source: Wikipedia, CC BY-SA 4.0_

Superconductivity is a set of physical properties observed in superconductors: materials where electrica
  score=0.0804  On a simple 76 millimeter diameter, 1-micrometer thick disk, next to a magnetic field of 28 kA/m, there are approximately 100 billion flux tubes that hold 70,00
  score=0.0789  # Type-I superconductor

_Source: Wikipedia, CC BY-SA 4.0_

The interior of a bulk superconductor cannot be penetrated by a weak magnetic field, a phenomenon kn
  score=0.0786  At a higher critical field Hc2, typically of the order of tens of teslas superconductivity is destroyed. Type-II superconductors do not exhibit a complete Meiss

Step 5 — Wrap as `answer_question`¶

Standard contract. The fan-out and RRF happen inside.

In [6]:

Copied!





PROMPT = (
    'Use only the passages below to answer the question. '
    'If they do not contain the answer, say so plainly.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    queries = fan_out(question, n=4)
    rankings = [store.search(client.embed([q])[0], top_k=10) for q in queries]
    fused = reciprocal_rank_fusion(rankings, top_k=k)
    contexts = [h.text for h in fused]
    return client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question)), contexts

ans, _ = answer_question('What is the role of Cooper pairs in BCS theory?')
print(ans)
PROMPT = (
    'Use only the passages below to answer the question. '
    'If they do not contain the answer, say so plainly.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    queries = fan_out(question, n=4)
    rankings = [store.search(client.embed([q])[0], top_k=10) for q in queries]
    fused = reciprocal_rank_fusion(rankings, top_k=k)
    contexts = [h.text for h in fused]
    return client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question)), contexts

ans, _ = answer_question('What is the role of Cooper pairs in BCS theory?')
print(ans)

In BCS theory, Cooper pairs play the role of pairs of electrons that move through the lattice without resistance, and their condensation is the microscopic effect that causes superconductivity.

Look Inside¶

Inspect — paraphrase quality¶

Read three sets of paraphrases. Good paraphrases use different vocabulary. Bad paraphrases just rearrange words.

In [7]:

Copied!





for q in [
    'How does flux pinning enable Type-II superconductor levitation?',
    'When was YBCO discovered?',
    'What is the connection between BCS theory and the Meissner effect?',
]:
    print(f'Q: {q}')
    for r in fan_out(q, n=3)[1:]:
        print(f'  - {r}')
    print()
for q in [
    'How does flux pinning enable Type-II superconductor levitation?',
    'When was YBCO discovered?',
    'What is the connection between BCS theory and the Meissner effect?',
]:
    print(f'Q: {q}')
    for r in fan_out(q, n=3)[1:]:
        print(f'  - {r}')
    print()

Q: How does flux pinning enable Type-II superconductor levitation?
  - What role does flux pinning play in allowing Type-II superconductors to suspend in mid-air without support.
  - In what ways does the phenomenon of flux pinning contribute to the stable levitation of Type-II superconducting materials.
  - How do pinned magnetic flux lines facilitate the ability of Type-II superconductors to defy gravity and remain suspended.

Q: When was YBCO discovered?
  - What is the discovery date of the chemical compound Yttrium Barium Copper Oxide, commonly referred to as YBCO?
  - In what year did scientists first identify and isolate the high-temperature superconductor known as YBCO?
  - At what point in time was the superconducting material YBCO, composed of yttrium, barium, copper, and oxygen, initially synthesized and characterized?

Q: What is the connection between BCS theory and the Meissner effect?
  - How does the BCS theory of superconductivity relate to the phenomenon of the Meissner effect, where magnetic fields are expelled from a material.
  - In what ways does the Bardeen-Cooper-Schrieffer theory explain the observed behavior of superconducting materials in exhibiting the Meissner effect.
  - What is the underlying link between the microscopic description of superconductivity provided by BCS theory and the macroscopic manifestation of the Meissner effect in superconducting materials.

Inspect — how much do the rankings overlap?¶

If paraphrase rankings agree completely, fusion adds nothing. The interesting case is partial overlap — fusion promotes the chunks that appear in many rankings.

In [8]:

Copied!





q = 'How does the Meissner effect distinguish a superconductor from a perfect conductor?'
qs = fan_out(q, n=4)
rankings = [store.search(client.embed([qq])[0], top_k=5) for qq in qs]
from collections import Counter
top1_ids = Counter(r[0].doc_id for r in rankings)
print('Top-1 distribution across rankings:')
for doc_id, count in top1_ids.most_common():
    print(f'  {count}x: {doc_id}')
q = 'How does the Meissner effect distinguish a superconductor from a perfect conductor?'
qs = fan_out(q, n=4)
rankings = [store.search(client.embed([qq])[0], top_k=5) for qq in qs]
from collections import Counter
top1_ids = Counter(r[0].doc_id for r in rankings)
print('Top-1 distribution across rankings:')
for doc_id, count in top1_ids.most_common():
    print(f'  {count}x: {doc_id}')

Top-1 distribution across rankings:
  5x: wiki:Meissner_effect#sw0000

Inspect — sweep N (number of paraphrases)¶

Diminishing returns past N=4 on most corpora. Measure on the eval slice.

In [9]:

Copied!





from cookbook.corpora import load_eval_questions
qs = [q for q in load_eval_questions() if q['corpus'] == 'wikipedia-superconductors'][:6]
import pandas as pd
rows = []
for n in (1, 2, 3, 4, 5):
    hits = 0
    for q in qs:
        queries = fan_out(q['question'], n=n)
        rankings = [store.search(client.embed([qq])[0], top_k=10) for qq in queries]
        fused = reciprocal_rank_fusion(rankings, top_k=5)
        gold = [w.lower() for w in q['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in h.text.lower() for w in gold) for h in fused):
            hits += 1
    rows.append({'n_paraphrases': n, 'recall@5': hits / max(1, len(qs))})
pd.DataFrame(rows)
from cookbook.corpora import load_eval_questions
qs = [q for q in load_eval_questions() if q['corpus'] == 'wikipedia-superconductors'][:6]
import pandas as pd
rows = []
for n in (1, 2, 3, 4, 5):
    hits = 0
    for q in qs:
        queries = fan_out(q['question'], n=n)
        rankings = [store.search(client.embed([qq])[0], top_k=10) for qq in queries]
        fused = reciprocal_rank_fusion(rankings, top_k=5)
        gold = [w.lower() for w in q['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in h.text.lower() for w in gold) for h in fused):
            hits += 1
    rows.append({'n_paraphrases': n, 'recall@5': hits / max(1, len(qs))})
pd.DataFrame(rows)

Out[9]:

	n_paraphrases	recall@5
0	1	1.0
1	2	1.0
2	3	1.0
3	4	1.0
4	5	1.0

Inspect — cost¶

Fan-out is 1 LLM call + N retrievals. Each retrieval is 1 embedding + 1 search. We measure cache entries before and after one query.

In [10]:

Copied!





from cookbook import _cache
before = _cache.stats()['entries']
_ = answer_question('What is critical temperature?')
after = _cache.stats()['entries']
print(f'New cache entries: {after - before}')
print('Rough breakdown for n=4:')
print('  1 fan-out LLM call')
print('  5 query embeds (raw + 4 paraphrases)')
print('  1 final-answer LLM call')
from cookbook import _cache
before = _cache.stats()['entries']
_ = answer_question('What is critical temperature?')
after = _cache.stats()['entries']
print(f'New cache entries: {after - before}')
print('Rough breakdown for n=4:')
print('  1 fan-out LLM call')
print('  5 query embeds (raw + 4 paraphrases)')
print('  1 final-answer LLM call')

New cache entries: 0
Rough breakdown for n=4:
  1 fan-out LLM call
  5 query embeds (raw + 4 paraphrases)
  1 final-answer LLM call

Run It¶

End-to-end on a representative query.

In [11]:

Copied!

ans, ctxs = answer_question('How does the Meissner effect distinguish a superconductor from a perfect conductor?')
print('=== Multi-query fusion answer ===')
print(ans)
ans, ctxs = answer_question('How does the Meissner effect distinguish a superconductor from a perfect conductor?')
print('=== Multi-query fusion answer ===')
print(ans)

=== Multi-query fusion answer ===
The passages do not contain the answer. They describe the Meissner effect and its relation to superconductors, but they do not explicitly distinguish a superconductor from a perfect conductor using the Meissner effect.

Side by Side: Vanilla Baseline vs This Technique¶

Vanilla baseline (single query) vs multi-query fusion.

In [12]:

Copied!





from cookbook.baselines import vanilla_pipeline

q = 'How does the Meissner effect distinguish a superconductor from a perfect conductor?'
base = vanilla_pipeline(q, corpus='wikipedia-superconductors', top_k=5)
ours_a, ours_c = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'multi-query fusion', 'preview': ours_a[:160]},
])
from cookbook.baselines import vanilla_pipeline

q = 'How does the Meissner effect distinguish a superconductor from a perfect conductor?'
base = vanilla_pipeline(q, corpus='wikipedia-superconductors', top_k=5)
ours_a, ours_c = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'multi-query fusion', 'preview': ours_a[:160]},
])

Out[12]:

	pipeline	preview
0	vanilla	The passages do not contain the answer. They d...
1	multi-query fusion	The passages do not contain the answer. They d...

Knobs to Turn¶

Five knobs in priority order:

N (paraphrase count). 3–4 is the sweet spot. Higher pays diminishing returns and inflates cost. Lower may not cover the query space adequately.
Rewriter prompt. Ask explicitly for different angles. Without this, the model produces near-duplicates that all retrieve the same chunks and waste calls.
Top-k per retrieval. We use 10. Higher catches more recall, slower RRF aggregation. Sweep on a held-out eval to find your sweet spot.
RRF k constant. Default 60. Rarely worth tuning — the algorithm is robust to k in [30, 100].
Rewriter model. A cheap fast model is fine. Paraphrasing is easy; don't waste your best model on this step.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [13]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'wikipedia-superconductors']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'wikipedia-superconductors']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[13]:

	question	expected	actual	contexts_retrieved
0	Who first observed superconductivity, and in w...	Heike Kamerlingh Onnes observed it in mercury ...	The passages provided do not contain the answe...	5
1	What is the Meissner effect?	The complete expulsion of magnetic flux from a...	The Meissner effect is the expulsion of a magn...	5
2	Distinguish Type-I from Type-II superconductors.	Type-I has a single critical field above which...	Type-I and Type-II superconductors can be dist...	5
3	What does BCS theory explain?	It explains conventional superconductivity thr...	BCS theory explains many thermodynamic and ele...	5
4	What is a Cooper pair?	Two electrons bound together by phonon exchang...	A Cooper pair is a pair of electrons bound tog...	5

Closing Thoughts¶

Three failure modes:

Bad paraphrases. The rewriter produces near-duplicates or off-topic rewordings. Re-tune the prompt.
Cost balloon. N=10 with 10 chunks each is 100 retrievals. Parallelise; otherwise latency suffers.
Diminishing returns invisible without measurement. Always sweep N on a held-out eval before committing.

Compose with HyDE (Recipe 13) — one of the paraphrases can be a HyDE hallucination. Compose with step-back (Recipe 15) — another can be an abstracted version. Compose with reranking (Recipe 22) — the fused top-20 is a great input to a cross-encoder.