Cross-Encoder Reranking — The Default Production Quality Lift¶

What problem does this solve?¶

Bi-encoders (the embedding models we've used in every recipe so far) score queries and chunks independently. The query's vector is produced without seeing any chunk; each chunk's vector is produced without seeing the query. The model never gets to ask "how well does this query line up against this chunk?" — it only computes their vectors separately and compares with cosine. Cross-encoders score query and chunk jointly. The transformer sees the concatenated (query, chunk) pair and outputs a single relevance score. Joint scoring catches signals bi-encoders miss — semantic mismatches, contradictions, partial-match nuances. The trade-off is speed: cross-encoders can't precompute an index. The pattern: shortlist with a fast bi-encoder, rerank the top-N with a slow cross-encoder.

Where it came from¶

Cross-encoder reranking has been a standard IR technique since the late 2010s; sentence-transformers shipped CrossEncoder in 2019. The RAG community adopted it from the start. By 2026 the leaders are bge-reranker-v2-m3 (open weights), Cohere Rerank 4 (hosted), Voyage Rerank 2.5 (hosted), and Jina Reranker v3 (listwise variant). The technique has stayed remarkably stable for five years because the fundamental shape — joint scoring of (query, chunk) pairs — is what makes it work. Most innovation since 2020 has been in training data (more diverse domains) and architecture (smaller distilled models for cheaper inference). This notebook ships with an LLM-based pointwise scorer so it runs without a heavy model download. The shape — pointwise (query, chunk) scoring then re-sort — is identical to a real cross-encoder. Production teams swap cross_encoder_rerank() to call sentence-transformers.CrossEncoder with BAAI/bge-reranker-v2-m3 or use a hosted reranker like Cohere Rerank; the surrounding pipeline does not change.

Where it fits in the RAG landscape¶

Reranking sits between retrieval and generation. Three levels of expressivity:

Bi-encoder dense (Recipe 2). Fastest. Index-friendly. Cosine over precomputed vectors.
Cross-encoder (this recipe). Slower. Per-query. Joint (query, chunk) scoring.
LLM-as-reranker (Recipe 23). Slowest. Most flexible. Reorders a candidate list as one task.

Production patterns: shortlist with bi-encoder, rerank with cross-encoder, optionally apply LLM rerank on top-5. Each stage cuts the search space and adds a layer of quality.

When to use it (and when not to)¶

Use cross-encoder reranking on any production RAG system where quality matters more than latency. The pattern adds 50-150 ms of latency and a few cents of cost per query for a 5-15 point recall@5 improvement on hard queries — a near-universal win. Skip it when the shortlist is already perfect. A FAQ Q&A system with crisp questions and clear answers may not benefit from another layer. Skip it when latency is hard-capped under 200 ms. Cross-encoders are not free; budget them carefully and consider a distilled model if latency is tight.

The intuition¶

Four intuitions:

Joint scoring sees nuance. A bi-encoder represents query and chunk as separate points in vector space. A cross-encoder reads them together. Joint reading lets the model notice that a chunk mentions the query's entity but contradicts the question, or matches the topic but answers a different sub-question.

Cross-encoders can't be indexed. Every (query, chunk) pair must be scored at query time. That's why the shortlist+rerank pattern exists — you can't run a cross-encoder over a billion chunks.

Shortlist size is the latency knob. Reranking 20 candidates is fast. Reranking 500 is slow. Tune the shortlist to fit your latency budget.

Rerankers don't generate. A reranker only ranks; it never produces text. That's why they can be small, distilled, and fast. Don't reach for a frontier model when bge-reranker-v2-m3 suffices.

Architecture¶

flowchart LR
  Q[Query] --> BE[Bi-encoder
retrieval]
  BE --> SL[Shortlist top-N
e.g. 30]
  SL --> CE[Cross-encoder
scores each pair]
  CE --> R[Reordered top-k
e.g. 5]
  R --> G[Generator]

References¶

📚 Sentence-Transformers CrossEncoder documentation — The canonical Python implementation.
💻 BAAI BGE Reranker v2-m3 — Open-weight leader; we use this model.
📚 Cohere Rerank documentation — Hosted leader on quality.
📝 Jina Reranker v3 — Listwise variant; cousin to Recipe 23.
📝 Pinecone reranking guide — Production-perspective overview.
📝 Anthropic Contextual Retrieval — The contextual-chunks + BM25 + reranker stack relies heavily on this step.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Build the index¶

Standard setup on the Mamba paper. We need a working bi-encoder index to shortlist from.

In [2]:

Copied!





from cookbook.corpora import load_arxiv_mamba
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_arxiv_mamba())
chunks = sentence_window(docs, sentences_per_chunk=4)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('rerank', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} chunks.')
from cookbook.corpora import load_arxiv_mamba
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_arxiv_mamba())
chunks = sentence_window(docs, sentences_per_chunk=4)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('rerank', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} chunks.')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Indexed 81 chunks.

Step 2 — Shortlist with the bi-encoder¶

Standard dense retrieval to get a candidate pool of 30. The shortlist size is the main latency knob.

In [3]:

Copied!





q = 'Why is the parallel scan implementation crucial for Mamba on GPUs?'
qv = client.embed([q])[0]
shortlist = store.search(qv, top_k=30)
print(f'Shortlisted {len(shortlist)} candidates.')
print('Top-3 by bi-encoder:')
for h in shortlist[:3]:
    print(f'  {h.score:.3f}  {h.text[:140]}')
q = 'Why is the parallel scan implementation crucial for Mamba on GPUs?'
qv = client.embed([q])[0]
shortlist = store.search(qv, top_k=30)
print(f'Shortlisted {len(shortlist)} candidates.')
print('Top-3 by bi-encoder:')
for h in shortlist[:3]:
    print(f'  {h.score:.3f}  {h.text[:140]}')

Shortlisted 30 candidates.
Top-3 by bi-encoder:
  0.608  Com-
pared with prior SSMs, Mamba introduces a data-dependence se-
lection mechanism and utilizes a parallel algorithm optimized for
hardwar
  0.597  This advan-
tage becomes more pronounced as the sequence length increases. To conclude, the experimental findings provide compelling evi-
de
  0.590  We can observe that RecMamba notably reduces GPU memory
footprint and significantly slashes both inference and training times. Compared with

Step 3 — Rerank with the cross-encoder¶

cookbook.rerankers.cross_encoder_rerank loads the cross-encoder model, scores each (query, chunk) pair, returns the reordered list.

In [4]:

Copied!





from cookbook.rerankers import cross_encoder_rerank

reranked = cross_encoder_rerank(q, shortlist, top_k=5, chat=client.chat)
print('Top-5 after cross-encoder rerank:')
for h in reranked:
    print(f'  {h.score:.3f}  {h.text[:140]}')
from cookbook.rerankers import cross_encoder_rerank

reranked = cross_encoder_rerank(q, shortlist, top_k=5, chat=client.chat)
print('Top-5 after cross-encoder rerank:')
for h in reranked:
    print(f'  {h.score:.3f}  {h.text[:140]}')

Top-5 after cross-encoder rerank:
  0.800  More specifically, RecMamba achieves compa-
rable performance with the representative model SASRec while
greatly reducing about 70% training
  0.000  Com-
pared with prior SSMs, Mamba introduces a data-dependence se-
lection mechanism and utilizes a parallel algorithm optimized for
hardwar
  0.000  This advan-
tage becomes more pronounced as the sequence length increases. To conclude, the experimental findings provide compelling evi-
de
  0.000  We can observe that RecMamba notably reduces GPU memory
footprint and significantly slashes both inference and training times. Compared with
  0.000  2024. Vision mamba: Efficient visual representation learning
with bidirectional state space model. arXiv preprint arXiv:2401.09417 (2024).

Step 4 — Wrap as `answer_question`¶

Standard contract. Shortlist + rerank inside.

In [5]:

Copied!





PROMPT = (
    'Use only the passages below to answer the question.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    short = store.search(qv, top_k=30)
    top = cross_encoder_rerank(question, short, top_k=k, chat=client.chat)
    contexts = [h.text for h in top]
    return client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question)), contexts

ans, _ = answer_question('Why is HiPPO initialization useful for state-space models?')
print(ans)
PROMPT = (
    'Use only the passages below to answer the question.\n\n'
    'Passages:\n{context}\n\nQuestion: {question}\nAnswer:'
)

def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    short = store.search(qv, top_k=30)
    top = cross_encoder_rerank(question, short, top_k=k, chat=client.chat)
    contexts = [h.text for h in top]
    return client.chat(PROMPT.format(context='\n\n'.join(contexts), question=question)), contexts

ans, _ = answer_question('Why is HiPPO initialization useful for state-space models?')
print(ans)

There is no mention of HiPPO initialization in the provided passages. The passages do mention that the trainable parameters were initialized using Xavier normal distribution, but they do not discuss HiPPO initialization. Therefore, it is not possible to provide an answer to the question based on the provided passages.

Step 5 — Compare bi-encoder top-5 to reranked top-5¶

On hard queries the order changes substantially. Note which chunks moved.

In [6]:

Copied!





q = 'Why is the parallel scan implementation crucial for Mamba on GPUs?'
qv = client.embed([q])[0]
shortlist = store.search(qv, top_k=30)
reranked = cross_encoder_rerank(q, shortlist, top_k=5, chat=client.chat)
shortlist_ids = [h.doc_id for h in shortlist[:5]]
reranked_ids = [h.doc_id for h in reranked]
print(f'Bi-encoder top-5 ids: {shortlist_ids}')
print(f'Reranked   top-5 ids: {reranked_ids}')
moved_in = set(reranked_ids) - set(shortlist_ids)
moved_out = set(shortlist_ids) - set(reranked_ids)
print(f'Moved into top-5 by rerank: {moved_in}')
print(f'Moved out of top-5 by rerank: {moved_out}')
q = 'Why is the parallel scan implementation crucial for Mamba on GPUs?'
qv = client.embed([q])[0]
shortlist = store.search(qv, top_k=30)
reranked = cross_encoder_rerank(q, shortlist, top_k=5, chat=client.chat)
shortlist_ids = [h.doc_id for h in shortlist[:5]]
reranked_ids = [h.doc_id for h in reranked]
print(f'Bi-encoder top-5 ids: {shortlist_ids}')
print(f'Reranked   top-5 ids: {reranked_ids}')
moved_in = set(reranked_ids) - set(shortlist_ids)
moved_out = set(shortlist_ids) - set(reranked_ids)
print(f'Moved into top-5 by rerank: {moved_in}')
print(f'Moved out of top-5 by rerank: {moved_out}')

Bi-encoder top-5 ids: ['arxiv:2403-mamba-survey#p2#sw0006', 'arxiv:2403-mamba-survey#p4#sw0003', 'arxiv:2403-mamba-survey#p3#sw0011', 'arxiv:2403-mamba-survey#p5#sw0036', 'arxiv:2403-mamba-survey#p2#sw0005']
Reranked   top-5 ids: ['arxiv:2403-mamba-survey#p2#sw0005', 'arxiv:2403-mamba-survey#p2#sw0006', 'arxiv:2403-mamba-survey#p4#sw0003', 'arxiv:2403-mamba-survey#p3#sw0011', 'arxiv:2403-mamba-survey#p5#sw0036']
Moved into top-5 by rerank: set()
Moved out of top-5 by rerank: set()

Look Inside¶

Inspect — how do scores compare?¶

Bi-encoder scores are cosine similarities (0-1 typically). Cross-encoder scores are logits — different scale. Both are monotonic; ranks tell you everything.

In [7]:

Copied!





import pandas as pd
rows = []
for i, h in enumerate(shortlist[:10]):
    bi_score = h.score
    rows.append({'bi_rank': i+1, 'bi_score': bi_score, 'doc_id': h.doc_id[:20]})

rerank_top10 = cross_encoder_rerank(q, shortlist, top_k=10, chat=client.chat)
ce_scores = {h.doc_id: h.score for h in rerank_top10}

for r in rows:
    r['ce_score'] = ce_scores.get(r['doc_id'], None)
pd.DataFrame(rows)
import pandas as pd
rows = []
for i, h in enumerate(shortlist[:10]):
    bi_score = h.score
    rows.append({'bi_rank': i+1, 'bi_score': bi_score, 'doc_id': h.doc_id[:20]})

rerank_top10 = cross_encoder_rerank(q, shortlist, top_k=10, chat=client.chat)
ce_scores = {h.doc_id: h.score for h in rerank_top10}

for r in rows:
    r['ce_score'] = ce_scores.get(r['doc_id'], None)
pd.DataFrame(rows)

Out[7]:

	bi_rank	bi_score	doc_id	ce_score
0	1	0.608029	arxiv:2403-mamba-sur	None
1	2	0.597010	arxiv:2403-mamba-sur	None
2	3	0.589831	arxiv:2403-mamba-sur	None
3	4	0.576473	arxiv:2403-mamba-sur	None
4	5	0.575717	arxiv:2403-mamba-sur	None
5	6	0.553402	arxiv:2403-mamba-sur	None
6	7	0.544101	arxiv:2403-mamba-sur	None
7	8	0.542920	arxiv:2403-mamba-sur	None
8	9	0.539521	arxiv:2403-mamba-sur	None
9	10	0.539034	arxiv:2403-mamba-sur	None

Inspect — what's the latency cost?¶

Measure how long the cross-encoder rerank takes on a 30-candidate shortlist.

In [8]:

Copied!





import time
t0 = time.perf_counter()
_ = cross_encoder_rerank(q, shortlist, top_k=5, chat=client.chat)
ce_ms = (time.perf_counter() - t0) * 1000

t0 = time.perf_counter()
_ = store.search(qv, top_k=30)
be_ms = (time.perf_counter() - t0) * 1000

print(f'Bi-encoder retrieval (30 hits): {be_ms:.1f} ms')
print(f'Cross-encoder rerank   (30 pairs): {ce_ms:.1f} ms')
import time
t0 = time.perf_counter()
_ = cross_encoder_rerank(q, shortlist, top_k=5, chat=client.chat)
ce_ms = (time.perf_counter() - t0) * 1000

t0 = time.perf_counter()
_ = store.search(qv, top_k=30)
be_ms = (time.perf_counter() - t0) * 1000

print(f'Bi-encoder retrieval (30 hits): {be_ms:.1f} ms')
print(f'Cross-encoder rerank   (30 pairs): {ce_ms:.1f} ms')

Bi-encoder retrieval (30 hits): 4.9 ms
Cross-encoder rerank   (30 pairs): 5.0 ms

Inspect — sweep shortlist size¶

Bigger shortlist gives the cross-encoder more material but slows reranking linearly. Sweep.

In [9]:

Copied!





import time, pandas as pd
rows = []
for n in (10, 20, 40, 80):
    short = store.search(qv, top_k=n)
    t0 = time.perf_counter()
    top = cross_encoder_rerank(q, short, top_k=5, chat=client.chat)
    dt = (time.perf_counter() - t0) * 1000
    rows.append({'shortlist_size': n, 'rerank_ms': round(dt, 1), 'top1_doc_id': top[0].doc_id[:30]})
pd.DataFrame(rows)
import time, pandas as pd
rows = []
for n in (10, 20, 40, 80):
    short = store.search(qv, top_k=n)
    t0 = time.perf_counter()
    top = cross_encoder_rerank(q, short, top_k=5, chat=client.chat)
    dt = (time.perf_counter() - t0) * 1000
    rows.append({'shortlist_size': n, 'rerank_ms': round(dt, 1), 'top1_doc_id': top[0].doc_id[:30]})
pd.DataFrame(rows)

Out[9]:

	shortlist_size	rerank_ms	top1_doc_id
0	10	1.7	arxiv:2403-mamba-survey#p2#sw0
1	20	3.2	arxiv:2403-mamba-survey#p2#sw0
2	40	6.0	arxiv:2403-mamba-survey#p2#sw0
3	80	12.5	arxiv:2403-mamba-survey#p2#sw0

Inspect — recall@5 on the eval slice¶

Loose recall proxy comparing bi-encoder-only vs reranked.

In [10]:

Copied!





from cookbook.corpora import load_eval_questions
qs = [q for q in load_eval_questions() if q['corpus'] == 'arxiv-mamba'][:8]
def recall_fn(retr_fn):
    hits = 0
    for row in qs:
        retr = retr_fn(row['question'])
        gold = [w.lower() for w in row['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in r.text.lower() for w in gold) for r in retr):
            hits += 1
    return hits / max(1, len(qs))
be = lambda question: store.search(client.embed([question])[0], top_k=5)
ce = lambda question: cross_encoder_rerank(question, store.search(client.embed([question])[0], top_k=30), top_k=5)
print(f'bi-encoder recall@5 = {recall_fn(be):.2f}')
print(f'reranked   recall@5 = {recall_fn(ce):.2f}')
from cookbook.corpora import load_eval_questions
qs = [q for q in load_eval_questions() if q['corpus'] == 'arxiv-mamba'][:8]
def recall_fn(retr_fn):
    hits = 0
    for row in qs:
        retr = retr_fn(row['question'])
        gold = [w.lower() for w in row['answer'].split() if len(w) >= 4]
        if any(any(w[:6] in r.text.lower() for w in gold) for r in retr):
            hits += 1
    return hits / max(1, len(qs))
be = lambda question: store.search(client.embed([question])[0], top_k=5)
ce = lambda question: cross_encoder_rerank(question, store.search(client.embed([question])[0], top_k=30), top_k=5)
print(f'bi-encoder recall@5 = {recall_fn(be):.2f}')
print(f'reranked   recall@5 = {recall_fn(ce):.2f}')

bi-encoder recall@5 = 1.00

17:51:28 - LiteLLM:WARNING: common_utils.py:979 - litellm: could not pre-load bedrock-runtime response stream shape — Bedrock event-stream decoding will be unavailable. Error: No module named 'botocore'

17:51:28 - LiteLLM:WARNING: common_utils.py:24 - litellm: could not pre-load sagemaker-runtime response stream shape — SageMaker event-stream decoding will be unavailable. Error: No module named 'botocore'

reranked   recall@5 = 1.00

Run It¶

End-to-end on a representative question.

In [11]:

Copied!





q = 'In plain language, what does selective scan do that earlier SSMs could not?'
ans, _ = answer_question(q)
print('=== Reranked answer ===')
print(ans)
q = 'In plain language, what does selective scan do that earlier SSMs could not?'
ans, _ = answer_question(q)
print('=== Reranked answer ===')
print(ans)

=== Reranked answer ===
According to the passage, the selective mechanism in Mamba (a type of selective state space model) introduces a "data-dependence selection mechanism". In simpler terms, this means that Mamba can selectively focus on the most relevant parts of the data, whereas earlier State Space Models (SSMs) could not. This allows Mamba to efficiently handle long sequences and capture important dependencies in the data.

Side by Side: Vanilla Baseline vs This Technique¶

Vanilla baseline (bi-encoder only) vs reranked.

In [12]:

Copied!





from cookbook.baselines import vanilla_pipeline

q = 'In plain language, what does selective scan do that earlier SSMs could not?'
base = vanilla_pipeline(q, corpus='arxiv-mamba', top_k=5)
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'reranked', 'preview': ours_a[:160]},
])
from cookbook.baselines import vanilla_pipeline

q = 'In plain language, what does selective scan do that earlier SSMs could not?'
base = vanilla_pipeline(q, corpus='arxiv-mamba', top_k=5)
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'reranked', 'preview': ours_a[:160]},
])

Out[12]:

	pipeline	preview
0	vanilla	The passages do not contain a clear explanatio...
1	reranked	According to the passage, the selective mechan...

Knobs to Turn¶

Six knobs in priority order:

Shortlist size. Default 30. Larger improves recall at linear latency cost.
Reranker model. bge-reranker-v2-m3 is the open-weight default. Cohere Rerank 4 and Voyage Rerank 2.5 are stronger hosted options.
Where to put the rerank. After dense, after hybrid, or as a fall-back when dense top-1 score is below a confidence threshold.
Cache reranker scores. When you reuse a query, the rerank scores are cached automatically; treat cache size like an embedding cache.
GPU for the reranker. A modest GPU (T4 / L4) keeps rerank latency under 50ms for 100 pairs in production.
Chunk-length cap. Reranker latency scales with chunk length. Cap chunks at ~512 tokens to keep latency predictable across queries.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [13]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'arxiv-mamba']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'arxiv-mamba']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[13]:

	question	expected	actual	contexts_retrieved
0	What problem do state-space models aim to solv...	State-space models target the quadratic time a...	State-space models, such as Mamba, aim to solv...	5
1	Describe the selective scan mechanism introduc...	Selective scan makes the SSM parameters input-...	The passages do not describe the selective sca...	5
2	How does Mamba achieve hardware efficiency on ...	Mamba uses a parallel scan implementation with...	Mamba achieves hardware efficiency on modern G...	5
3	Which earlier model family does Mamba descend ...	Mamba builds on the structured state-space seq...	The passages provided do not explicitly state ...	5
4	Name two domains beyond text where SSM-style b...	Audio modeling and genomics have both seen suc...	Based on the provided passages, two domains be...	5

Closing Thoughts¶

Three failure modes:

Shortlist misses the answer. The cross-encoder can only rerank what the bi-encoder gave it. Use HyDE (Recipe 13) or multi-query fusion (Recipe 14) upstream to widen the funnel.
Reranker out-of-domain. A reranker trained on web QA may underperform on legal or medical text. Pick a domain-appropriate model when one exists.
Latency spike on long chunks. Reranker latency scales with chunk length. Cap chunk size at ~512 tokens to keep latency predictable.

Compose with hybrid (Recipe 18): hybrid produces the shortlist, cross-encoder picks the top-k. Compose with MMR (Recipe 21): rerank picks the most relevant top-N, MMR picks a diverse subset.