MCP Tool Retrieval — Retrieving Across Tool Servers¶

What problem does this solve?¶

Production assistants don't retrieve from one corpus — they retrieve from many: an internal wiki, a code search, a customer-data database, a docs site. Each one has its own API. Hard-coding that mesh into the agent breaks the day someone adds another tool. The Model Context Protocol (MCP) standardises how an agent discovers and calls remote tools. For retrieval, MCP servers expose vector stores, knowledge graphs, and search APIs as tools the model can call by name. The agent doesn't know it's talking to Pinecone vs Postgres vs Confluence — just to an MCP tool with a name and a description.

Where it came from¶

MCP was announced by Anthropic in November 2024 and donated to the Linux Foundation in December 2025. The spec describes a JSON-RPC protocol for tool discovery, invocation, and resource subscription. The retrieval pattern is a natural application: expose every searchable store as an MCP tool, let the agent pick which one to call. The standardisation was overdue — every framework had been inventing its own tool-use protocol, and the proliferation was hurting the ecosystem. By 2026 MCP servers exist for most major data stores and APIs — Notion, Slack, Linear, GitHub, the major vector stores. The cookbook mocks the protocol here to keep the notebook deterministic; production MCP setups use the official mcp Python client. The pattern below maps directly to real MCP usage — only the transport layer differs.

Where it fits in the RAG landscape¶

MCP is one of three production tool-use patterns to know:

OpenAI function calling. Native JSON schemas, vendor-bound. Works inside OpenAI's API but not portable.
LangChain Tool / LlamaIndex Tool. Framework-bound abstractions over function calling. Portable across LLM providers but tied to one framework.
MCP (this recipe). Protocol-level standard for tool discovery and use, vendor-agnostic and framework-agnostic.

MCP composes with everything — it's a transport, not a runtime. The agent loop logic still has to live somewhere; LangGraph (Recipe 28) is a good chassis. The protocol layer is for the discovery and invocation, not the orchestration.

When to use it (and when not to)¶

Use MCP when your agent needs to retrieve from multiple distinct stores, and you want a single protocol to handle them. Multi-tool agents, enterprise search assistants, anywhere ad-hoc API integrations would be painful — and where tools come from teams that don't want to maintain framework-specific adapters. Skip it for single-corpus systems. There's no benefit to the protocol layer when you have one store. Skip it when your tools change rarely. Hard-coded integrations are simpler when stable; MCP's value is at the discovery layer, not at the runtime layer.

The intuition¶

Five intuitions to carry:

Tools are named with descriptions. The model picks by reading the descriptions, so descriptions matter as much as code. Tune them.

Tool calls are LLM-driven. The model decides which tool to call and what arguments to pass. This is not a deterministic dispatcher.

Validate before executing. A model can call any tool; validation belongs in the agent loop, not in the LLM call.

Observe everything. Tool calls are the most leveraged points in your stack — instrument them with tracing.

Treat tool output as untrusted. Anything coming back from a tool becomes context for the next LLM call. Don't let it invoke destructive tools.

Architecture¶

flowchart TB
  Q[Query] --> AG[Agent loop]
  AG --> LLM{LLM: pick
tool & args}
  LLM --> T1[MCP tool: arxiv]
  LLM --> T2[MCP tool: rust]
  LLM --> T3[MCP tool: wiki]
  T1 --> AG
  T2 --> AG
  T3 --> AG
  AG --> A[Compose answer]

References¶

📝 Model Context Protocol — Anthropic announcement — Original announcement.
📚 MCP specification — The protocol spec.
💻 MCP Python SDK — Official Python client and server.
📚 LangGraph + MCP tutorial — Composing MCP with stateful agents.
📚 Anthropic agentic coding patterns — Production patterns.
📚 OpenAI function calling — The vendor-bound predecessor.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Build two retrieval stores¶

We'll register them as MCP tools. The cookbook mocks the protocol layer; the rest of the pattern matches real MCP usage.

In [2]:

Copied!





from cookbook.corpora import load_arxiv_mamba, load_rust_book
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

stores = {}
for name, loader in [('arxiv', load_arxiv_mamba), ('rust', load_rust_book)]:
    docs = list(loader())
    chunks = sentence_window(docs, sentences_per_chunk=4)
    vecs = client.embed([c.text for c in chunks])
    s = QdrantBackend(f'mcp-{name}', dim=len(vecs[0]))
    s.add([c.text for c in chunks], vecs, ids=[c.chunk_id for c in chunks])
    stores[name] = s
    print(f'Built tool: search_{name}_corpus ({len(chunks)} chunks)')
from cookbook.corpora import load_arxiv_mamba, load_rust_book
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

stores = {}
for name, loader in [('arxiv', load_arxiv_mamba), ('rust', load_rust_book)]:
    docs = list(loader())
    chunks = sentence_window(docs, sentences_per_chunk=4)
    vecs = client.embed([c.text for c in chunks])
    s = QdrantBackend(f'mcp-{name}', dim=len(vecs[0]))
    s.add([c.text for c in chunks], vecs, ids=[c.chunk_id for c in chunks])
    stores[name] = s
    print(f'Built tool: search_{name}_corpus ({len(chunks)} chunks)')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Built tool: search_arxiv_corpus (81 chunks)

Built tool: search_rust_corpus (173 chunks)

Step 2 — Define the tool registry¶

Each tool has a name and a short description. The model picks by description.

In [3]:

Copied!





TOOLS = [
    {'name': 'search_arxiv_mamba', 'description': 'Search the Mamba state-space-model survey paper.'},
    {'name': 'search_rust_book',   'description': 'Search The Rust Programming Language book.'},
]
print('Tools:')
for t in TOOLS:
    print(f'  - {t["name"]}: {t["description"]}')
TOOLS = [
    {'name': 'search_arxiv_mamba', 'description': 'Search the Mamba state-space-model survey paper.'},
    {'name': 'search_rust_book',   'description': 'Search The Rust Programming Language book.'},
]
print('Tools:')
for t in TOOLS:
    print(f'  - {t["name"]}: {t["description"]}')

Tools:
  - search_arxiv_mamba: Search the Mamba state-space-model survey paper.
  - search_rust_book: Search The Rust Programming Language book.

Step 3 — Build the agent prompt¶

The model is shown the tool list and asked to issue a call.

In [4]:

Copied!





TOOL_PROMPT = (
    'You have access to these tools (call by writing CALL: tool_name("query")):\n{tools}\n\n'
    'Question: {q}\nCall the most appropriate tool first.'
)

def render_tools():
    return '\n'.join(f"- {t['name']}: {t['description']}" for t in TOOLS)

print(TOOL_PROMPT.format(tools=render_tools(), q='What is selective scan?'))
TOOL_PROMPT = (
    'You have access to these tools (call by writing CALL: tool_name("query")):\n{tools}\n\n'
    'Question: {q}\nCall the most appropriate tool first.'
)

def render_tools():
    return '\n'.join(f"- {t['name']}: {t['description']}" for t in TOOLS)

print(TOOL_PROMPT.format(tools=render_tools(), q='What is selective scan?'))

You have access to these tools (call by writing CALL: tool_name("query")):
- search_arxiv_mamba: Search the Mamba state-space-model survey paper.
- search_rust_book: Search The Rust Programming Language book.

Question: What is selective scan?
Call the most appropriate tool first.

Step 4 — Parse and execute tool calls¶

We parse the model's output for CALL: tool_name("query") and route to the right store.

In [5]:

Copied!





import re

def execute_call(tool: str, query: str) -> list[str]:
    key = 'arxiv' if 'arxiv' in tool else 'rust'
    qv = client.embed([query])[0]
    return [h.text for h in stores[key].search(qv, top_k=3)]

def parse_call(text: str) -> tuple[str, str] | None:
    m = re.search(r'CALL:\s*(\w+)\("([^"]+)"\)', text)
    return (m.group(1), m.group(2)) if m else None

raw = client.chat(TOOL_PROMPT.format(tools=render_tools(), q='What is selective scan?'))
print('Model raw output:')
print(raw)
print()
call = parse_call(raw)
print(f'Parsed call: {call}')
import re

def execute_call(tool: str, query: str) -> list[str]:
    key = 'arxiv' if 'arxiv' in tool else 'rust'
    qv = client.embed([query])[0]
    return [h.text for h in stores[key].search(qv, top_k=3)]

def parse_call(text: str) -> tuple[str, str] | None:
    m = re.search(r'CALL:\s*(\w+)\("([^"]+)"\)', text)
    return (m.group(1), m.group(2)) if m else None

raw = client.chat(TOOL_PROMPT.format(tools=render_tools(), q='What is selective scan?'))
print('Model raw output:')
print(raw)
print()
call = parse_call(raw)
print(f'Parsed call: {call}')

Model raw output:
CALL: search_arxiv_mamba("selective scan")

Parsed call: ('search_arxiv_mamba', 'selective scan')

Step 5 — Multi-step agent loop¶

Loop until the model produces an answer or we hit the cap. Each iteration: ask the model what to do, execute the tool, accumulate observations.

In [6]:

Copied!





def mcp_loop(question: str, max_steps: int = 3):
    history = []
    for step in range(max_steps):
        prompt = TOOL_PROMPT.format(tools=render_tools(), q=question)
        if history:
            prompt += '\n\nObservations:\n' + '\n'.join(history)
        out = client.chat(prompt)
        call = parse_call(out)
        if not call:
            return out, history
        tool, q = call
        results = execute_call(tool, q)
        history.append(f'[{tool}({q!r})] -> ' + ' | '.join(r[:120] for r in results))
    final = client.chat(
        'Summarize and answer based on these observations.\n'
        + '\n'.join(history) + f'\nQuestion: {question}'
    )
    return final, history

ans, hist = mcp_loop('In the Mamba paper, what is selective scan, and how would I implement it in Rust given ownership constraints?')
print(ans[:400])
def mcp_loop(question: str, max_steps: int = 3):
    history = []
    for step in range(max_steps):
        prompt = TOOL_PROMPT.format(tools=render_tools(), q=question)
        if history:
            prompt += '\n\nObservations:\n' + '\n'.join(history)
        out = client.chat(prompt)
        call = parse_call(out)
        if not call:
            return out, history
        tool, q = call
        results = execute_call(tool, q)
        history.append(f'[{tool}({q!r})] -> ' + ' | '.join(r[:120] for r in results))
    final = client.chat(
        'Summarize and answer based on these observations.\n'
        + '\n'.join(history) + f'\nQuestion: {question}'
    )
    return final, history

ans, hist = mcp_loop('In the Mamba paper, what is selective scan, and how would I implement it in Rust given ownership constraints?')
print(ans[:400])

Based on the provided observations, it appears that the term "selective scan" is related to the "Vision Mamba" paper, which discusses efficient visual representation learning with a bidirectional state space model. However, the text does not explicitly define "selective scan."

Given the context, it's possible that "selective scan" refers to a technique or algorithm used in the Vision Mamba paper,

Step 6 — Wrap as `answer_question`¶

Cookbook contract.

In [7]:

Copied!

def answer_question(question: str) -> tuple[str, list[str]]:
    return mcp_loop(question)

ans, _ = answer_question('What is selective scan?')
print(ans[:300])
def answer_question(question: str) -> tuple[str, list[str]]:
    return mcp_loop(question)

ans, _ = answer_question('What is selective scan?')
print(ans[:300])

Based on the observations, it appears that "selective scan" is a term related to various fields, including computer vision, recommendation systems, and programming. 

In the context of computer vision, "selective scan" is mentioned in the paper "Vision Mamba: Efficient Visual Representation Learning

Look Inside¶

Inspect — which tool gets picked for various questions?¶

Test the model's tool selection across a battery.

In [8]:

Copied!





for q in [
    'What is selective scan?',                       # arxiv
    'When should I use Arc over Rc?',                # rust
    'How do state-space models scale linearly?',     # arxiv
    'Walk me through borrow checking.',              # rust
]:
    raw = client.chat(TOOL_PROMPT.format(tools=render_tools(), q=q))
    call = parse_call(raw)
    tool = call[0] if call else '(no call)'
    print(f'  {q[:55]:55s} -> {tool}')
for q in [
    'What is selective scan?',                       # arxiv
    'When should I use Arc over Rc?',                # rust
    'How do state-space models scale linearly?',     # arxiv
    'Walk me through borrow checking.',              # rust
]:
    raw = client.chat(TOOL_PROMPT.format(tools=render_tools(), q=q))
    call = parse_call(raw)
    tool = call[0] if call else '(no call)'
    print(f'  {q[:55]:55s} -> {tool}')

  What is selective scan?                                 -> search_arxiv_mamba
  When should I use Arc over Rc?                          -> search_rust_book
  How do state-space models scale linearly?               -> search_arxiv_mamba
  Walk me through borrow checking.                        -> search_rust_book

Inspect — what happens when no tool fits?¶

Ask a question outside both corpora. The model should either pick the closest tool or refuse.

In [9]:

Copied!

raw = client.chat(TOOL_PROMPT.format(tools=render_tools(), q='What is the capital of France?'))
print(raw)
print(f'Parsed: {parse_call(raw)}')
raw = client.chat(TOOL_PROMPT.format(tools=render_tools(), q='What is the capital of France?'))
print(raw)
print(f'Parsed: {parse_call(raw)}')

Neither of the available tools seems directly related to general knowledge or geography. However, I'll start by trying to find any mention of the capital of France in the Mamba state-space-model survey paper, as it's possible that the paper mentions it in a context unrelated to its primary topic.

CALL: search_arxiv_mamba("capital of France")
Parsed: ('search_arxiv_mamba', 'capital of France')

Inspect — full agent loop trace¶

Print every step of the loop for one cross-corpus question. Useful for debugging the agent's reasoning.

In [10]:

Copied!





ans, hist = mcp_loop('In the Mamba paper, what is selective scan, and how would I implement it in Rust given ownership constraints?')
print('=== Agent trace ===')
for step, obs in enumerate(hist):
    print(f'Step {step+1}: {obs[:200]}')
    print()
print('=== Final answer ===')
print(ans[:400])
ans, hist = mcp_loop('In the Mamba paper, what is selective scan, and how would I implement it in Rust given ownership constraints?')
print('=== Agent trace ===')
for step, obs in enumerate(hist):
    print(f'Step {step+1}: {obs[:200]}')
    print()
print('=== Final answer ===')
print(ans[:400])

=== Agent trace ===
Step 1: [search_arxiv_mamba('selective scan')] -> 2024. Vision mamba: Efficient visual representation learning
with bidirectional state space model. arXiv preprint arXiv: | Association for Computing Machinery

Step 2: [search_arxiv_mamba('selective scan')] -> 2024. Vision mamba: Efficient visual representation learning
with bidirectional state space model. arXiv preprint arXiv: | Association for Computing Machinery

Step 3: [search_arxiv_mamba('selective scan definition')] -> 2024. Vision mamba: Efficient visual representation learning
with bidirectional state space model. arXiv preprint arXiv: | Association for Computin

=== Final answer ===
Based on the provided observations, it appears that the term "selective scan" is related to the "Vision Mamba" paper, which discusses efficient visual representation learning with a bidirectional state space model. However, the text does not explicitly define "selective scan."

Given the context, it's possible that "selective scan" refers to a technique or algorithm used in the Vision Mamba paper,

Inspect — cost¶

Each agent step is one LLM call. Track them.

In [11]:

Copied!





from cookbook import _cache
before = _cache.stats()['entries']
_ = answer_question('Explain Rc.')
after = _cache.stats()['entries']
print(f'New cache entries: {after - before}')
print('Each step: 1 LLM call + 1 query embed + 1 vector search.')
from cookbook import _cache
before = _cache.stats()['entries']
_ = answer_question('Explain Rc.')
after = _cache.stats()['entries']
print(f'New cache entries: {after - before}')
print('Each step: 1 LLM call + 1 query embed + 1 vector search.')

New cache entries: 0
Each step: 1 LLM call + 1 query embed + 1 vector search.

Run It¶

End-to-end on a cross-corpus question.

In [12]:

Copied!

ans, _ = answer_question('How would I write selective scan in idiomatic Rust given borrow-checker constraints?')
print('=== MCP-routed answer ===')
print(ans[:400])
ans, _ = answer_question('How would I write selective scan in idiomatic Rust given borrow-checker constraints?')
print('=== MCP-routed answer ===')
print(ans[:400])

=== MCP-routed answer ===
To write a selective scan in idiomatic Rust while navigating borrow-checker constraints, it's essential to understand how Rust's ownership and borrowing system works, as it directly impacts how you can implement such functionality. The borrow checker ensures that references to data are always valid, preventing common errors like null or dangling pointers.

Given the context of your question, let's

Side by Side: Vanilla Baseline vs This Technique¶

Vanilla (single corpus) vs MCP (router across corpora). Vanilla can only see one corpus; MCP picks.

In [13]:

Copied!





from cookbook.baselines import vanilla_pipeline

q = 'What is selective scan in Mamba?'
base = vanilla_pipeline(q, corpus='rust-book', top_k=5)  # wrong corpus on purpose
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla (rust only)', 'preview': base.answer[:160]},
    {'pipeline': 'mcp routed', 'preview': ours_a[:160]},
])
from cookbook.baselines import vanilla_pipeline

q = 'What is selective scan in Mamba?'
base = vanilla_pipeline(q, corpus='rust-book', top_k=5)  # wrong corpus on purpose
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla (rust only)', 'preview': base.answer[:160]},
    {'pipeline': 'mcp routed', 'preview': ours_a[:160]},
])

Out[13]:

	pipeline	preview
0	vanilla (rust only)	The passages do not contain the answer. There ...
1	mcp routed	Based on the provided observations, it appears...

Knobs to Turn¶

Six knobs in priority order:

Tool descriptions. The most important lever. Vague descriptions cause mis-routing; specific descriptions with example queries route well.
Tool list size. Long lists confuse the model. Cap at ~10 visible tools per call. For larger tool sets, route to a tool-list first (a tool that finds tools).
Loop cap. Default 3. Cap aggressively to prevent runaway agents and unbounded LLM bills.
Argument parsing. Validate before executing. Models will pass bad arguments; treat tool input like any other user input.
Real MCP client. Swap the mock for the mcp library in production. The cookbook pattern matches MCP usage.
Authentication. Many MCP servers require authentication. Plan for credential management before going to production.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [14]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[14]:

	question	expected	actual	contexts_retrieved
0	What problem do state-space models aim to solv...	State-space models target the quadratic time a...	Based on the search results, it appears that s...	3
1	Describe the selective scan mechanism introduc...	Selective scan makes the SSM parameters input-...	Based on the provided observations, it appears...	3
2	How does Mamba achieve hardware efficiency on ...	Mamba uses a parallel scan implementation with...	Based on the observations, Mamba achieves hard...	3
3	Which earlier model family does Mamba descend ...	Mamba builds on the structured state-space seq...	Based on the provided observations, there is n...	3
4	Name two domains beyond text where SSM-style b...	Audio modeling and genomics have both seen suc...	Two domains beyond text where SSM-style backbo...	2

Closing Thoughts¶

Four failure modes you'll meet:

Tool description drift. As you add tools, descriptions get vague. Refactor periodically; treat tool descriptions like API documentation.
Hallucinated tool names. The model may invent tool names. Validate against the registry before executing.
Loop runaway. Without the cap the agent loops forever. Cap.
Prompt-injection via tool output. Tool outputs go into the model's context. Treat them as untrusted input; never let them invoke destructive tools without explicit confirmation.

Compose with LangGraph (Recipe 28) for the loop logic. Compose with prompt caching for the tool registry — it's identical across queries and is the largest single block in the prompt.