Lynx Guardrails — Block Hallucinated Answers Before They Ship¶

What problem does this solve?¶

RAG systems hallucinate. Even with good retrieval, the LLM occasionally fabricates details, mixes up entities, or asserts claims the passages didn't make. In low-stakes UIs that's annoying; in customer-facing or regulated contexts it's a real liability. Guardrails sit between the LLM and the user. They read the answer and the contexts, score how grounded the answer is, and block ungrounded answers. Patronus Lynx is the open-weight small model trained specifically for this scoring task. The cookbook implements a Lynx-style judge using a small fast LLM; production teams swap in the real Lynx model.

Where it came from¶

Patronus AI released Lynx in mid-2024 as a small (~8B) open-weight model fine-tuned for hallucination detection. The training data came from HaluBench and similar grounding benchmarks. The model outputs a binary grounded/not-grounded score and is fast enough for production gates without disturbing latency budgets. By 2026 hallucination guardrails are table stakes for production RAG. Lynx is the open-source baseline; commercial options include NeMo Guardrails (NVIDIA), Guardrails AI, and bespoke fine-tuned models. The pattern below is universal: judge the answer against the context, block on low grounding, and refuse with a useful message rather than ship the hallucination.

Where it fits in the RAG landscape¶

Three layers of safety in production RAG that cookbook recipes cover:

Lynx guardrails (this recipe). Grounding check on the answer just before it ships.
NeMo Guardrails / Guardrails AI. Content moderation, structured output validation, prompt-injection detection.
Self-RAG / CRAG (Recipes 24, 25). Upstream filtering of passages before generation.

All three compose. Lynx is the final gate; Self-RAG and CRAG prevent bad inputs from reaching it; NeMo / Guardrails AI handle modalities Lynx doesn't (PII, prompt injection, schema validation). Production-grade safety stacks all three.

When to use it (and when not to)¶

Use guardrails in any production RAG that customers, regulators, or downstream automation will see. The cost is one extra small-LLM call per answer; the benefit is catching the worst failure mode in production RAG. Skip it for internal exploratory tools. The guardrail latency is wasted. Skip it when grounding isn't the main concern. Some content-moderation use cases want different guardrails (PII detection, prompt injection).

The intuition¶

Four intuitions:

Judging is cheaper than generating. A small fast LLM scoring "is this grounded?" is much cheaper than the answer LLM. Even at high volume, the guardrail layer is affordable.

The judge needs both answer and context. Without context, the judge can't tell what's grounded. The cookbook pattern passes both.

Refuse-on-fail. When the guardrail fails, refuse with a fixed message rather than ship the answer. "I don't have enough evidence to answer confidently" is better than a hallucinated answer.

Tune for false positives. A too-strict guardrail refuses correct answers. Calibrate on a labelled grounding set.

Architecture¶

flowchart LR
  Q[Question] --> R[Retrieve]
  R --> A[Answer LLM]
  A --> G[Guardrail
Lynx judge]
  R --> G
  G --> D{Grounded?}
  D -->|yes| OUT[Ship answer]
  D -->|no| REF[Refuse politely]

References¶

💻 Patronus Lynx model card — The open-weight Lynx model.
💻 HaluBench benchmark — The benchmark Lynx was trained for.
💻 NeMo Guardrails — NVIDIA's alternative.
📚 Guardrails AI — Validation-focused guardrail framework.
📄 Self-RAG (Recipe 24) — Upstream filter that composes with Lynx.
📝 Anthropic Responsible Scaling Policy — Production-safety context.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Build a pipeline¶

Standard Wikipedia superconductors RAG. We'll wrap it with Lynx-style guardrails.

In [2]:

Copied!





from cookbook.corpora import load_wikipedia_superconductors
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_wikipedia_superconductors())
chunks = sentence_window(docs, sentences_per_chunk=4)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('lynx', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print('Pipeline ready.')
from cookbook.corpora import load_wikipedia_superconductors
from cookbook.chunkers import sentence_window
from cookbook.stores import QdrantBackend

docs = list(load_wikipedia_superconductors())
chunks = sentence_window(docs, sentences_per_chunk=4)
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('lynx', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print('Pipeline ready.')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Pipeline ready.

Step 2 — Build the guardrail judge¶

The judge takes (answer, contexts) and returns a score 0..1 plus a verdict. The cookbook uses a prompt-engineered judge; production swaps in the real Lynx model.

In [3]:

Copied!





import re

JUDGE = (
    'You are a hallucination detector. Score 0.0 if the answer contradicts or is unsupported by the passages, '
    '1.0 if every claim is in the passages. Reply with just the number.\n'
    'Passages:\n{ctx}\nAnswer:\n{ans}'
)

def guard(answer: str, contexts: list[str]) -> tuple[float, str]:
    raw = client.chat(JUDGE.format(ctx='\n\n'.join(contexts), ans=answer))
    m = re.search(r'[01](?:\.\d+)?', raw)
    score = float(m.group()) if m else 0.0
    verdict = 'GROUNDED' if score >= 0.5 else 'BLOCK'
    return score, verdict

score, verdict = guard(
    'Superconductors expel magnetic fields completely.',
    ['The Meissner effect is the complete expulsion of magnetic flux from a superconductor.'],
)
print(f'score={score}  verdict={verdict}')
import re

JUDGE = (
    'You are a hallucination detector. Score 0.0 if the answer contradicts or is unsupported by the passages, '
    '1.0 if every claim is in the passages. Reply with just the number.\n'
    'Passages:\n{ctx}\nAnswer:\n{ans}'
)

def guard(answer: str, contexts: list[str]) -> tuple[float, str]:
    raw = client.chat(JUDGE.format(ctx='\n\n'.join(contexts), ans=answer))
    m = re.search(r'[01](?:\.\d+)?', raw)
    score = float(m.group()) if m else 0.0
    verdict = 'GROUNDED' if score >= 0.5 else 'BLOCK'
    return score, verdict

score, verdict = guard(
    'Superconductors expel magnetic fields completely.',
    ['The Meissner effect is the complete expulsion of magnetic flux from a superconductor.'],
)
print(f'score={score}  verdict={verdict}')

score=1.0  verdict=GROUNDED

Step 3 — Test the guardrail on a hallucinated answer¶

Pair a contextually-correct passage with an incorrect answer. The guardrail should fire.

In [4]:

Copied!





score, verdict = guard(
    'Superconductors were invented by Heike Kamerlingh Onnes in 1842.',  # wrong year
    ['Heike Kamerlingh Onnes discovered superconductivity in mercury in 1911.'],
)
print(f'score={score}  verdict={verdict}')
score, verdict = guard(
    'Superconductors were invented by Heike Kamerlingh Onnes in 1842.',  # wrong year
    ['Heike Kamerlingh Onnes discovered superconductivity in mercury in 1911.'],
)
print(f'score={score}  verdict={verdict}')

score=0.0  verdict=BLOCK

Step 4 — Build the guarded pipeline¶

Standard RAG + Lynx gate. On BLOCK, refuse politely.

In [5]:

Copied!





def guarded(question: str) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    hits = store.search(qv, top_k=5)
    ctx = [h.text for h in hits]
    a = client.chat('Use only these passages.\n' + '\n\n'.join(ctx) + f'\nQ: {question}\nA:')
    score, verdict = guard(a, ctx)
    if verdict == 'BLOCK':
        return 'I do not have enough grounded evidence to answer that confidently.', ctx
    return a, ctx

for q in [
    'What is the Meissner effect?',
    'Who personally invented the iPhone superconductor in 1842?',  # nonsense
]:
    a, _ = guarded(q)
    print(f'Q: {q}')
    print(f'  -> {a}')
    print()
def guarded(question: str) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    hits = store.search(qv, top_k=5)
    ctx = [h.text for h in hits]
    a = client.chat('Use only these passages.\n' + '\n\n'.join(ctx) + f'\nQ: {question}\nA:')
    score, verdict = guard(a, ctx)
    if verdict == 'BLOCK':
        return 'I do not have enough grounded evidence to answer that confidently.', ctx
    return a, ctx

for q in [
    'What is the Meissner effect?',
    'Who personally invented the iPhone superconductor in 1842?',  # nonsense
]:
    a, _ = guarded(q)
    print(f'Q: {q}')
    print(f'  -> {a}')
    print()

Q: What is the Meissner effect?
  -> The Meissner effect is the expulsion of a magnetic field from a superconductor during its transition to the superconducting state when it is cooled below the critical temperature, resulting in the repulsion of a nearby magnet.

Q: Who personally invented the iPhone superconductor in 1842?
  -> I do not have enough grounded evidence to answer that confidently.

Step 5 — Wrap as `answer_question`¶

Cookbook contract.

In [6]:

Copied!

def answer_question(question: str) -> tuple[str, list[str]]:
    return guarded(question)

ans, _ = answer_question('What is BCS theory?')
print(ans[:200])
def answer_question(question: str) -> tuple[str, list[str]]:
    return guarded(question)

ans, _ = answer_question('What is BCS theory?')
print(ans[:200])

BCS theory is a microscopic theory of superconductivity that explains many thermodynamic and electromagnetic properties of superconductors, describing superconductivity as a microscopic effect caused

Look Inside¶

Inspect — score distribution across a battery¶

On a labelled set you'd plot the distribution and pick a threshold. Here we approximate.

In [7]:

Copied!





scores = []
for q in [
    'What is the Meissner effect?',
    'Who discovered superconductivity?',
    'What is the boiling point of liquid nitrogen?',
    'When was YBCO discovered?',
]:
    qv = client.embed([q])[0]
    hits = store.search(qv, top_k=5)
    ctx = [h.text for h in hits]
    a = client.chat('Use only these.\n' + '\n\n'.join(ctx) + f'\nQ: {q}\nA:')
    s, v = guard(a, ctx)
    print(f'  score={s:.2f}  verdict={v}  q={q[:40]}')
    scores.append(s)
import statistics
print(f'mean grounded score: {statistics.mean(scores):.2f}')
scores = []
for q in [
    'What is the Meissner effect?',
    'Who discovered superconductivity?',
    'What is the boiling point of liquid nitrogen?',
    'When was YBCO discovered?',
]:
    qv = client.embed([q])[0]
    hits = store.search(qv, top_k=5)
    ctx = [h.text for h in hits]
    a = client.chat('Use only these.\n' + '\n\n'.join(ctx) + f'\nQ: {q}\nA:')
    s, v = guard(a, ctx)
    print(f'  score={s:.2f}  verdict={v}  q={q[:40]}')
    scores.append(s)
import statistics
print(f'mean grounded score: {statistics.mean(scores):.2f}')

  score=1.00  verdict=GROUNDED  q=What is the Meissner effect?
  score=0.00  verdict=BLOCK  q=Who discovered superconductivity?
  score=1.00  verdict=GROUNDED  q=What is the boiling point of liquid nitr
  score=0.00  verdict=BLOCK  q=When was YBCO discovered?
mean grounded score: 0.50

Inspect — false-positive case¶

Sometimes a correct answer scores low because the model paraphrases heavily. We illustrate.

In [8]:

Copied!





score, v = guard(
    'Resistance drops to zero below a certain temperature.',
    ['Superconductivity is the property of zero electrical resistance below a critical temperature.'],
)
print(f'score={score} verdict={v}')
print('Even a clear paraphrase can score low. Tune the threshold accordingly.')
score, v = guard(
    'Resistance drops to zero below a certain temperature.',
    ['Superconductivity is the property of zero electrical resistance below a critical temperature.'],
)
print(f'score={score} verdict={v}')
print('Even a clear paraphrase can score low. Tune the threshold accordingly.')

score=1.0 verdict=GROUNDED
Even a clear paraphrase can score low. Tune the threshold accordingly.

Inspect — cost¶

Guardrails add one LLM call per answer. With Lynx-8B locally, that's a few milliseconds; with a hosted small model, a few cents per thousand answers.

In [9]:

Copied!





print('Per-answer guardrail cost:')
print('  - 1 small-LLM call.')
print('  - ~200-500 input tokens (passages + answer).')
print('  - At GPT-4o-mini pricing: well under a cent per answer.')
print('Per-answer guardrail cost:')
print('  - 1 small-LLM call.')
print('  - ~200-500 input tokens (passages + answer).')
print('  - At GPT-4o-mini pricing: well under a cent per answer.')

Per-answer guardrail cost:
  - 1 small-LLM call.
  - ~200-500 input tokens (passages + answer).
  - At GPT-4o-mini pricing: well under a cent per answer.

Inspect — compose with Self-RAG¶

If Self-RAG (Recipe 24) already filtered passages, the answer has less reason to hallucinate. Lynx becomes a safety net rather than a primary gate.

In [10]:

Copied!

print('Stack: Self-RAG (filter) -> answer -> Lynx (grounding gate) -> ship.')
print('Self-RAG reduces the false-block rate on Lynx by keeping passages relevant.')
print('Stack: Self-RAG (filter) -> answer -> Lynx (grounding gate) -> ship.')
print('Self-RAG reduces the false-block rate on Lynx by keeping passages relevant.')

Stack: Self-RAG (filter) -> answer -> Lynx (grounding gate) -> ship.
Self-RAG reduces the false-block rate on Lynx by keeping passages relevant.

Run It¶

End-to-end with guardrails on.

In [11]:

Copied!

ans, _ = answer_question('In what year did Bednorz and Mueller discover their copper-oxide superconductor?')
print('=== Guarded answer ===')
print(ans)
ans, _ = answer_question('In what year did Bednorz and Mueller discover their copper-oxide superconductor?')
print('=== Guarded answer ===')
print(ans)

=== Guarded answer ===
According to the passage "High-temperature superconductivity", the answer is: 1986.

Side by Side: Vanilla Baseline vs This Technique¶

Ungrounded vs guarded. On the nonsense question, ungrounded RAG hallucinates and guarded RAG refuses.

In [12]:

Copied!





from cookbook.baselines import vanilla_pipeline

q = 'Who personally invented the iPhone superconductor in 1842?'
base = vanilla_pipeline(q, corpus='wikipedia-superconductors', top_k=5)
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'guarded', 'preview': ours_a[:160]},
])
from cookbook.baselines import vanilla_pipeline

q = 'Who personally invented the iPhone superconductor in 1842?'
base = vanilla_pipeline(q, corpus='wikipedia-superconductors', top_k=5)
ours_a, _ = answer_question(q)

import pandas as pd
pd.DataFrame([
    {'pipeline': 'vanilla', 'preview': base.answer[:160]},
    {'pipeline': 'guarded', 'preview': ours_a[:160]},
])

Out[12]:

	pipeline	preview
0	vanilla	The passages do not contain the answer. In fac...
1	guarded	I do not have enough grounded evidence to answ...

Knobs to Turn¶

Seven knobs in priority order:

Threshold. 0.5 is the cookbook default. Higher refuses more (safer); lower lets more through (more useful).
Judge model. Real Lynx-8B is recommended; prompt-engineered judges work but are noisier.
Refusal message. Make it actionable. "I don't have enough evidence to answer that confidently" is better than "sorry I can't help".
Calibration set. Tune threshold on a labelled set; don't pick a number out of the air.
Logging. Log every BLOCK; use them to improve retrieval upstream.
Composition. Stack with Self-RAG (Recipe 24) upstream and content moderation downstream.
Per-segment thresholds. Different domains have different baseline grounding scores; calibrate per segment if your traffic is heterogeneous.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [13]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'wikipedia-superconductors']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'wikipedia-superconductors']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[13]:

	question	expected	actual	contexts_retrieved
0	Who first observed superconductivity, and in w...	Heike Kamerlingh Onnes observed it in mercury ...	I do not have enough grounded evidence to answ...	5
1	What is the Meissner effect?	The complete expulsion of magnetic flux from a...	The Meissner effect is the expulsion of a magn...	5
2	Distinguish Type-I from Type-II superconductors.	Type-I has a single critical field above which...	Type-I and Type-II superconductors can be dist...	5
3	What does BCS theory explain?	It explains conventional superconductivity thr...	BCS theory explains many thermodynamic and ele...	5
4	What is a Cooper pair?	Two electrons bound together by phonon exchang...	A Cooper pair is a pair of electrons bound tog...	5

Closing Thoughts¶

Three failure modes:

False blocks on paraphrase. Lynx sometimes flags correct paraphrases as ungrounded. Tune threshold or use entailment-style judges.
False passes on subtle hallucination. Small fabrications (a wrong number, a swapped entity) can slip through. Compose with structured-output validation.
Cost at scale. Every answer pays guardrail cost. Sample if budget is tight; gate only high-stakes endpoints.

Compose with Self-RAG (Recipe 24) upstream to reduce false blocks. Compose with Phoenix (Recipe 39) to debug guardrail decisions. Compose with NeMo Guardrails for content moderation in addition to grounding.