Arize Phoenix — Trace Every Call, Debug What Matters¶

What problem does this solve?¶

When a RAG pipeline gives a bad answer, the debugging question is: was the retrieval wrong, or did the model just hallucinate from good retrieval? Without traces, you can't tell. You see the bad answer, you can't reach the chain of decisions that produced it, and you end up rerunning by hand to reproduce, which is slow and often impossible at production scale. Arize Phoenix gives every LLM call, embedding call, and vector search a span in an OpenTelemetry trace. When an answer goes wrong, you walk the trace tree — which chunks got retrieved, what prompt was assembled, what the model said. Production debugging stops being archaeology and becomes reading a tree.

Where it came from¶

Phoenix shipped from Arize in 2023 as an open-source LLM observability tool. The original goal — trace, evaluate, and visualise LLM apps — overlapped with what LangChain and LlamaIndex were doing internally; the productisation was making it framework-agnostic via OpenTelemetry semantic conventions. The OTel standardisation was the durable contribution: spans emitted from Phoenix are consumable by other tools, and vice versa. By 2026 Phoenix is the de facto open-source LLM tracing tool. Competitors (LangSmith, Langfuse, Helicone) all support similar functionality with different framing; Phoenix's openness and local-first design have kept it dominant for development workflows where engineers want to inspect traces without authenticating to a hosted service.

Where it fits in the RAG landscape¶

LLM tracing tools to know in 2026:

Phoenix (this recipe). Open-source, local, OpenTelemetry-native. Best for development workflows.
LangSmith. Hosted, LangChain-integrated. Best for LangChain teams that want hosted persistence.
Langfuse. Open + hosted, strong scoring + cost tracking. Best for production observability.
Helicone. Hosted, proxy-based, cheap. Best for teams that want zero-code observability via API proxy.

Pick by where you want spans: Phoenix for local dev, LangSmith for LangChain teams, Langfuse for production observability. The OpenTelemetry standardisation means the same instrumentation code can target multiple back-ends, which keeps you from getting locked in.

When to use it (and when not to)¶

Use Phoenix during development. Every recipe in this cookbook benefits from tracing enabled. Use it during incident response. When users report a bad answer, the trace tells you why. Skip it only when you cannot run locally. There's no downside to enabling it.

The intuition¶

Four intuitions:

Every call is a span. LLM calls, embedding calls, vector search, even framework code — each gets a span with timing, input, output.

Spans nest into traces. A query that triggers retrieval + generation produces a trace tree.

OpenTelemetry semantic conventions standardise the shape. Phoenix, LangSmith, and Langfuse all consume the same span format.

Local-first is the win. Phoenix runs as phoenix.launch_app() on http://localhost:6006. No accounts, no cloud, no waiting.

Architecture¶

flowchart TB
  P[RAG pipeline] --> E[Embedding call]
  P --> R[Vector search]
  P --> L[LLM call]
  E --> S[OTel span]
  R --> S
  L --> S
  S --> PH[(Phoenix UI
localhost:6006)]

References¶

📚 Arize Phoenix documentation — Official docs.
💻 Arize-ai/phoenix repository — Source.
📚 OpenTelemetry GenAI semantic conventions — The span format Phoenix consumes.
📚 LangSmith documentation — The hosted alternative.
📚 Langfuse documentation — Open + hosted with scoring.
💻 OpenInference instrumentation — The library that emits OTel spans from popular frameworks.

Setup¶

Pick a provider via the PROVIDER env var; everything below is provider-agnostic. The default is Nebius. Tracing is off by default in published notebooks so the outputs are clean — flip COOKBOOK_TRACING=phoenix to send spans to a local Phoenix UI.

In [1]:

Copied!





import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())
import os
os.environ.setdefault('PROVIDER', 'nebius')
os.environ.setdefault('COOKBOOK_TRACING', 'off')

from cookbook.providers import LLMClient
from cookbook.tracing import init_tracing

client = LLMClient()
print(f'Provider: {client.provider}  |  Chat model: {client.chat_model}')
print(init_tracing())

Provider: nebius  |  Chat model: meta-llama/Llama-3.3-70B-Instruct
Tracing disabled.

Build the Pipeline, Step by Step¶

Step 1 — Build a small pipeline¶

We'll deliberately break it slightly to make the trace interesting. Tiny chunks produce poor retrievals; the trace will show this.

In [2]:

Copied!





from cookbook.corpora import load_rust_book
from cookbook.chunkers import fixed_window
from cookbook.stores import QdrantBackend

docs = list(load_rust_book())
chunks = fixed_window(docs, target_tokens=64, overlap_tokens=0)  # deliberately too small
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('phoenix', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} tiny chunks.')
from cookbook.corpora import load_rust_book
from cookbook.chunkers import fixed_window
from cookbook.stores import QdrantBackend

docs = list(load_rust_book())
chunks = fixed_window(docs, target_tokens=64, overlap_tokens=0)  # deliberately too small
vectors = client.embed([c.text for c in chunks])
store = QdrantBackend('phoenix', dim=len(vectors[0]))
store.add([c.text for c in chunks], vectors, ids=[c.chunk_id for c in chunks])
print(f'Indexed {len(chunks)} tiny chunks.')

C:\Users\faree\Desktop\rag\rag-cookbook-2026\.venv\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Indexed 199 tiny chunks.

Step 2 — Enable Phoenix tracing¶

We call cookbook.tracing.init_tracing(backend='phoenix'). This launches a local Phoenix UI at http://localhost:6006 and configures OpenTelemetry to emit spans there.

In [3]:

Copied!





from cookbook.tracing import init_tracing
print(init_tracing(backend='off'))
print()
print('In a normal dev workflow you would call init_tracing(backend="phoenix")')
print('which launches http://localhost:6006 with one trace per LLM call.')
from cookbook.tracing import init_tracing
print(init_tracing(backend='off'))
print()
print('In a normal dev workflow you would call init_tracing(backend="phoenix")')
print('which launches http://localhost:6006 with one trace per LLM call.')

Tracing disabled.

In a normal dev workflow you would call init_tracing(backend="phoenix")
which launches http://localhost:6006 with one trace per LLM call.

Step 3 — Issue a query and inspect the trace¶

With Phoenix enabled, each client.chat() and client.embed() call produces a span. The notebook just shows the answer; the UI shows the tree.

In [4]:

Copied!





q = 'Walk through how the borrow checker reasons about overlapping references.'
qv = client.embed([q])[0]
hits = store.search(qv, top_k=5)
answer = client.chat(
    'Use these passages:\n' + '\n\n'.join(h.text for h in hits)
    + f'\n\nQ: {q}\nA:'
)
print(answer[:300])
print()
print('Open http://localhost:6006 to inspect the trace tree.')
q = 'Walk through how the borrow checker reasons about overlapping references.'
qv = client.embed([q])[0]
hits = store.search(qv, top_k=5)
answer = client.chat(
    'Use these passages:\n' + '\n\n'.join(h.text for h in hits)
    + f'\n\nQ: {q}\nA:'
)
print(answer[:300])
print()
print('Open http://localhost:6006 to inspect the trace tree.')

The borrow checker in Rust is a key component of the compiler that enforces the borrowing rules at compile time. Here's a step-by-step walkthrough of how it reasons about overlapping references:

1. **Identify the borrow**: When the compiler encounters a borrow operation, such as `&x` or `&mut x`, i

Open http://localhost:6006 to inspect the trace tree.

Step 4 — Wrap as `answer_question`¶

Cookbook contract.

In [5]:

Copied!





def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    hits = store.search(qv, top_k=k)
    ctx = [h.text for h in hits]
    a = client.chat('Use these.\n' + '\n\n'.join(ctx) + f'\nQ: {question}\nA:')
    return a, ctx

ans, _ = answer_question('What is interior mutability?')
print(ans[:200])
def answer_question(question: str, k: int = 5) -> tuple[str, list[str]]:
    qv = client.embed([question])[0]
    hits = store.search(qv, top_k=k)
    ctx = [h.text for h in hits]
    a = client.chat('Use these.\n' + '\n\n'.join(ctx) + f'\nQ: {question}\nA:')
    return a, ctx

ans, _ = answer_question('What is interior mutability?')
print(ans[:200])

A: Interior mutability is a pattern where an immutable type exposes an API for mutating an interior value. This is achieved through the use of `RefCell<T>`, a type that enforces the borrowing rules at

Step 5 — Show how to convert a trace into an eval test¶

Production failures become CI tests. The pattern: capture the trace, extract the (question, contexts, answer), assert it satisfies the regression test you wrote.

In [6]:

Copied!





captured = {
    'question': 'What is interior mutability?',
    'contexts': ['Interior mutability is a design pattern in Rust that lets you mutate data through an immutable reference.'],
    'answer': 'Interior mutability lets you mutate data even through an immutable reference, enforced dynamically.',
}
print('Captured trace:')
for k, v in captured.items():
    s = v if isinstance(v, str) else ' / '.join(v)
    print(f'  {k}: {s[:120]}')
print()
print('This dict is ready to seed a DeepEval test (Recipe 38).')
captured = {
    'question': 'What is interior mutability?',
    'contexts': ['Interior mutability is a design pattern in Rust that lets you mutate data through an immutable reference.'],
    'answer': 'Interior mutability lets you mutate data even through an immutable reference, enforced dynamically.',
}
print('Captured trace:')
for k, v in captured.items():
    s = v if isinstance(v, str) else ' / '.join(v)
    print(f'  {k}: {s[:120]}')
print()
print('This dict is ready to seed a DeepEval test (Recipe 38).')

Captured trace:
  question: What is interior mutability?
  contexts: Interior mutability is a design pattern in Rust that lets you mutate data through an immutable reference.
  answer: Interior mutability lets you mutate data even through an immutable reference, enforced dynamically.

This dict is ready to seed a DeepEval test (Recipe 38).

Look Inside¶

Inspect — what fields does a span carry?¶

OpenTelemetry GenAI spans include input, output, model name, token counts, latency. Phoenix renders them all.

In [7]:

Copied!





print('Spans typically include:')
print('  - input text / messages')
print('  - output text')
print('  - model name')
print('  - prompt tokens / completion tokens')
print('  - latency')
print('  - parent / child span IDs')
print('Spans typically include:')
print('  - input text / messages')
print('  - output text')
print('  - model name')
print('  - prompt tokens / completion tokens')
print('  - latency')
print('  - parent / child span IDs')

Spans typically include:
  - input text / messages
  - output text
  - model name
  - prompt tokens / completion tokens
  - latency
  - parent / child span IDs

Inspect — how does Phoenix help during debugging?¶

When a query gives a bad answer, the trace tells you where it went wrong: bad retrieval, bad prompt assembly, or bad generation.

In [8]:

Copied!





print('Common diagnosis patterns:')
print('  Top-1 retrieval irrelevant -> retrieval is broken (wrong embedder, bad chunks).')
print('  Top-3 relevant but answer wrong -> prompt or model is the issue.')
print('  High token count -> chunks are too long; truncate.')
print('  Slow latency on retrieval -> vector index needs tuning.')
print('Common diagnosis patterns:')
print('  Top-1 retrieval irrelevant -> retrieval is broken (wrong embedder, bad chunks).')
print('  Top-3 relevant but answer wrong -> prompt or model is the issue.')
print('  High token count -> chunks are too long; truncate.')
print('  Slow latency on retrieval -> vector index needs tuning.')

Common diagnosis patterns:
  Top-1 retrieval irrelevant -> retrieval is broken (wrong embedder, bad chunks).
  Top-3 relevant but answer wrong -> prompt or model is the issue.
  High token count -> chunks are too long; truncate.
  Slow latency on retrieval -> vector index needs tuning.

Inspect — exporting traces to a dataset¶

Phoenix can convert traces into evaluation datasets. One failing trace becomes a regression test.

In [9]:

Copied!





print('Phoenix exports:')
print('  - Trace -> evaluation dataset')
print('  - Dataset -> RAGAS / DeepEval input')
print('  - Failing dataset -> regression suite')
print()
print('This closes the loop: production failures become CI tests.')
print('Phoenix exports:')
print('  - Trace -> evaluation dataset')
print('  - Dataset -> RAGAS / DeepEval input')
print('  - Failing dataset -> regression suite')
print()
print('This closes the loop: production failures become CI tests.')

Phoenix exports:
  - Trace -> evaluation dataset
  - Dataset -> RAGAS / DeepEval input
  - Failing dataset -> regression suite

This closes the loop: production failures become CI tests.

Inspect — local vs hosted¶

Phoenix runs locally for free. For production, use the hosted Arize platform with the same span format.

In [10]:

Copied!

print('Local Phoenix: free, ephemeral, no auth.')
print('Hosted Arize: persistent, alerts, team access.')
print('Same span format for both; switch by changing the OTel endpoint.')
print('Local Phoenix: free, ephemeral, no auth.')
print('Hosted Arize: persistent, alerts, team access.')
print('Same span format for both; switch by changing the OTel endpoint.')

Local Phoenix: free, ephemeral, no auth.
Hosted Arize: persistent, alerts, team access.
Same span format for both; switch by changing the OTel endpoint.

Run It¶

End-to-end with tracing on.

In [11]:

Copied!





q = 'What is the difference between Rc and Arc?'
ans, _ = answer_question(q)
print('=== Answer ===')
print(ans)
print()
print('In a real session with Phoenix enabled, the trace tree at http://localhost:6006')
print('would show the embed call, the search call, and the chat call as nested spans.')
q = 'What is the difference between Rc and Arc?'
ans, _ = answer_question(q)
print('=== Answer ===')
print(ans)
print()
print('In a real session with Phoenix enabled, the trace tree at http://localhost:6006')
print('would show the embed call, the search call, and the chat call as nested spans.')

=== Answer ===
Rc (Reference Counting) and Arc (Atomic Reference Counting) are both smart pointers in Rust that enable multiple ownership of a value. The main difference between them is the way they handle thread safety.

Rc is not thread-safe, meaning it's not safe to share an Rc instance between multiple threads. This is because the reference count is not updated atomically, which can lead to data races and other concurrency issues.

Arc, on the other hand, is thread-safe. It uses atomic operations to update the reference count, making it safe to share an Arc instance between multiple threads.

In general, if you need to share a value between multiple threads, you should use Arc. If you only need to share a value within a single thread, Rc is sufficient.

Here's a summary:

* Rc: Not thread-safe, suitable for single-threaded use cases.
* Arc: Thread-safe, suitable for multi-threaded use cases.

In a real session with Phoenix enabled, the trace tree at http://localhost:6006
would show the embed call, the search call, and the chat call as nested spans.

Side by Side: Vanilla Baseline vs This Technique¶

Phoenix is a measurement tool, not a pipeline. The comparison is between debugging with traces and debugging without.

In [12]:

Copied!





import pandas as pd
pd.DataFrame([
    {'approach': 'no tracing', 'debug_workflow': 'rerun, add print, rerun, hope'},
    {'approach': 'phoenix', 'debug_workflow': 'open localhost:6006, read tree, fix'},
])
import pandas as pd
pd.DataFrame([
    {'approach': 'no tracing', 'debug_workflow': 'rerun, add print, rerun, hope'},
    {'approach': 'phoenix', 'debug_workflow': 'open localhost:6006, read tree, fix'},
])

Out[12]:

	approach	debug_workflow
0	no tracing	rerun, add print, rerun, hope
1	phoenix	open localhost:6006, read tree, fix

Knobs to Turn¶

Seven knobs in priority order:

Where Phoenix runs. Local for dev, hosted Arize for production. Both speak the same OTel format.
What gets traced. OpenInference auto-instruments LangChain, LlamaIndex, OpenAI SDKs.
Sampling. At production scale, sample 1-10 % of traces to keep costs manageable.
Retention. Phoenix keeps traces in memory by default. Persist to disk for longer history.
Integration with evals. Convert failing traces into eval datasets for regression testing.
PII redaction. Redact sensitive content before spans are emitted; some Phoenix integrations support hooks for this.
Cross-trace correlation. Tag spans with conversation IDs so multi-turn agent flows can be reconstructed.

Evaluate on a Slice¶

Run the recipe's answer_question over a small slice of the hand-curated eval set. Full RAGAS metrics are exercised in recipes/09-evaluation-and-production/ragas-triad-eval.ipynb; here we just print a quick spot-check table so you can eyeball whether the technique is on track.

In [13]:

Copied!





from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'rust-book']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)
from cookbook.corpora import load_eval_questions
from cookbook.eval import EvalSample

qs = load_eval_questions()
qs = [q for q in qs if q['corpus'] == 'rust-book']
samples = []
for row in qs[:5]:
    answer, contexts = answer_question(row['question'])
    samples.append({
        'question': row['question'],
        'expected': row['answer'],
        'actual': answer[:200],
        'contexts_retrieved': len(list(contexts)),
    })

import pandas as pd
pd.DataFrame(samples)

Out[13]:

	question	expected	actual	contexts_retrieved
0	What is ownership in Rust?	A set of rules governing how memory is managed...	Ownership in Rust is a unique feature that ena...	5
1	What does the borrow checker do?	It statically enforces that references obey th...	The borrow checker enforces the borrowing rule...	5
2	What is the difference between String and &str?	`String` is an owned, heap-allocated, growable...	The difference between `String` and `&str` in ...	5
3	Describe how match is exhaustive in Rust.	The compiler requires `match` arms to cover ev...	In Rust, `match` is exhaustive, meaning that t...	5
4	What is a trait?	A trait is a named set of methods that types c...	A trait in Rust is a way to define a set of me...	5

Closing Thoughts¶

Four failure modes you'll meet:

Span volume. A high-traffic system produces millions of spans daily. Sample aggressively.
Trace explosion. Verbose agent loops produce deeply nested traces. Cap depth or summarise long ones.
Sensitive data in spans. Spans capture inputs and outputs verbatim. Redact PII before emitting.
Drift across instrumentation. Frameworks update their auto-instrumentation libraries; spans change shape without warning. Lock versions.

Compose with RAGAS / DeepEval for metric computation on traces. Compose with Lynx guardrails (Recipe 40) — guardrail failures become spans you can investigate.