TopolearnRAG and agent bugs hide in intermediate steps; traces show exactly what each step received and returned.
A single query can involve embedding calls, retrieval, postprocessing, several LLM calls and tool calls. When an answer is wrong or slow, you need to see each step.
Quick look. set_global_handler("simple") prints LLM inputs and outputs to the console. Good for local debugging, too noisy for anything else.
Instrumentation module. LlamaIndex emits events (things that happened, such as an LLM call starting or a retrieval finishing) and spans (timed sections of work, nested in a tree) through a dispatcher in llama_index.core.instrumentation. You can attach your own handlers:
dispatcher = get_dispatcher()
dispatcher.add_event_handler(MyEventHandler())A handler subclasses BaseEventHandler and receives every event in handle(event), for example LLMChatEndEvent or RetrievalEndEvent. Use this to log token usage, latency or retrieved node ids to your own systems. Your own functions can join traces with the @dispatcher.span decorator.
Tracing platforms. One-line integrations send full traces to tools such as Arize Phoenix, Langfuse, MLflow, Weights & Biases Weave, OpenLLMetry and others, many based on OpenTelemetry (an open standard for traces). These show the tree of steps for each request with inputs, outputs, tokens and timings, and usually support evaluation on recorded traces. Workflows also emit step-level events you can stream.
What to record in production: question, retrieved node ids and scores, final prompt size, answer, tokens, latency per step and errors. Remove or mask personal data before sending traces to a third party.
Example
from llama_index.core import Document, VectorStoreIndex
from llama_index.core.instrumentation import get_dispatcher
from llama_index.core.instrumentation.event_handlers import BaseEventHandler
from llama_index.core.instrumentation.events.llm import LLMChatEndEvent, LLMCompletionEndEvent
from llama_index.core.instrumentation.events.retrieval import RetrievalEndEvent
class TraceLogger(BaseEventHandler):
@classmethod
def class_name(cls) -> str:
return "TraceLogger"
def handle(self, event, **kwargs):
if isinstance(event, RetrievalEndEvent):
print("[retrieval]", [n.node.node_id[:8] for n in event.nodes])
elif isinstance(event, (LLMChatEndEvent, LLMCompletionEndEvent)):
print("[llm] done:", type(event).__name__)
get_dispatcher().add_event_handler(TraceLogger()) # root dispatcher sees everything
index = VectorStoreIndex.from_documents([Document(text="Refunds take 5 days.")])
index.as_query_engine().query("Refund time?")
# [retrieval] ['3f2a9c1b']
# [llm] done: LLMChatEndEventRemember
set_global_handler("simple")for quick console tracing- Instrumentation: events and spans via
get_dispatcher()and custom handlers - Tracing integrations (Phoenix, Langfuse, MLflow, OpenTelemetry) for production
Watch out for
- Sending full prompts with customer data to a third-party tracing service without masking or a data agreement.