TopolearnLLM apps fail in non-obvious ways; traces show exactly which prompt, retrieval or tool call went wrong.
A trace records one request through your application as a tree of spans. Each span has a name, type (LLM, RETRIEVER, TOOL, CHAIN...), inputs, outputs, timing, attributes and status. MLflow's tracing is OpenTelemetry-compatible.
Three ways to create traces:
- Autolog:
mlflow.openai.autolog(),mlflow.langchain.autolog(),mlflow.anthropic.autolog()and many more capture LLM calls, token usage and tool calls with no code changes. - Decorator:
@mlflow.traceon your own functions records their arguments and return values; nested decorated calls become child spans. - Context manager:
with mlflow.start_span("rerank") as span:withspan.set_inputs(...),span.set_outputs(...),span.set_attribute(...)for fine-grained blocks.
Traces appear in the experiment's Traces tab, where you can open the span tree, see latency per step and inspect prompts. Programmatically, mlflow.search_traces() returns a DataFrame for analysis, and mlflow.get_last_active_trace_id() gives the most recent one.
You can attach tags (user, session, version) with mlflow.update_current_trace(tags=...) and feedback with mlflow.log_feedback(...), which later feeds evaluation.
In production, use the lightweight mlflow-tracing package and asynchronous logging so tracing adds minimal latency.
Example
import mlflow
mlflow.set_experiment("rag-app")
@mlflow.trace(span_type="RETRIEVER")
def retrieve(question: str) -> list[str]:
return ["MLflow traces are trees of spans."]
@mlflow.trace(span_type="LLM")
def fake_llm(prompt: str) -> str:
return "A trace is a tree of spans." # stand-in for a real model call
@mlflow.trace
def answer(question: str) -> str:
docs = retrieve(question)
with mlflow.start_span("build_prompt") as span:
prompt = f"Context: {docs}\nQ: {question}"
span.set_attribute("n_docs", len(docs))
return fake_llm(prompt)
print(answer("What is a trace?"))
trace = mlflow.get_trace(mlflow.get_last_active_trace_id(), flush=True)
print([s.name for s in trace.data.spans]) # ['answer', 'retrieve', 'build_prompt', 'fake_llm']Remember
- A trace is a tree of typed spans with inputs, outputs and timing
- Use autolog for supported libraries and
@mlflow.trace/start_spanfor your code - Search, tag and attach feedback to traces for debugging and evaluation
Watch out for
- Tracing only the final LLM call, which hides retrieval or tool failures that cause most bad answers.