Topolearn

Learn MLflow

Track experiments, version models and evaluate ML and GenAI apps

17 lessons · 3 hours · 4 levels · covers MLflow 3.x

MLflow is an open-source platform for the machine learning lifecycle. It records experiment runs (parameters, metrics, artifacts), packages models in a standard format, manages versions in a model registry, serves models, and adds tracing and evaluation for LLM and agent applications. Teams use it to make ML work reproducible, comparable and deployable.

Sign in free to track progressOfficial docs ↗

Before you start

MLflow builds on these; the tree links to the exact lessons you need.

PythonPractical level

How deep do you want to go?

1. EssentialsRead and write simple code, do the everyday basics5 lessons · 50 min in total
2. PracticalUse it productively at work, with the usual idioms10 lessons · 2 hours in total
3. AdvancedThe less common features, performance and trade-offs14 lessons · 3 hours in total
4. ExpertInternals and the edge cases experts know17 lessons · 3 hours in total

The MLflow tree

MLflowWhat MLflow doesWhat MLflow doesRuns, params, metrics and tagsRuns, params,metrics and tagsLogging artifactsLogging artifactsAutologgingAutologgingLogging and loading modelsLogging and loadingmodelsSearching and comparing runsSearching andcomparing runsSignatures and input examplesSignatures and inputexamplesModel registry and aliasesModel registry andaliasesNested runs for tuningNested runs fortuningRemote tracking serverRemote trackingserverContext managers and withContext managersand with
Essentials and Practical shown (10 of 17 lessons); the full list is below. Click a lesson to read it.

All 17 lessons, in order

Experiment tracking

  1. What MLflow does · Essentials · 8 min

    Get the big picture: tracking, models, registry, serving and GenAI features, and how to start the local UI.

  2. Runs, params, metrics and tags · Essentials · 12 min

    Log parameters, step-wise metrics and tags inside runs grouped by experiment.

  3. Logging artifacts · Essentials · 10 min

    Store files, plots, dicts, text and tables alongside a run, and read them back later.

  4. Autologging · Essentials · 8 min

    Capture parameters, metrics and models automatically for scikit-learn, XGBoost, PyTorch Lightning and more.

  5. Searching and comparing runs · Practical · 10 min

    Query runs programmatically with filter strings and ordering, and use the UI's comparison views.

  6. Nested runs for tuning · Practical · 10 min

    Organize hyperparameter searches as a parent run with child runs per trial.

Packaging models

  1. Logging and loading models · Essentials · 12 min

    Save a trained model with log_model, load it back natively or as a generic pyfunc, and understand model URIs.

  2. Signatures and input examples · Practical · 10 min

    Describe a model's expected inputs and outputs with a signature so callers and servers can validate requests.

  3. Custom pyfunc models · Advanced · 14 min

    Wrap preprocessing, ensembles or any Python logic as an MLflow model with PythonModel and artifacts.

  4. Evaluating classic ML models · Advanced · 12 min

    Run mlflow.models.evaluate to compute standard metrics, plots and custom metrics, and log them to a run.

Model registry and deployment

  1. Model registry and aliases · Practical · 12 min

    Register model versions under a name, point aliases like champion at them, and load models by alias.

  2. Serving and deploying models · Advanced · 12 min

    Serve a logged or registered model as a REST API locally, build a Docker image, and run batch inference.

Running MLflow for a team

  1. Remote tracking server · Practical · 12 min

    Share experiments across a team with mlflow server, a database backend and object-store artifacts.

  2. LoggedModels and model-centric tracking · Expert · 12 min

    Use MLflow 3's LoggedModel entity to tie metrics, traces and evaluations to model versions across runs.

  3. Operating MLflow at scale · Expert · 14 min

    Run MLflow reliably for many teams: storage layout, access control, upgrades, cleanup and performance.

Tracing and GenAI evaluation

  1. Tracing LLM and agent apps · Advanced · 12 min

    Record every LLM call, tool call and step of a GenAI app as a trace with spans, automatically or with decorators.

  2. Evaluating GenAI apps with scorers · Expert · 15 min

    Use mlflow.genai.evaluate with built-in LLM judges and custom scorers to measure and compare app versions.