TopolearnStandardized evaluation makes runs comparable and catches regressions before a model is promoted.
mlflow.models.evaluate scores a model on a dataset and logs the results to the active run:
result = mlflow.models.evaluate(model_uri, eval_df, targets="label",
model_type="classifier")model_type="classifier"computes accuracy, precision, recall, F1, log loss, ROC AUC and logs a confusion matrix and ROC/PR curves;"regressor"computes MAE, RMSE, R2 and more.result.metricsis a dict of values;result.artifactsholds plots and tables.- Pass
predictions="pred_col"withmodel=Noneto evaluate precomputed predictions, for models MLflow cannot load. extra_metrics=[make_metric(eval_fn=..., greater_is_better=...)]adds business metrics such as expected profit.
Validation thresholds turn evaluation into a gate: mlflow.validate_evaluation_results(candidate_result=..., baseline_result=..., validation_thresholds={"accuracy": MetricThreshold(threshold=0.8, greater_is_better=True)}) raises if the candidate fails absolute or relative checks against the baseline. Use this in CI before moving the champion alias.
For LLM and agent applications use mlflow.genai.evaluate instead, which is built around traces and scorers (an Expert lesson). SHAP-based explanations can be added with evaluator_config={"log_explainer": True} when shap is installed.
Evaluate on a fixed, versioned dataset so metrics across runs are comparable.
Example
import mlflow
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
data = load_breast_cancer(as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(data.data, data.target, random_state=0)
model = LogisticRegression(max_iter=5000).fit(X_tr, y_tr)
eval_df = X_te.assign(label=y_te)
with mlflow.start_run():
info = mlflow.sklearn.log_model(model, name="lr", input_example=X_te.head(2))
result = mlflow.models.evaluate(
info.model_uri, eval_df, targets="label", model_type="classifier")
print(round(result.metrics["accuracy_score"], 3)) # about 0.95
print("roc_auc" in result.metrics) # TrueRemember
mlflow.models.evaluatelogs standard metrics and plots per model type- Add business metrics with
extra_metrics - Gate promotions with validation thresholds against a baseline
Watch out for
- Evaluating each run on a different random split, which makes metric comparisons meaningless.