TopolearnThe metric you choose decides which model looks best, so it should match what errors cost you.
Regression metrics compare predicted numbers with the true ones. The main ones:
- MAE (
mean_absolute_error): average absolute error, in the target's units. Robust to a few large errors. - MSE (
mean_squared_error) and RMSE (root_mean_squared_error): squaring punishes large errors heavily. RMSE is back in the target's units and is always at least the MAE. - R² (
r2_score): the fraction of variance explained, unitless, handy for comparing across datasets, but it says nothing about error size in real units. - MAPE (
mean_absolute_percentage_error): relative error; returned as a fraction (0.1 means 10%) and unstable when true values are near zero.
Which to use depends on the cost of errors. If a 20-unit miss is twice as bad as a 10-unit miss, use MAE. If large misses are disproportionately bad, use RMSE. Report the metric in units stakeholders understand, and always compare against a baseline such as DummyRegressor() predicting the training mean.
In model-selection tools the metrics appear as scorer strings like "neg_mean_absolute_error" and "neg_root_mean_squared_error". They are negated because scikit-learn's search tools always maximise a score; just flip the sign back when reporting.
Example
import numpy as np
from sklearn.metrics import (mean_absolute_error, root_mean_squared_error,
r2_score, mean_absolute_percentage_error)
y_true = np.array([100, 150, 200, 250, 300])
y_pred = np.array([110, 140, 210, 240, 360]) # one big miss at the end
print(mean_absolute_error(y_true, y_pred)) # 20.0
print(round(root_mean_squared_error(y_true, y_pred), 2)) # 28.28
print(round(r2_score(y_true, y_pred), 3)) # 0.84
print(round(mean_absolute_percentage_error(y_true, y_pred), 3)) # 0.091Remember
- MAE treats all errors linearly; RMSE punishes big errors more
- R² is unitless; MAE and RMSE are in the target's units
- Scorer strings for errors are negated, e.g.
neg_mean_absolute_error
Watch out for
- Using MAPE on a target that can be zero or near zero.