TopolearnThe objective defines what 'good' means during training; the wrong one quietly optimises the wrong thing.
The objective is the loss function XGBoost minimises. The scikit-learn classes pick sensible defaults, but you can set it explicitly with objective=:
reg:squarederror(regressor default): mean squared error,reg:absoluteerror,reg:pseudohubererror: less sensitive to outliers,reg:quantileerrorwithquantile_alpha: predict quantiles, for example 0.9 for an upper bound,count:poisson,reg:tweedie,reg:gamma: counts, insurance-style losses, positive skewed targets (log link),binary:logistic(binary classifier default): log loss, outputs probabilities,multi:softprob: multi-class probabilities, used automatically when y has more than two classes,rank:ndcg,rank:pairwise: learning to rank.
The evaluation metric (eval_metric=) is what XGBoost reports on validation data and uses for early stopping. Options include rmse, mae, logloss, auc, aucpr, error, mlogloss, ndcg@10. You can pass a list; early stopping uses the last one.
Pick the objective to match how the target is distributed and what you will do with predictions, and the metric to match how you will judge the model. For example, an imbalanced fraud model might train on binary:logistic but monitor aucpr. Scikit-learn metrics computed on predictions afterwards remain the final judge.
Example
import numpy as np
from xgboost import XGBRegressor
rng = np.random.default_rng(0)
X = rng.uniform(0, 10, size=(2000, 3))
y = rng.poisson(np.exp(0.2 * X[:, 0] - 0.1 * X[:, 1])) # count target
sq = XGBRegressor(n_estimators=200, max_depth=3).fit(X, y)
pois = XGBRegressor(n_estimators=200, max_depth=3, objective="count:poisson").fit(X, y)
x_new = np.array([[9.0, 0.0, 5.0], [0.0, 9.0, 5.0]])
print(np.exp(0.2 * x_new[:, 0] - 0.1 * x_new[:, 1]).round(2)) # [6.05 0.41] (true means)
print(sq.predict(x_new).round(2), pois.predict(x_new).round(2)) # [7.2 0.29] [6.9 0.27]
q90 = XGBRegressor(objective="reg:quantileerror", quantile_alpha=0.9, n_estimators=200).fit(X, y)
print(round(float((y <= q90.predict(X)).mean()), 2)) # 0.92 (about 90% lie below)Remember
objective= what is optimised;eval_metric= what is reported and early-stopped on- Match the objective to the target: counts, quantiles, robust errors, ranking
- Classifiers default to logistic or softprob objectives automatically
Watch out for
- Using squared error on a heavily skewed count target when a Poisson or Tweedie objective fits better.