TopolearnWhen the business loss isn't a built-in objective, a custom one optimises what you actually care about.
Because XGBoost only needs per-row gradients and hessians, any twice-differentiable loss can be plugged in.
With the native API, write obj(predt, dtrain) that returns (grad, hess) arrays of the same length as the data, and pass xgb.train(params, dtrain, obj=obj). predt is the raw margin (before any link function), so for a custom classification loss you apply the sigmoid yourself. With the scikit-learn API, pass a callable objective(y_true, y_pred) returning (grad, hess) to the constructor.
Requirements and tips:
- Hessians must be positive; for non-convex losses clip them to a small positive value, or the gain formula breaks down.
- Set
base_scoresensibly (for example the target mean, or 0 for margins), since XGBoost cannot estimate it for an unknown loss. - Check your gradient numerically on a few points before training.
- Predictions from
booster.predictare raw margins for custom objectives; apply the inverse link yourself.
Custom metrics: in the native API pass custom_metric=f(predt, dtrain) -> (name, value); in the scikit-learn API pass a scikit-learn style function as eval_metric=, which then drives early stopping (lower is better unless maximize=True in xgb.train).
Typical uses: asymmetric costs (under-forecasting stock is worse than over-forecasting), focal loss for imbalance, or a log-cosh loss for robust regression.
Example
import numpy as np
import xgboost as xgb
def asymmetric_sq(predt, dtrain, under_cost=4.0):
"""Squared error where under-prediction costs 4x more."""
y = dtrain.get_label()
r = predt - y
w = np.where(r < 0, under_cost, 1.0)
return 2 * w * r, 2 * w # grad, hess (positive)
rng = np.random.default_rng(4)
X = rng.uniform(0, 10, size=(5000, 2))
y = 10 + 3 * X[:, 0] + rng.normal(0, 5, 5000)
d = xgb.DMatrix(X, label=y)
params = {"max_depth": 3, "eta": 0.1, "base_score": float(y.mean())}
plain = xgb.train({**params, "objective": "reg:squarederror"}, d, 300)
custom = xgb.train(params, d, 300, obj=asymmetric_sq)
for b in (plain, custom):
p = b.predict(d)
print(round(float((p < y).mean()), 2)) # share of under-predictions
# 0.49 (symmetric loss)
# 0.31 (custom loss biases predictions upward)Remember
- A custom objective returns per-row gradient and positive hessian w.r.t. the raw margin
- Set
base_scoreand apply the inverse link to predictions yourself - Custom metrics can drive early stopping
Watch out for
- Returning hessians that are zero or negative, which breaks leaf weight calculation.