TopolearnMatching the loss to the problem often beats any amount of hyperparameter tuning.
The objective defines the loss. Beyond the defaults (regression = L2, binary, multiclass):
regression_l1,huber,fair: robust to outliers,quantilewithalpha=0.9: conditional quantiles, for prediction intervals,poisson,gamma,tweedie(tweedie_variance_power): counts, positive skewed amounts, insurance losses with many zeros (log link),cross_entropy(xentropy): targets that are probabilities in [0, 1],multiclassova: one-vs-all multi-class,lambdarank,rank_xendcg: ranking.
metric (or eval_metric in the wrapper) controls what is evaluated: auc, average_precision, binary_logloss, l1, l2, quantile, ndcg, map and more. A custom metric in the scikit-learn API is a function (y_true, y_pred) -> (name, value, is_higher_better) passed as eval_metric.
Imbalanced classes: is_unbalance=True or scale_pos_weight=n_neg/n_pos (use one, not both) reweight the positive class; class_weight="balanced" works in the wrapper, including multi-class. As with any reweighting, probabilities become inflated; tune the decision threshold or recalibrate if you need real probabilities. Monitor average_precision rather than accuracy.
For prediction intervals, train two quantile models (for example alpha 0.05 and 0.95) and check their empirical coverage on validation data.
Example
import lightgbm as lgb
import numpy as np
rng = np.random.default_rng(1)
X = rng.uniform(0, 10, size=(4000, 2))
y = 5 * X[:, 0] + rng.normal(0, 1 + X[:, 0], 4000) # noise grows with x0
X_tr, X_val, y_tr, y_val = X[:3000], X[3000:], y[:3000], y[3000:]
lo = lgb.LGBMRegressor(objective="quantile", alpha=0.05, verbose=-1).fit(X_tr, y_tr)
hi = lgb.LGBMRegressor(objective="quantile", alpha=0.95, verbose=-1).fit(X_tr, y_tr)
inside = (y_val >= lo.predict(X_val)) & (y_val <= hi.predict(X_val))
print(round(inside.mean(), 2)) # 0.82 (target 0.90: check coverage, quantile models often undercover)
width = hi.predict([[1.0, 5.0], [9.0, 5.0]]) - lo.predict([[1.0, 5.0], [9.0, 5.0]])
print(width.round(1)) # [ 4.7 27. ] (wider where noisier)
def mape(y_true, y_pred):
return "mape", float(np.mean(np.abs((y_true - y_pred) / y_true))), False
m = lgb.LGBMRegressor(verbose=-1).fit(X_tr, y_tr + 100, eval_X=X_val, eval_y=y_val + 100,
eval_metric=mape)
print(sorted(m.evals_result_["valid_0"])) # ['l2', 'mape']Remember
- Match the objective to the target: quantile, Poisson/Tweedie, robust losses
- Custom metrics return (name, value, is_higher_better)
- Use one of is_unbalance or scale_pos_weight, and expect inflated probabilities
Watch out for
- Setting both is_unbalance and scale_pos_weight, which LightGBM rejects.