TopolearnFraud, defaults and failures are rare events, and the defaults optimise for the majority class.
With a rare positive class, three XGBoost-specific levers help, on top of the general advice to evaluate with precision-recall metrics.
Reweight the classes. scale_pos_weight multiplies the gradient of positive examples. A common starting value is n_negative / n_positive; treat it as a hyperparameter. For multi-class or per-row weighting pass sample_weight= to fit (scikit-learn's compute_sample_weight("balanced", y) helps).
Weighting improves ranking of rare cases in some problems, but it distorts predicted probabilities upward. If you need calibrated probabilities (expected losses, risk scores), either don't reweight and instead tune the decision threshold, or recalibrate afterwards with CalibratedClassifierCV.
Monitor the right metric. Use eval_metric="aucpr" (area under the precision-recall curve) or logloss for early stopping rather than error, which rewards predicting the majority class.
Stabilise extreme cases. max_delta_step (for example 1-10) caps each leaf's update, which can help logistic regression on extremely imbalanced data converge.
Finally, choose the threshold for predict on validation data for your cost trade-off: scikit-learn's TunedThresholdClassifierCV works with XGBoost estimators directly. Always split with stratification so each fold contains positives.
Example
import numpy as np
from sklearn.datasets import make_classification
from sklearn.metrics import average_precision_score, recall_score
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier
X, y = make_classification(n_samples=20000, weights=[0.99], class_sep=0.8,
flip_y=0, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=0)
ratio = (y_tr == 0).sum() / (y_tr == 1).sum()
print(round(ratio)) # 99
for spw in (1, ratio):
m = XGBClassifier(n_estimators=300, learning_rate=0.05, max_depth=4,
scale_pos_weight=spw, eval_metric="aucpr").fit(X_tr, y_tr)
p = m.predict_proba(X_te)[:, 1]
print(round(average_precision_score(y_te, p), 3), round(recall_score(y_te, p > 0.5), 3),
round(float(p.mean()), 3))
# 0.467 0.4 0.009 (unweighted: mean probability matches the 1% base rate)
# 0.456 0.54 0.053 (weighted: higher recall at 0.5, probabilities inflated 5x)Remember
scale_pos_weight~ negatives/positives is a starting point, not a rule- Reweighting distorts probabilities; tune the threshold or recalibrate
- Early-stop on
aucprorlogloss, noterror
Watch out for
- Using scale_pos_weight and then reading predict_proba as true probabilities.