TopolearnThe scikit-learn wrapper is the quickest way to use XGBoost and plugs into pipelines and model selection.
XGBoost builds an ensemble of decision trees by gradient boosting: each new tree is fitted to correct the errors of the trees before it, and the final prediction is the sum of all trees' outputs (passed through a sigmoid or softmax for classification).
The xgboost package ships scikit-learn compatible estimators:
XGBClassifierfor binary and multi-class classification,XGBRegressorfor regression,XGBRankerfor learning to rank.
They follow the familiar API: hyperparameters in the constructor, then fit(X, y), predict(X), predict_proba(X) and score. They accept NumPy arrays, pandas DataFrames (keeping feature names) and SciPy sparse matrices. Because they are real scikit-learn estimators, they work with cross_val_score, GridSearchCV, Pipeline and clone.
Defaults are reasonable: 100 trees (n_estimators), learning_rate=0.3, max_depth=6 and the fast histogram tree method. Unlike linear models or k-NN, tree boosting needs no feature scaling, and missing values (NaN) are handled natively.
For classification, labels must be integers 0..k-1. String labels raise an error, so encode them with scikit-learn's LabelEncoder first and decode predictions afterwards.
Example
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=0)
model = XGBClassifier(n_estimators=200, learning_rate=0.1, max_depth=4)
model.fit(X_tr, y_tr)
print(round(model.score(X_te, y_te), 3)) # 0.951 (accuracy)
print(model.predict_proba(X_te.iloc[:2]).round(3))
# [[0. 1.]
# [1. 0.]] (very confident on these two rows)
print(model.n_features_in_, model.feature_names_in_[:2])
# 30 ['mean radius' 'mean texture']Remember
- XGBClassifier/XGBRegressor follow the scikit-learn fit/predict API
- No feature scaling needed; NaN is handled natively
- Class labels must be integers 0..k-1
Watch out for
- Passing string class labels to XGBClassifier without encoding them.