TopolearnThe native API exposes features and performance the scikit-learn wrapper hides, and is what the wrapper calls internally.
Under the scikit-learn wrapper sits the native API:
xgb.DMatrix(X, label=y, weight=w, missing=np.nan, enable_categorical=True)is XGBoost's internal data container. Building it once and reusing it saves time.xgb.QuantileDMatrixpre-bins features for the histogram method and uses much less memory; passref=dtrainwhen building the validation matrix so it uses the same bins.xgb.train(params, dtrain, num_boost_round, evals=[(dval, "val")], early_stopping_rounds=50)returns aBooster. Parameters go in a dict using native names:eta,max_depth,objective,eval_metric,lambda,alpha.booster.predict(dtest)returns probabilities forbinary:logistic(there is no separate predict_proba). Useiteration_range=(0, booster.best_iteration + 1)to predict with the best trees.xgb.cv(params, dtrain, num_boost_round, nfold=5, early_stopping_rounds=50, stratified=True)runs cross-validation with early stopping across folds and returns a DataFrame of mean and std metrics per round; its length is the best number of rounds.
The native API also supports callbacks (for example xgb.callback.LearningRateScheduler), custom objectives and metrics, and xgb_model= to continue training an existing booster.
You can move between the two worlds: model.get_booster() gives the booster behind a scikit-learn estimator.
Example
import xgboost as xgb
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_tr, X_val, y_tr, y_val = train_test_split(X, y, stratify=y, random_state=0)
dtrain = xgb.QuantileDMatrix(X_tr, label=y_tr)
dval = xgb.QuantileDMatrix(X_val, label=y_val, ref=dtrain) # same bins as train
params = {"objective": "binary:logistic", "eta": 0.05, "max_depth": 3, "eval_metric": "auc"}
cv = xgb.cv(params, xgb.DMatrix(X_tr, label=y_tr), num_boost_round=1000, nfold=5,
early_stopping_rounds=50, seed=0)
print(len(cv), round(cv["test-auc-mean"].iloc[-1], 4)) # 67 0.994
booster = xgb.train(params, dtrain, num_boost_round=len(cv), evals=[(dval, "val")],
verbose_eval=False)
proba = booster.predict(dval) # probabilities
print(proba[:3].round(3)) # [0.982 0.026 0.026]Remember
- DMatrix/QuantileDMatrix hold data;
xgb.trainreturns a Booster - Native parameter names differ:
eta,lambda,alpha xgb.cvfinds the number of rounds with early stopping across folds
Watch out for
- Building a validation QuantileDMatrix without
ref=dtrain, so it uses different bins.