TopolearnReal datasets outgrow a laptop, and XGBoost's scaling options cut training from hours to minutes.
GPU training is a parameter away: XGBClassifier(device="cuda") (or "cuda:1" for a specific GPU) with the default hist method. Training is often 5-20x faster on large data. Keep data on the GPU when you can (CuPy arrays, cuDF DataFrames) to avoid host-device copies; a warning tells you when data is copied at prediction time. The older gpu_hist and gpu_id parameters have been replaced by device.
Memory efficiency on one machine:
QuantileDMatrixstores pre-binned features instead of raw floats, cutting memory severalfold,- lowering
max_binreduces memory and speeds up training at a small accuracy cost, - external memory: subclass
xgb.DataIterto stream data in batches from disk; XGBoost builds the quantile sketch and pages through the data, so training data can exceed RAM. Combined with GPUs this scales to very large datasets.
Distributed training: xgboost.dask (DaskXGBClassifier, dxgb.train) and xgboost.spark (SparkXGBClassifier) partition data across workers and synchronise histograms with collective communication (AllReduce), producing the same kind of model as single-node training. Ensure partitions are balanced and every worker has data.
CPU threads: n_jobs (or nthread) controls threads; leave cores for data loading. Inference can be accelerated too: booster.inplace_predict(X) skips DMatrix construction and is thread-safe for serving.
Example
import numpy as np
import xgboost as xgb
class Batches(xgb.DataIter):
"""Stream data in chunks, e.g. from many files on disk."""
def __init__(self, n_batches=5):
self.i, self.n = 0, n_batches
self.rng = np.random.default_rng(5)
super().__init__()
def next(self, input_data):
if self.i == self.n:
return False
X = self.rng.normal(size=(20000, 10))
input_data(data=X, label=(X[:, 0] + X[:, 1] > 0).astype(float))
self.i += 1
return True
def reset(self):
self.i = 0
self.rng = np.random.default_rng(5)
dtrain = xgb.QuantileDMatrix(Batches(), max_bin=128) # built batch by batch
print(dtrain.num_row(), dtrain.num_col()) # 100000 10
booster = xgb.train({"objective": "binary:logistic", "max_bin": 128}, dtrain, 50)
X_new = np.array([[2.0, 1.0] + [0.0] * 8])
print(booster.inplace_predict(X_new).round(3)) # [1.] (no DMatrix needed)Remember
device="cuda"enables GPU training with the hist method- QuantileDMatrix and
DataItercut memory and allow data larger than RAM - Use
xgboost.daskorxgboost.sparkfor multi-node training
Watch out for
- Using the removed
tree_method="gpu_hist"/gpu_idparameters from old tutorials.