TopolearnNative handling avoids imputation and one-hot blow-up, and often gives better models.
Missing values. XGBoost treats NaN as missing by default (missing=np.nan). At every split it learns a default direction: during training, rows with a missing value are tried on both sides, and the side that reduces loss more is remembered. At prediction time, missing values follow that branch. So you usually should not impute for XGBoost: missingness itself becomes usable information. If your data encodes missing as a sentinel such as -999 or 0, either convert it to NaN or pass missing=-999.
Categorical features. Columns with pandas category dtype are split natively: XGBoost groups categories into the two sides of a split directly, instead of needing one column per category. In XGBoost 3.x the scikit-learn estimators enable this by default (enable_categorical=True); with the native DMatrix you pass enable_categorical=True yourself. Object/string columns are not accepted: convert them with df[col].astype("category").
Useful controls:
max_cat_to_onehot: below this many categories, use one-vs-rest splits,max_cat_threshold: limits categories per split to curb overfitting on high-cardinality features.
The categories seen during training are stored with the model; at prediction time, make sure the column uses the same categories (re-applying the same CategoricalDtype is the safe way).
Example
import numpy as np
import pandas as pd
from xgboost import XGBRegressor
rng = np.random.default_rng(1)
n = 3000
df = pd.DataFrame({"city": rng.choice(["Oslo", "Rome", "Lima", "Kyiv"], n),
"income": rng.normal(50, 15, n)})
y = df["city"].map({"Oslo": 30, "Rome": 20, "Lima": 10, "Kyiv": 15}) + 0.5 * df["income"]
df.loc[rng.random(n) < 0.2, "income"] = np.nan # 20% missing
df["city"] = df["city"].astype("category") # native categorical
model = XGBRegressor(n_estimators=200, max_depth=4).fit(df, y)
new = pd.DataFrame({"city": pd.Categorical(["Oslo", "Lima"], categories=df["city"].cat.categories),
"income": [60.0, np.nan]})
print(model.predict(new).round(1)) # [59.8 34.9] (true: 60.0 and 10 + mean income term)Remember
- NaN gets a learned default direction at each split; don't impute by reflex
- Use pandas
categorydtype for native categorical splits - Keep the same categories at prediction time
Watch out for
- Encoding missing values as 0 or -1, which XGBoost then treats as a real value.