TopolearnNative categorical handling is CatBoost's signature feature and removes most manual one-hot or target encoding work.
Most boosting libraries need categories turned into numbers first. One-hot encoding explodes high-cardinality columns such as city or product id, and naive target encoding (replacing a category with its mean target) leaks the label and overfits.
CatBoost does the encoding itself. Tell it which columns are categorical with `cat_features`, by column names or indices:
model.fit(X, y, cat_features=["city", "device"])You can also set cat_features in the constructor. Values must be strings or integers. Floats are rejected, so a numeric code column read as float needs .astype(int) or .astype(str); missing values in categorical columns must be filled first, for example with "missing", while numeric columns may contain NaN.
Internally CatBoost:
- one-hot encodes columns with very few values (up to
one_hot_max_size, small by default), - converts others to ordered target statistics: each row's category value is replaced by a smoothed average of the target over earlier rows in a random permutation, so a row's own label never leaks into its feature,
- builds combinations of categorical features (such as city x device) greedily during training.
Pandas category dtype columns are fine too, but you still list them in cat_features. At prediction time pass the raw categories in the same columns; unseen categories are handled gracefully.
Example
import pandas as pd
from catboost import CatBoostClassifier
df = pd.DataFrame({
"city": ["Paris", "Lyon", "Paris", "Nice", "Lyon", "Paris", "Nice", "Lyon"] * 25,
"device": ["ios", "android", "web", "ios", "web", "android", "web", "ios"] * 25,
"visits": [3, 1, 5, 2, 8, 1, 4, 6] * 25,
"bought": [1, 0, 1, 0, 1, 0, 0, 1] * 25,
})
X, y = df.drop(columns="bought"), df["bought"]
model = CatBoostClassifier(iterations=100, verbose=0, allow_writing_files=False)
model.fit(X, y, cat_features=["city", "device"]) # no encoding needed
print(model.get_cat_feature_indices()) # [0, 1]
new = pd.DataFrame({"city": ["Paris", "Berlin"], "device": ["ios", "web"],
"visits": [4, 2]})
print(model.predict_proba(new)[:, 1].round(2)) # Berlin was never seen: still works
bad = X.copy(); bad.loc[0, "city"] = None
try:
model.fit(bad, y, cat_features=["city", "device"])
except Exception as e:
print(type(e).__name__) # CatBoostError: fill NaN firstRemember
- List categorical columns in cat_features (names or indices)
- Categorical values must be str or int, with missing values filled
- CatBoost uses ordered target statistics instead of leaky target encoding
- Unseen categories at prediction time are handled
Watch out for
- Passing a categorical column containing NaN or floats, which raises an error.