TopolearnNative handling is faster and often more accurate than one-hot encoding or imputing.
Categorical features. LightGBM splits categorical features natively: for a categorical column it sorts categories by their gradient statistics and finds the best partition into two groups, which is much better than one-hot encoding for high-cardinality features. Two ways to declare them:
- pandas columns with
categorydtype are detected automatically (categorical_feature="auto"), - or pass
categorical_feature=["city", ...](names or indices) tofit/lgb.Dataset; those columns must contain non-negative integers.
Category codes are stored with the model, and at prediction time pandas categories are mapped consistently as long as the column has the same categories; re-use the training CategoricalDtype to be safe. Controls against overfitting rare categories: min_data_per_group (default 100), cat_smooth, cat_l2 and max_cat_threshold. With very many rare categories, consider grouping them first.
Missing values. NaN is treated as missing by default (use_missing=True): each split learns which side missing values go to. Zeros can also be treated as missing with zero_as_missing=True, useful for sparse data where absent means unknown. Don't impute by reflex: the learned direction can use missingness as information.
Object (string) columns are rejected; convert with astype("category").
Example
import lightgbm as lgb
import numpy as np
import pandas as pd
rng = np.random.default_rng(0)
n = 5000
df = pd.DataFrame({"store": rng.choice([f"s{i}" for i in range(50)], n),
"temp": rng.normal(20, 8, n)})
effect = {f"s{i}": rng.normal(0, 10) for i in range(50)}
y = df["store"].map(effect) + 0.5 * df["temp"] + rng.normal(0, 2, n)
df.loc[rng.random(n) < 0.1, "temp"] = np.nan # 10% missing
df["store"] = df["store"].astype("category") # native categorical
model = lgb.LGBMRegressor(n_estimators=300, learning_rate=0.05, verbose=-1).fit(df, y)
new = pd.DataFrame({"store": pd.Categorical(["s3", "s7"], categories=df["store"].cat.categories),
"temp": [25.0, np.nan]})
print(model.predict(new).round(1)) # [6.1 1.2]
print(round(effect["s3"] + 0.5 * 25, 1)) # 5.7 (true mean for the first row)Remember
- pandas
categorycolumns are used as native categorical features - Each split learns where NaN goes; avoid imputing by default
- Keep the same categories at prediction time
Watch out for
- Passing integer-coded categories without declaring them, so they are split as ordered numbers.