TopolearnStakeholders need to know why a model predicts what it does, and importance analysis catches leakage and bugs.
model.get_feature_importance(type=...) offers several views:
- `PredictionValuesChange` (default for most models): how much predictions change, on average, when a feature's value changes. Fast and needs no data.
- `LossFunctionChange`: how much the loss gets worse without the feature, computed on a dataset you pass; closer to "how useful is this feature" and better for feature selection.
- `ShapValues`: per-row SHAP contributions, computed exactly and quickly for trees.
- `Interaction`: strength of pairwise feature interactions.
prettified=True returns a sorted DataFrame. Importances on categorical features cover the feature as a whole, including its use in combinations.
SHAP values split each prediction into additive contributions: for every row, the expected value plus the sum of the row's SHAP values equals the model's raw prediction (log-odds for classifiers). get_feature_importance(data=pool, type="ShapValues") returns an array with one column per feature plus a last column holding the expected value. The shap package can plot them (shap.summary_plot, waterfall plots) and also accepts the CatBoost model through shap.TreeExplainer.
Interpretation cautions: importance is not causation, correlated features share and split credit unpredictably, and a suspiciously dominant feature often signals target leakage (for example an id or timestamp that encodes the label). Investigate before celebrating.
Example
import numpy as np
import pandas as pd
from catboost import CatBoostClassifier, Pool
rng = np.random.default_rng(0)
X = pd.DataFrame({"income": rng.normal(50, 15, 800),
"age": rng.integers(18, 70, 800),
"noise": rng.normal(0, 1, 800)})
y = ((X["income"] > 55) ^ (X["age"] < 25)).astype(int)
pool = Pool(X, y)
model = CatBoostClassifier(iterations=300, verbose=0, random_seed=0,
allow_writing_files=False).fit(pool)
print(model.get_feature_importance(prettified=True))
# income and age at the top, noise near zero
print(model.get_feature_importance(pool, type="LossFunctionChange").round(3))
shap = model.get_feature_importance(pool, type="ShapValues")
print(shap.shape) # (800, 4): 3 features + expected value
raw = model.predict(pool, prediction_type="RawFormulaVal")
print(np.allclose(shap.sum(axis=1), raw)) # True: contributions add up
inter = model.get_feature_importance(pool, type="Interaction")
print(inter[0][:2]) # strongest interacting pair (indices)Remember
- PredictionValuesChange is the fast default; LossFunctionChange measures usefulness on data
- ShapValues give per-row contributions plus an expected-value column
- SHAP contributions sum to the raw prediction (log-odds for classifiers)
- A dominant feature can signal target leakage
Watch out for
- Treating feature importance as causal effect, or comparing importances of strongly correlated features.