TopolearnUnderstanding what drives predictions is required for debugging, trust and catching leakage.
LightGBM reports two built-in importances:
importance_type="split"(the scikit-learn wrapper's default forfeature_importances_): how many times a feature is used,"gain": the total loss reduction from its splits, usually more meaningful.
Get them with model.booster_.feature_importance(importance_type="gain"), or set LGBMRegressor(importance_type="gain") so feature_importances_ uses gain. lgb.plot_importance(model, importance_type="gain") plots them. Split counts favour continuous, high-cardinality features that offer many thresholds.
For per-prediction explanations, model.predict(X, pred_contrib=True) returns exact SHAP values computed with the TreeSHAP algorithm: one column per feature plus a final expected-value column. For each row they add up to the raw prediction (log-odds for binary classification). Averages of absolute SHAP values give a consistent global ranking, and the shap library can plot them.
For a model-agnostic check of how much a feature matters for your actual metric, use scikit-learn's permutation_importance on held-out data.
All of these describe the model, not causality, and correlated features split credit unpredictably.
Example
import lightgbm as lgb
import numpy as np
from sklearn.datasets import load_diabetes
X, y = load_diabetes(return_X_y=True, as_frame=True)
model = lgb.LGBMRegressor(n_estimators=200, learning_rate=0.05, num_leaves=8,
verbose=-1).fit(X, y)
split = dict(zip(X.columns, model.booster_.feature_importance("split")))
gain = dict(zip(X.columns, model.booster_.feature_importance("gain").round()))
print(sorted(split, key=split.get, reverse=True)[:3]) # ['bmi', 'bp', 's5']
print(sorted(gain, key=gain.get, reverse=True)[:3]) # ['s5', 'bmi', 'bp']
contrib = model.predict(X.iloc[:2], pred_contrib=True)
print(contrib.shape) # (2, 11) = 10 features + expected value
print(np.allclose(contrib.sum(axis=1), model.predict(X.iloc[:2]))) # TrueRemember
- Prefer gain importance over split counts
pred_contrib=Truereturns per-row SHAP values plus an expected-value column- Validate with permutation importance on held-out data
Watch out for
- Reporting split-count importance as 'how much a feature matters'.