Leaf-wise growth and num_leaves

num_leaves is LightGBM's most important parameter, and it behaves differently from max_depth in other libraries.

Most tree libraries grow trees level-wise: split every node at one depth before going deeper. LightGBM grows leaf-wise (best-first): at each step it splits the single leaf with the largest loss reduction, anywhere in the tree. For the same number of leaves this reduces loss more, but it can build deep, lopsided trees that overfit small datasets.

So model capacity is controlled mainly by:

num_leaves (default 31): the maximum number of leaves per tree. This is the key parameter. A level-wise tree of depth d has up to 2^d leaves, so num_leaves=31 is comparable in size to depth 5, but shaped very differently. Keep num_leaves below 2^max_depth when both are set.
max_depth (default -1, unlimited): an extra guard against very deep branches, useful on small data.
min_child_samples (min_data_in_leaf, default 20): minimum rows per leaf. Raising it (50-500 for large data) is one of the most effective ways to curb overfitting.
min_child_weight (minimum hessian sum) and min_split_gain provide similar control.

Together with learning_rate and n_estimators, these define most of the model's behaviour. A typical starting point for medium data: num_leaves 15-63, min_child_samples 20-100, learning_rate 0.05, and early stopping for the number of trees.

Example

import lightgbm as lgb
from sklearn.datasets import make_regression

X, y = make_regression(n_samples=5000, n_features=10, noise=10, random_state=0)

for params in [dict(num_leaves=31),
               dict(num_leaves=31, max_depth=5),
               dict(num_leaves=31, min_child_samples=300)]:
    m = lgb.LGBMRegressor(n_estimators=50, verbose=-1, **params).fit(X, y)
    t = m.booster_.trees_to_dataframe()
    leaves = t[t["left_child"].isna()]
    avg_leaves = leaves.groupby("tree_index").size().mean()
    print(params, round(avg_leaves, 1), int(leaves["node_depth"].max()) - 1)
# {'num_leaves': 31} 31.0 14                    (31 leaves, but 14 levels deep: lopsided)
# {'num_leaves': 31, 'max_depth': 5} 27.3 5      (depth cap limits the branches)
# {'num_leaves': 31, 'min_child_samples': 300} 13.0 12  (big leaves stop splitting early)

Remember

Trees grow leaf-wise: always split the best leaf
num_leaves is the main capacity control; keep it below 2^max_depth
Raise min_child_samples to fight overfitting

Watch out for

Copying max_depth=6 from XGBoost and leaving num_leaves=31, then wondering why trees behave differently.

Check yourself

Your LightGBM model overfits a small dataset. Which change is most directly helpful?

Official docs: lightgbm.readthedocs.io/en/stable/Parameters-Tuning.html

Example

Remember

Watch out for

Check yourself

Builds on

Unlocks