Topolearn

Learn LightGBM

Fast, memory-efficient gradient boosting with leaf-wise trees

14 lessons · 3 hours · 4 levels · covers LightGBM 4.x

LightGBM is Microsoft's gradient boosting framework, built for speed and large datasets. It grows trees leaf-wise on histogram-binned features, handles categorical features and missing values natively, and supports classification, regression, ranking and custom objectives. With a scikit-learn compatible API and a native API, it is a go-to choice for tabular data from Kaggle to production.

Sign in free to track progressOfficial docs ↗

Before you start

LightGBM builds on these; the tree links to the exact lessons you need.

PythonEssentials levelscikit-learnEssentials level

How deep do you want to go?

1. EssentialsRead and write simple code, do the everyday basics4 lessons · 39 min in total
2. PracticalUse it productively at work, with the usual idioms8 lessons · 1 hour in total
3. AdvancedThe less common features, performance and trade-offs11 lessons · 2 hours in total
4. ExpertInternals and the edge cases experts know14 lessons · 3 hours in total

The LightGBM tree

LightGBMLightGBM with the scikit-learn APILightGBM with thescikit-learn APILeaf-wise growth and num_leavesLeaf-wise growth andnum_leavesCategorical features and missing valuesCategorical featuresand missing valuesEarly stopping with callbacksEarly stopping withcallbacksThe native API: Dataset, train and cvThe native API:Dataset, train and cvRegularisation and tuningRegularisation andtuningFeature importance and SHAP contributionsFeature importanceand SHAP contribut…Saving, loading and serving modelsSaving, loading andserving modelsThe estimator API: fit, predict, transformThe estimator API: fit,predict, transformTrain/test splitTrain/test splitDecision treesDecision treesAccuracy and the confusion matrixAccuracy and theconfusion matrixHyperparameter searchHyperparametersearchModel inspection and feature importanceModel inspection andfeature importance
Essentials and Practical shown (8 of 14 lessons); the full list is below. Click a lesson to read it.

All 14 lessons, in order

Getting started

  1. LightGBM with the scikit-learn API · Essentials · 9 min

    Train LGBMClassifier and LGBMRegressor like any scikit-learn estimator.

  2. Leaf-wise growth and num_leaves · Essentials · 11 min

    Understand how leaf-wise tree growth works and control it with num_leaves, max_depth and min_child_samples.

  3. Categorical features and missing values · Essentials · 10 min

    Use pandas category columns for native categorical splits and let LightGBM handle NaN.

Training and tuning

  1. Early stopping with callbacks · Essentials · 9 min

    Stop training when a validation metric stops improving, using eval_set and the early_stopping callback.

  2. The native API: Dataset, train and cv · Practical · 12 min

    Use lgb.Dataset, lgb.train and lgb.cv with parameter dicts for full control and faster repeated training.

  3. Regularisation and tuning · Practical · 12 min

    Tune bagging, feature fraction, L1/L2 penalties and leaf constraints in a sensible order.

  4. Feature importance and SHAP contributions · Practical · 11 min

    Read split and gain importances and compute per-prediction SHAP values with pred_contrib.

  5. Saving, loading and serving models · Practical · 8 min

    Save boosters as text or JSON, reload them for prediction, and keep serving consistent with training.

Objectives and special tasks

  1. Objectives, metrics and imbalanced data · Advanced · 12 min

    Pick objectives for counts, quantiles and skewed targets, add custom metrics, and handle class imbalance.

  2. Learning to rank with LGBMRanker · Advanced · 11 min

    Train lambdarank models with query groups, label gains and NDCG evaluation.

  3. DART, GOSS, random forests and linear trees · Advanced · 12 min

    Know when to use LightGBM's alternative boosting modes and sampling strategies.

Internals and performance

  1. Histograms, EFB and GOSS internals · Expert · 14 min

    Learn how binning, exclusive feature bundling and histogram subtraction make LightGBM fast.

  2. Custom objective functions · Expert · 12 min

    Train on your own loss by returning gradients and hessians, in both the scikit-learn and native APIs.

  3. Performance, determinism and scale · Expert · 12 min

    Tune threads, row/column-wise histograms, Dataset reuse, GPU/CUDA builds and Dask for big workloads.