Topolearn

Learn CatBoost

Gradient boosting on decision trees with native categorical feature support

14 lessons · 3 hours · 4 levels · covers CatBoost 1.2

CatBoost is Yandex's open-source gradient boosting library. It handles categorical features natively with ordered target statistics, resists overfitting with ordered boosting, and builds symmetric trees that predict very fast. It gives strong results on tabular data with little tuning and exports models for production use.

Sign in free to track progressOfficial docs ↗

Before you start

CatBoost builds on these; the tree links to the exact lessons you need.

PythonEssentials levelpandasEssentials levelscikit-learnEssentials level

How deep do you want to go?

1. EssentialsRead and write simple code, do the everyday basics4 lessons · 40 min in total
2. PracticalUse it productively at work, with the usual idioms8 lessons · 1 hour in total
3. AdvancedThe less common features, performance and trade-offs11 lessons · 2 hours in total
4. ExpertInternals and the edge cases experts know14 lessons · 3 hours in total

The CatBoost tree

CatBoostYour first CatBoost modelYour first CatBoostmodelCategorical features without encodingCategorical featureswithout encodingThe Pool data structureThe Pool datastructureLoss functions and evaluation metricsLoss functions andevaluation metricsValidation sets and early stoppingValidation sets andearly stoppingThe parameters that matterThe parameters thatmatterCross-validation and hyperparameter searchCross-validation andhyperparameter se…Feature importance and SHAP valuesFeature importanceand SHAP valuesDefining functionsDefining functionsModules and importsModules and importsDataFrameDataFrameCategorical dataCategorical dataDictionariesDictionaries
Essentials and Practical shown (8 of 14 lessons); the full list is below. Click a lesson to read it.

All 14 lessons, in order

First models

  1. Your first CatBoost model · Essentials · 10 min

    Train CatBoostClassifier and CatBoostRegressor with the familiar fit/predict API and read predictions and probabilities.

  2. Loss functions and evaluation metrics · Essentials · 10 min

    Choose loss_function for the task (Logloss, MultiClass, RMSE, MAE, Quantile...) and track other metrics with eval_metric and custom_metric.

Data and features

  1. Categorical features without encoding · Essentials · 12 min

    Pass string or integer categories directly via cat_features and let CatBoost encode them safely.

  2. The Pool data structure · Essentials · 8 min

    Wrap data, labels, categorical, text and weight information in a Pool for training, evaluation and prediction.

  3. Text and embedding features · Advanced · 12 min

    Feed raw text columns via text_features and dense vectors via embedding_features, and control the built-in text processing.

Training and tuning

  1. Validation sets and early stopping · Practical · 10 min

    Use eval_set with early_stopping_rounds and use_best_model to stop at the right number of trees and avoid overfitting.

  2. The parameters that matter · Practical · 12 min

    Know what learning_rate, iterations, depth, l2_leaf_reg, random_strength, bagging_temperature and border_count do, and what to tune first.

  3. Cross-validation and hyperparameter search · Practical · 12 min

    Estimate performance with catboost.cv and search parameters with grid_search, randomized_search or external tuners like Optuna.

  4. Imbalanced and multiclass classification · Advanced · 12 min

    Handle rare positive classes with auto_class_weights, scale_pos_weight and threshold tuning, and evaluate with the right metrics.

Interpretation

  1. Feature importance and SHAP values · Practical · 12 min

    Explain models with PredictionValuesChange, LossFunctionChange and per-prediction SHAP values computed natively.

Algorithm internals

  1. Ordered target statistics and ordered boosting · Advanced · 15 min

    Understand prediction shift and how CatBoost's permutation-based target statistics and ordered boosting avoid it.

  2. Symmetric (oblivious) trees and quantization · Expert · 15 min

    Learn how CatBoost's symmetric trees, feature quantization and leaf estimation work, and when to use other grow policies.

  3. CTR settings and feature combinations · Expert · 12 min

    Control how categorical features become numbers: CTR types, priors, one_hot_max_size and greedy feature combinations.

Production

  1. Saving, exporting and serving models · Expert · 12 min

    Save models in CatBoost's binary format, export to JSON, ONNX, CoreML or C++, and keep training-serving consistency.