TopolearnCross-validation gives reliable estimates on small data and tells you how many iterations to use when retraining.
`catboost.cv(pool, params, fold_count=5, ...)` runs k-fold cross-validation natively and returns a pandas DataFrame with the mean and standard deviation of each metric per iteration, for train and test folds. Combined with early_stopping_rounds, the row with the best mean test score tells you the iteration count to use when retraining on all the data. Pass stratified=True for classification and shuffle=True unless the data order matters; for time series use type="TimeSeries".
For hyperparameters, models have built-in search methods:
model.grid_search(param_grid, X, y, cv=3)tries every combinationmodel.randomized_search(param_distributions, X, y, n_iter=20)samples combinations
Both evaluate each candidate with cross-validation (or a train/test split), refit the model with the best parameters by default (refit=True), and return a dict containing params and cv_results.
For larger searches, Bayesian tuners such as Optuna or Hyperopt explore more efficiently; inside the objective, train with an eval set and early stopping and return the validation score. scikit-learn's GridSearchCV also works because CatBoost estimators follow the sklearn interface.
Tune a few impactful parameters (depth, l2_leaf_reg, learning_rate, random_strength, bagging_temperature) rather than everything. Small datasets show high variance across folds: compare the mean differences with the standard deviation before declaring a winner.
Example
from catboost import CatBoostClassifier, Pool, cv
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=1500, n_features=15, flip_y=0.05, random_state=0)
params = {"loss_function": "Logloss", "eval_metric": "AUC", "iterations": 1000,
"learning_rate": 0.05, "random_seed": 0, "logging_level": "Silent"}
scores = cv(Pool(X, y), params, fold_count=5, stratified=True, shuffle=True,
early_stopping_rounds=50)
best = scores["test-AUC-mean"].idxmax()
print(best, round(scores.loc[best, "test-AUC-mean"], 3),
round(scores.loc[best, "test-AUC-std"], 3)) # e.g. 182 0.967 0.003
model = CatBoostClassifier(iterations=200, logging_level="Silent", random_seed=0,
allow_writing_files=False)
result = model.grid_search({"depth": [4, 6], "l2_leaf_reg": [1, 5]}, X, y,
cv=3, verbose=False, plot=False)
print(result["params"]) # e.g. {'depth': 4, 'l2_leaf_reg': 1}
print(model.is_fitted()) # True: refit with the best params
final = CatBoostClassifier(**{**params, "iterations": int(best) + 1}).fit(X, y)Remember
- catboost.cv returns per-iteration mean/std metrics across folds
- The best mean iteration sets iterations when retraining on all data
- grid_search/randomized_search evaluate with CV and refit the best model
- Use Optuna or similar for larger searches; tune a few key parameters
Watch out for
- Declaring one parameter set better when its advantage is smaller than the fold-to-fold standard deviation.