The native API: Dataset, train and cv

LightGBM · Practical · 12 min

The native API exposes every parameter, reuses binned data across runs and is what the wrapper uses inside.

The native API works with a Dataset and a parameter dict:

  • lgb.Dataset(X, label=y, weight=w, categorical_feature=..., free_raw_data=True) holds data. The binning ("construction") happens lazily on first use and can be reused across many trainings. A validation set should reference the training set: lgb.Dataset(X_val, y_val, reference=dtrain), so it uses the same bins.
  • lgb.train(params, dtrain, num_boost_round, valid_sets=[dval], callbacks=[...]) returns a Booster.
  • booster.predict(X) returns probabilities for binary objectives (no separate predict_proba), raw_score=True gives margins, and num_iteration=booster.best_iteration selects trees.
  • lgb.cv(params, dtrain, nfold=5, stratified=True, callbacks=[lgb.early_stopping(50)]) returns a dict of mean and stdv metric lists; their length is the best number of rounds. return_cvbooster=True returns the fold models.

Parameters use the canonical names from the parameter docs: objective, metric, learning_rate, num_leaves, min_data_in_leaf, feature_fraction, bagging_fraction, bagging_freq, lambda_l2, verbosity. Many have aliases (the scikit-learn wrapper's colsample_bytree is feature_fraction). Don't pass the same parameter under two names.

Dataset-level parameters such as max_bin are fixed at construction; changing them requires a new Dataset.

Example

import lightgbm as lgb
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_val, y_tr, y_val = train_test_split(X, y, stratify=y, random_state=0)
dtrain = lgb.Dataset(X_tr, label=y_tr)
dval = lgb.Dataset(X_val, label=y_val, reference=dtrain)    # same bins

params = {"objective": "binary", "metric": "auc", "learning_rate": 0.05,
          "num_leaves": 15, "feature_fraction": 0.8, "verbosity": -1}
cv = lgb.cv(params, dtrain, num_boost_round=2000, nfold=5, seed=0,
            callbacks=[lgb.early_stopping(50, verbose=False)])
best_rounds = len(cv["valid auc-mean"])
print(best_rounds, round(cv["valid auc-mean"][-1], 4))    # 154 0.9952

booster = lgb.train(params, dtrain, num_boost_round=best_rounds, valid_sets=[dval])
print(booster.predict(X_val[:3]).round(3))                # [0.999 0.    0.   ]

Remember

  • lgb.Dataset + params dict + lgb.train -> Booster
  • Validation Datasets need reference=dtrain
  • lgb.cv with early stopping finds the number of rounds

Watch out for

  • Passing a parameter twice under different aliases, for example colsample_bytree and feature_fraction.

Check yourself

  1. Why create the validation set with lgb.Dataset(X_val, y_val, reference=dtrain)?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: lightgbm.readthedocs.io/en/stable/Python-Intro.html