TopolearnOn large data, configuration choices change training time by an order of magnitude.
Threads. n_jobs (num_threads) defaults to all cores (OpenMP). Set it to the number of physical cores; hyper-threads rarely help. When running several models in parallel (cross-validation, searches), give each a share of cores to avoid oversubscription.
Histogram layout. LightGBM chooses between force_row_wise=True (better with many rows, few features, or many threads) and force_col_wise=True (better with many features or few threads) by timing both at start-up; setting one explicitly saves that overhead and silences the related log message.
Data handling. Build an lgb.Dataset once and reuse it across trainings with the same binning; ds.save_binary("train.bin") caches it on disk. free_raw_data=True (default) drops the raw copy after construction. Using float32 input halves memory.
Determinism. Results can vary slightly between runs with multithreading. For reproducibility set seed (and specific seeds like bagging_seed), deterministic=True and force_row_wise=True or force_col_wise=True, accepting some slowdown.
Hardware and scale. device_type="cuda" uses the CUDA implementation (requires a CUDA-enabled build); "gpu" is the older OpenCL version. lightgbm.dask (DaskLGBMClassifier and friends) trains across a Dask cluster with data-parallel histogram aggregation; the CLI and Spark (SynapseML) support distributed training too.
Profile first: often the bottleneck is data loading or a too-large max_bin, not training itself.
Example
import time
import lightgbm as lgb
import numpy as np
rng = np.random.default_rng(6)
X = rng.normal(size=(100_000, 20)).astype(np.float32) # float32 halves memory
y = X[:, 0] - 2 * X[:, 1] + rng.normal(0, 0.5, len(X))
ds = lgb.Dataset(X, label=y, params={"verbosity": -1}, free_raw_data=False).construct()
base = {"objective": "regression", "verbosity": -1, "num_threads": 1, "seed": 7,
"deterministic": True, "force_row_wise": True}
preds = []
for _ in range(2): # reuse the same binned Dataset
t0 = time.perf_counter()
booster = lgb.train(base, ds, num_boost_round=100)
preds.append(booster.predict(X[:1000]))
print(round(time.perf_counter() - t0, 1) < 30) # True
print(np.array_equal(preds[0], preds[1])) # True (deterministic)Remember
- Use physical-core thread counts and avoid nested parallelism
- Set
force_row_wiseorforce_col_wise, reuse Datasets, use float32 deterministic=Trueplus seeds for reproducible runs
Watch out for
- Expecting bit-identical models across machines with different thread counts without deterministic settings.