TopolearnThe sampler is the search algorithm; the right one can halve the trials you need.
The sampler decides which parameters each trial gets. Pass one to create_study(sampler=...).
TPESampler(default): Tree-structured Parzen Estimator. Robust, handles conditional and categorical spaces, scales to hundreds or thousands of trials. In Optuna 5 it models parameters jointly (multivariate) by default for single-objective studies.GPSampler: Gaussian-process Bayesian optimization. Very sample-efficient for small budgets (tens of trials) on mostly continuous spaces; heavier per trial and requires PyTorch and SciPy.CmaEsSampler: evolution strategy (needs thecmaespackage), strong for continuous parameters with larger budgets; categorical parameters fall back to independent sampling.RandomSampler: a useful baseline and for broad initial exploration.QMCSampler: quasi-random low-discrepancy points, which cover the space more evenly than random.GridSampler(search_space): exhaustive grid when you truly need every combination.NSGAIISampler/NSGAIIISampler: genetic algorithms for multi-objective problems.
Model-based samplers start with n_startup_trials random trials (10 for TPE) before modelling, so tiny budgets behave like random search.
Pass seed= for reproducible sequences in single-process runs. Parallel runs are inherently non-deterministic because trial completion order varies.
Rules of thumb: under ~100 trials with continuous parameters try GPSampler; otherwise TPE. Compare samplers on your problem with a few seeds before committing.
Example
import optuna
optuna.logging.set_verbosity(optuna.logging.WARNING)
def objective(trial):
x = trial.suggest_float("x", -5, 5)
y = trial.suggest_float("y", -5, 5)
return (x - 1) ** 2 + (y + 2) ** 2
samplers = {
"random": optuna.samplers.RandomSampler(seed=0),
"tpe": optuna.samplers.TPESampler(seed=0),
"cmaes": optuna.samplers.CmaEsSampler(seed=0),
"qmc": optuna.samplers.QMCSampler(seed=0),
}
for name, sampler in samplers.items():
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=60)
print(f"{name:7} {study.best_value:.4f}")
# model-based tpe and cmaes end much closer to 0 than random or qmc
grid = optuna.samplers.GridSampler({"x": [0, 1, 2], "y": [-2, 0]})
study = optuna.create_study(sampler=grid)
study.optimize(objective) # stops by itself after all 6 combinations
print(len(study.trials)) # 6Remember
- TPE is the robust default; GP is sample-efficient for small continuous problems
- CMA-ES suits continuous spaces with bigger budgets; Random/QMC are baselines
- Seed samplers for reproducible single-process runs
Watch out for
- Comparing samplers with a single seed and tiny budget, where differences are mostly noise.