TopolearnResearch ideas, domain-specific search strategies and hard constraints all require going beyond built-ins.
Custom samplers subclass optuna.samplers.BaseSampler and implement three methods:
infer_relative_search_space(study, trial): which parameters to sample jointly (return{}to sample everything independently),sample_relative(study, trial, search_space): joint sampling for that space, returning a dict,sample_independent(study, trial, param_name, param_distribution): one parameter at a time, for parameters outside the relative space (for example ones discovered during define-by-run).
Read history with study.get_trials(deepcopy=False, states=...). Many custom samplers delegate sample_independent to a RandomSampler.
Custom pruners subclass BasePruner and implement prune(study, trial) -> bool, using trial.intermediate_values and other trials' values. They plug into should_prune unchanged.
Constraints: TPE, NSGA-II/III and GP samplers accept constraints_func(frozen_trial) -> sequence of floats, where a value of 0 or less means satisfied. Store the constraint quantities as user attributes inside the objective and read them back in the function. Infeasible trials still complete but are deprioritized, and best_trials (multi-objective) or your own filtering returns feasible ones.
Ablation and reuse: optuna.samplers.PartialFixedSampler fixes some parameters while another sampler handles the rest, and optuna.copy_study or add_trials move history between studies, for example to warm-start a related problem.
Example
import numpy as np
import optuna
optuna.logging.set_verbosity(optuna.logging.WARNING)
class LocalSearchSampler(optuna.samplers.BaseSampler):
# Sample near the best trial so far; fall back to random sampling.
def __init__(self, seed=None):
self._random = optuna.samplers.RandomSampler(seed=seed)
self._rng = np.random.default_rng(seed)
def infer_relative_search_space(self, study, trial):
return {}
def sample_relative(self, study, trial, search_space):
return {}
def sample_independent(self, study, trial, name, dist):
done = study.get_trials(deepcopy=False, states=(optuna.trial.TrialState.COMPLETE,))
if len(done) < 5 or not isinstance(dist, optuna.distributions.FloatDistribution):
return self._random.sample_independent(study, trial, name, dist)
center = study.best_params[name]
width = (dist.high - dist.low) * 0.05
return float(min(max(self._rng.normal(center, width), dist.low), dist.high))
study = optuna.create_study(sampler=LocalSearchSampler(seed=0))
study.optimize(lambda t: (t.suggest_float("x", -10, 10) - 4.2) ** 2, n_trials=60)
print(round(study.best_params["x"], 2)) # close to 4.2Remember
- BaseSampler: infer_relative_search_space, sample_relative, sample_independent
- BasePruner: implement prune(study, trial) using intermediate values
- constraints_func returns values <= 0 for satisfied constraints
Watch out for
- Returning values outside the distribution's bounds from a custom sampler, which Optuna rejects.