TopolearnThese three objects are the whole Optuna mental model; everything else builds on them.
Optuna has three core concepts:
- The objective is a function you write. It receives a
trial, asks it for parameter values, evaluates something (usually trains and validates a model) and returns a number. - A trial is one evaluation of the objective. Calls such as
trial.suggest_float("lr", 1e-5, 1e-1, log=True)both define the search space and return a value chosen by the sampler. - A study is the optimization session.
optuna.create_study(direction="minimize")creates it (use"maximize"for accuracy-like scores), andstudy.optimize(objective, n_trials=100)runs trials.
Afterwards:
study.best_paramsis a dict of the best parameter values,study.best_valuethe best objective value,study.best_trialthe fullFrozenTrialwith number, params, user attributes and timing.
Optuna logs one INFO line per trial. Silence it with optuna.logging.set_verbosity(optuna.logging.WARNING).
The default sampler is TPE, a Bayesian method that learns from finished trials which regions look promising, so later trials concentrate there. That is why Optuna usually beats grid and random search for the same budget.
Make the objective deterministic where possible (fixed data split, seeds), otherwise noise can make a lucky trial look best.
Example
import optuna
optuna.logging.set_verbosity(optuna.logging.WARNING)
def objective(trial):
x = trial.suggest_float("x", -10, 10)
y = trial.suggest_int("y", -5, 5)
return (x - 2) ** 2 + (y + 1) ** 2 # minimum at x=2, y=-1
study = optuna.create_study(direction="minimize",
sampler=optuna.samplers.TPESampler(seed=0))
study.optimize(objective, n_trials=60)
print(study.best_params) # close to {'x': 2.0, 'y': -1}
print(round(study.best_value, 2)) # close to 0
print(len(study.trials)) # 60Remember
- The objective takes a trial, suggests parameters and returns a score
create_study(direction=...)thenoptimize(objective, n_trials=...)- Read
best_params,best_valueandbest_trialafterwards
Watch out for
- Forgetting direction='maximize' when returning accuracy, so Optuna searches for the worst model.