TopolearnWhen training is controlled by another framework or system, you need Optuna without an objective function.
study.optimize assumes Optuna calls your objective. The ask-and-tell interface inverts control:
trial = study.ask() # new trial
lr = trial.suggest_float("lr", 1e-5, 1e-1, log=True)
score = run_somewhere(lr) # your loop, job queue, external system
study.tell(trial, score) # or state=TrialState.PRUNED / FAILThis fits training scripts that cannot be wrapped in a function, batch evaluation of several trials at once, or optimization where results arrive asynchronously. You can also pass a fixed distribution dict to study.ask({"lr": FloatDistribution(1e-5, 1e-1, log=True)}) to define the space up front.
Integrations live in the separate optuna-integration package (pip install optuna-integration[...]):
LightGBMTunerCVtunes LightGBM's key parameters stepwise.- Pruning callbacks for XGBoost, LightGBM, PyTorch Lightning, Keras and others call
trial.report/should_prunefor you from inside the framework's training loop. MLflowCallbackandWeightsAndBiasesCallbacklog every trial to your tracker.
Other frameworks integrate the other way around: Hugging Face Trainer.hyperparameter_search(backend="optuna") and Ray Tune's OptunaSearch use Optuna as their search engine.
Whatever the interface, the study, samplers, pruners and storage behave identically.
Example
import optuna
from optuna.distributions import FloatDistribution, IntDistribution
optuna.logging.set_verbosity(optuna.logging.WARNING)
study = optuna.create_study(direction="maximize",
sampler=optuna.samplers.TPESampler(seed=0))
space = {"lr": FloatDistribution(1e-4, 1e-1, log=True),
"batch": IntDistribution(16, 128, log=True)}
def external_training(lr, batch): # stand-in for any training system
return 0.9 - abs(lr - 0.01) * 5 - abs(batch - 64) / 1000
for round_ in range(5): # e.g. launch 4 jobs at a time
trials = [study.ask(space) for _ in range(4)]
for t in trials:
score = external_training(t.params["lr"], t.params["batch"])
study.tell(t, score)
print(len(study.trials), round(study.best_value, 3)) # 20 and a value near 0.9Remember
ask()creates a trial,tell(trial, value)finishes it- Pass a distribution dict to
askto define the space without suggest calls - optuna-integration provides pruning callbacks and tracker callbacks
Watch out for
- Forgetting to call tell for an asked trial, leaving it RUNNING and confusing samplers and counts.