Optuna

Optuna is a framework that allows to finds optimal parameters for a given function. Its primary function is hyperparameter optimization for the machine learning models.

Optuna uses the following terminology for its objects:

• Trial is a signle call of the objective function.

• Study is an optimization session, which is a collection of trials.

• Parameter is a variable whose value is to be optimized, which is typically the loss or accuracy of the model.

import optuna

Parameters

The parameters must be defined in the objective function. Optuna will then assign different values to these parameters and save the corresponding objectvie function outputs. You must use the trial object that is passed to the objective function to define the variables.

The following table lists the variables that can be used in Optuna.

Method	Type of Variable	Description	Key Arguments
trial.suggest_float()	Floating Point (Continuous)	Suggests a floating-point value within a specified range. Can be sampled uniformly or logarithmically.	low, high, log (boolean), step (float or None)
trial.suggest_int()	Integer (Discrete)	Suggests an integer value within a specified range. Supports uniform or logarithmic sampling, and a step size.	low, high, log (boolean), step (int or None)
trial.suggest_categorical()	Categorical	Suggests a value from a predefined list of choices (e.g., strings, integers, booleans).	choices (list or tuple of possible values)
trial.suggest_loguniform()	Floating Point (Continuous, Log)	(Deprecated) Suggests a value from a log-uniform distribution. Use trial.suggest_float(..., log=True).	low, high
trial.suggest_uniform()	Floating Point (Continuous, Uniform)	(Deprecated) Suggests a value from a uniform distribution. Use trial.suggest_float(..., log=False).	low, high
trial.suggest_discrete_uniform()	Floating Point (Discrete/Stepped)	(Deprecated) Suggests a discrete value within a range with a specified step size. Use trial.suggest_float(..., step=...).	low, high, q (step size)

All of these functions take a str as the first argument. This defines the name of the variable for the Optuna, which will be used for the Optuna’s internal logs and outputs.

---

The following cell creates the objective functions with the two most popular types of the parameters: float and categorical. It also it gives them very specific names, so they cannot be confused with anything else in the optuna logs and outputs.

def objective(trial: optuna.Trial) -> float:
    float = trial.suggest_float("this my float", 10, 20)
    categorical = trial.suggest_categorical("this my categorical", ["x", "y", "z"])
    map = {"x": 10, "y": 5, "z": 20}

    return float + map[categorical]

study = optuna.create_study()
study.optimize(objective, n_trials=5)

[I 2025-09-26 13:12:05,809] A new study created in memory with name: no-name-85bb68c6-3425-4573-aa05-830721979098
[I 2025-09-26 13:12:05,811] Trial 0 finished with value: 20.22230158249816 and parameters: {'this my float': 15.222301582498162, 'this my categorical': 'y'}. Best is trial 0 with value: 20.22230158249816.
[I 2025-09-26 13:12:05,812] Trial 1 finished with value: 36.367369465572956 and parameters: {'this my float': 16.367369465572956, 'this my categorical': 'z'}. Best is trial 0 with value: 20.22230158249816.
[I 2025-09-26 13:12:05,813] Trial 2 finished with value: 35.59952057096707 and parameters: {'this my float': 15.599520570967073, 'this my categorical': 'z'}. Best is trial 0 with value: 20.22230158249816.
[I 2025-09-26 13:12:05,814] Trial 3 finished with value: 37.972541254122454 and parameters: {'this my float': 17.972541254122454, 'this my categorical': 'z'}. Best is trial 0 with value: 20.22230158249816.
[I 2025-09-26 13:12:05,815] Trial 4 finished with value: 19.4864897006154 and parameters: {'this my float': 14.4864897006154, 'this my categorical': 'y'}. Best is trial 4 with value: 19.4864897006154.

The outputs of Optuna refer to the names that were specified when they were defined.

study.best_params

{'this my float': 14.4864897006154, 'this my categorical': 'y'}

Study

A study is a set of trials in the Optuna. You can create a study using the optuna.create_study method. The following cell lists the parameters of the create_study method that can be used to configure the study.

The optuna.create_study method is the fundamental function in Optuna used to create a new optimization study, which manages the entire hyperparameter tuning process.

Parameter	Type	Default Value	Description
direction	str	'minimize'	The direction of the optimization. Must be 'minimize' or 'maximize'.
directions	Sequence[str]	None	A sequence of directions (e.g., ['minimize', 'maximize']) for multi-objective optimization. Cannot be specified with direction.
study_name	str	None	The name of the study. If None, a unique name is automatically generated. Essential for loading or resuming a study.
storage	str or BaseStorage	None	Specifies where to store the study’s results (trials).
			- If None, an in-memory storage is used (results are lost after the process ends).
			- A database URL (e.g., 'sqlite:///example.db') is typically used for persistence and distributed optimization.
sampler	BaseSampler	None	The algorithm used to suggest new hyperparameter values for each trial.
			- If None, TPESampler (Tree-structured Parzen Estimator) is used for single-objective, and NSGAIISampler for multi-objective optimization.
pruner	BasePruner	None	The strategy used to stop unpromising trials early (early stopping).
			- If None, MedianPruner is used as the default.
load_if_exists	bool	False	A flag to control behavior if a study with the given study_name already exists in the storage.
			- If False, a DuplicatedStudyError is raised.
			- If True, the existing study is loaded and returned.

After the optimization process, the study object collects all the necessary information. The following table shows the useful attributes of the study object:

Attribute	Type	Description	Applicable for
study_name	str	The unique name given to the study upon creation.	All Studies
direction	StudyDirection	The optimization direction (e.g., 'minimize' or 'maximize').	Single-Objective
directions	Sequence[StudyDirection]	The optimization directions for each objective.	Multi-Objective
best_value	float	The best (minimized or maximized) objective value found across all completed trials.	Single-Objective
best_params	dict	A dictionary containing the hyperparameter names and values of the best trial.	Single-Objective
best_trial	FrozenTrial	The entire best trial object, containing all metadata (value, parameters, start time, etc.).	Single-Objective
best_trials	List[FrozenTrial]	A list of trials that form the Pareto front (non-dominated trials).	Multi-Objective
trials	List[FrozenTrial]	A list of all trials in the study, ordered by trial number.	All Studies
user_attrs	dict	A dictionary to store user-defined metadata that is persistent across the study (e.g., dataset name, model type).	All Studies
system_attrs	dict	A dictionary to store internal system-defined attributes.	All Studies