What is Bayesian optimization?

Bayesian optimization is a sequential design strategy for global optimization of black-box functions that are expensive to evaluate. It is commonly used to optimize parameters of machine learning algorithms, tune hyperparameters, and in other fields where evaluating the objective function is costly or time-consuming. It works by constructing a probabilistic model of the objective function and using it to decide where to sample next, balancing exploration (sampling in uncertain regions) and exploitation (sampling where the model predicts high values).