Hvarfner’s Vanilla Bayesian Optimization

Hvarfner’s Vanilla Bayesian Optimization#

About#

This is an implementation of vanilla Bayesian Optimization as described in [Hvarfner et al., 2024]. We adapt the official GitHub implementation, which relies on Ax, to our interface.

The core differences with standard Bayesian Optimization are threefold: they include a prior on the lengthscales that scales with the dimensionality of the inputs, the use of log-expected improvement instead of the usual expected improvement, and keeping the outputscale constant at 1.

How to run#

Warning

This solver runs in a different conda environment than base.

You can find a conda environment where this solver can run here.

If you have cloned poli-baselines locally:

conda env create --file src/poli_baselines/core/utils/ax/environment.ax.yml
conda activate poli__ax

import numpy as np

from poli.objective_repository import ToyContinuousBlackBox

from poli_baselines.solvers.bayesian_optimization.vanilla_bo_hvarfner import (
    VanillaBOHvarfner,
)


f_ackley = ToyContinuousBlackBox(function_name="ackley_function_01", n_dimensions=2)

x0 = np.random.randn(2).reshape(1, -1).clip(-2.0, 2.0)
y0 = f_ackley(x0)

bo_solver = VanillaBOHvarfner(
    black_box=f_ackley,
    x0=x0,
    y0=y0,
    bounds=(-4.0, 4.0),
    noise_std=0.0
)

bo_solver.solve(max_iter=10)

See more#

The original reference for this solver: Vanilla Bayesian Optimization Performs Great in High Dimensions.
Taking the human out of the loop is a great tutorial of Bayesian Optimization [Shahriari et al., 2016].
Since poli works mostly on discrete inputs, this baseline is implemented with the intention of optimizing in the latent spaces of deep generative models like Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs) [Gómez-Bombarelli et al., 2018].

References#

If you use this solver, we expect that you cite the following resources:

References as text

[1] Hvarfner, C., Hellsten, E. O., & Nardi, L. (2024). Vanilla Bayesian Optimization Performs Great in High Dimensions (arXiv:2402.02229). arXiv. https://doi.org/10.48550/arXiv.2402.02229

[2] González-Duque, M., Bartels, S., & Michael, R. (2024). poli: a libary of discrete sequence objectives [Computer software]. MachineLearningLifeScience/poli

References as BibTeX

@article{Hvarfner:VanillaBO:2024,
     title={Vanilla Bayesian Optimization Performs Great in High Dimensions},
     url={http://arxiv.org/abs/2402.02229},
     DOI={10.48550/arXiv.2402.02229},
     note={arXiv:2402.02229 [cs, stat]},
    number={arXiv:2402.02229},
    publisher={arXiv},
    author={Hvarfner, Carl and Hellsten, Erik Orm and Nardi, Luigi},
    year={2024},
    month=feb
}


@software{Gonzalez-Duque:poli:2024,
author = {González-Duque, Miguel and Bartels, Simon and Michael, Richard},
month = jan,
title = {{poli: a libary of discrete sequence objectives}},
url = {https://github.com/MachineLearningLifeScience/poli},
version = {0.0.1},
year = {2024}
}