Adaptive Linear Embedding Bayesian Optimization (ALEBO)

Adaptive Linear Embedding Bayesian Optimization (ALEBO)#

About#

This is an implementation of Adaptive Linear Embeddings (ALEBO) as described in [Letham et al., 2020]. We use the model provided by Ax.

How to run#

Warning

This solver runs in a different conda environment than base.

You can find a conda environment where this solver can run here.

If you have cloned poli-baselines locally:

conda env create --file src/poli_baselines/core/utils/ax/environment.ax.yml
conda activate poli__ax

import numpy as np

from poli.objective_repository import ToyContinuousBlackBox

from poli_baselines.solvers.bayesian_optimization.alebo import (
    ALEBO,
)


f_ackley = ToyContinuousBlackBox(function_name="ackley_function_01", n_dimensions=10)

x0 = np.random.randn(10).reshape(1, -1).clip(-2.0, 2.0)
y0 = f_ackley(x0)

bo_solver = ALEBO(
    black_box=f_ackley,
    x0=x0,
    y0=y0,
    lower_dim=2,  # It's necessary to specify the assumed effective dim.
)

bo_solver.solve(max_iter=10)

See more#

The original reference: Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization.
Taking the human out of the loop is a great tutorial of Bayesian Optimization [Shahriari et al., 2016].
Since poli works mostly on discrete inputs, this baseline is implemented with the intention of optimizing in the latent spaces of deep generative models like Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs) [Gómez-Bombarelli et al., 2018].

References#

If you use this solver, we expect that you cite the following resources:

References as text

[1] Letham, B., Calandra, R., Rai, A., & Bakshy, E. (2020). Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization. Advances in Neural Information Processing Systems, 33, 1546–1558. https://proceedings.neurips.cc/paper/2020/hash/10fb6cfa4c990d2bad5ddef4f70e8ba2-Abstract.html

[2] González-Duque, M., Bartels, S., & Michael, R. (2024). poli: a libary of discrete sequence objectives [Computer software]. MachineLearningLifeScience/poli

References as BibTeX

@inproceedings{Letham:ALEBO:2020,
     title={Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization},
     volume={33},
     url={https://proceedings.neurips.cc/paper/2020/hash/10fb6cfa4c990d2bad5ddef4f70e8ba2-Abstract.html},
     booktitle={Advances in Neural Information Processing Systems},
     publisher={Curran Associates, Inc.},
     author={Letham, Ben and Calandra, Roberto and Rai, Akshara and Bakshy, Eytan},
     year={2020},
     pages={1546–1558}
}



@software{Gonzalez-Duque:poli:2024,
author = {González-Duque, Miguel and Bartels, Simon and Michael, Richard},
month = jan,
title = {{poli: a libary of discrete sequence objectives}},
url = {https://github.com/MachineLearningLifeScience/poli},
version = {0.0.1},
year = {2024}
}