Benchmarking HDBO HDBO A review of high-dimensional Bayesian optimization

A review of high-dimensional Bayesian optimization

We have collected a list of high-dimensional Bayesian optimization methods and their implementations (at least those that we could find) in the following table. If you know of any other methods or implementations that should be included, please let us know by reaching out to us on GitHub.

Method	Date of publishing	Implementation	Title	Family
CoRel	April 26, 2024		A Continuous Relaxation for Discrete Bayesian Optimization	Structured Spaces
Expected Coordinate Improvement (ECI)	April 18, 2024		Expected Coordinate Improvement for High-Dimensional Bayesian Optimization	Variable selection
MAVE-BO	March 8, 2024	n/a	An Adaptive Dimension Reduction Estimation Method for High-dimensional Bayesian Optimization	Linear embeddings
Dual-Space Optimization (DSO)	February 27, 2024	n/a	Dual-Space Optimization: Improved Molecule Sequence Design by Latent Prompt Transformer	Non-linear embeddings
Vanilla BO is great	February 25, 2024		Vanilla Bayesian Optimization Performs Great in High Dimensions	Other
CMA-BO	February 5, 2024		High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategy	Other
Standard BO	February 5, 2024		Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization	Other
AIBO	January 24, 2024		Unleashing the Potential of Acquisition Functions in High-Dimensional Bayesian Optimization	Other
PG-LBO	December 28, 2023		PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance	Non-linear embeddings
LaMBO-2	December 12, 2023		Protein Design with Guided Discrete Diffusion	Non-linear embeddings
GAUCHE	December 10, 2023		GAUCHE: A Library for Gaussian Processes in Chemistry	Structured Spaces
CoBO	December 10, 2023		Advancing Bayesian Optimization via Learning Correlated Latent Space	Non-linear embeddings
GTBO	October 5, 2023		High-dimensional Bayesian Optimization with Group Testing	Variable selection
RTDK-BO	October 5, 2023	n/a	RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernels	Non-linear embeddings
Bounce	July 2, 2023		Bounce: a Reliable Bayesian Optimization Algorithm for Combinatorial and Mixed Spaces	Linear embeddings, Structured Spaces, Trust regions
RDUCB	May 29, 2023		Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?	Additive models
TSBO	May 4, 2023	n/a	High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling	Other
SCoreBO	April 21, 2023		Self-Correcting Bayesian Optimization through Bayesian Active Learning	Other
BODi	March 3, 2023		Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings	Non-linear embeddings, Structured Spaces
MPD	January 16, 2023		Local Bayesian optimization via maximizing probability of descent	Gradient information
AntBO	January 3, 2023		Toward real-world automated antibody design with combinatorial Bayesian optimization	Structured Spaces
BAxUS	November 28, 2022		Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces	Linear embeddings, Trust regions
MahalanobisBatchBO	November 2, 2022		Linear Embedding-based High-dimensional Batch Bayesian Optimization without Reconstruction Mappings	Linear embeddings
MCTS-VS	October 31, 2022		Monte Carlo Tree Search based Variable Selection for High Dimensional Bayesian Optimization	Variable selection
ROBOT	October 20, 2022		Discovering Many Diverse Solutions with Bayesian Optimization	Trust regions
PR (Prob. reparametrization)	October 18, 2022		Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization	Structured Spaces
TREGO	October 10, 2022	n/a	TREGO: a trust-region framework for efficient global optimization	Trust regions
CombBO + Random mapping functions	August 2, 2022	n/a	Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytopes	Linear embeddings, Structured Spaces
LaMBO	July 22, 2022		Accelerating Bayesian Optimization for Protein Design with Denoising Autoencoders	Non-linear embeddings
VAE-BO w. Inv	July 17, 2022	n/a	High-Dimensional Bayesian Optimization with Invariance	Non-linear embeddings
Scalable FOBO	June 16, 2022		Scalable First-Order Bayesian Optimization via Structured Automatic Differentiation	Gradient information
KPCA-BO	April 28, 2022		High Dimensional Bayesian Optimization with Kernel Principal Component Analysis	Non-linear embeddings
LOL-BO	January 28, 2022		Local Latent Space Bayesian Optimization over Structured Inputs	Non-linear embeddings
VAE + Uncertainty censoring	December 6, 2021		Improving black-box optimization in VAE latent space using decoder uncertainty	Non-linear embeddings, Structured Spaces
LADDER	December 6, 2021		Combining latent space and structured kernels for Bayesian optimization over combinatorial spaces	Non-linear embeddings, Structured Spaces
GaBO	November 22, 2021		Bayesian Optimization Meets Riemannian Manifolds in Robot Learning	Structured Spaces
GIBO	November 22, 2021		Local policy search with Bayesian optimization	Gradient information
VAEs & Deep Metric Learning	November 1, 2021		High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning	Non-linear embeddings
MORBO (multi-objective)	September 22, 2021		Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces	Trust regions
HyBO	July 18, 2021		Bayesian Optimization over Hybrid Spaces	Structured Spaces
CASMOPOLITAN	June 18, 2021		Think Global and Act Local: Bayesian Optimisation for Categorical and Mixed Search Spaces	Trust regions
SAASBO	June 10, 2021		High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces.	Variable selection
Add-GP-UCB using Trees	May 1, 2021		High-Dimensional Bayesian Optimization via Tree-Structured Additive Models	Additive models
AlgFOO	March 18, 2021	n/a	Significance of Gradient Information in Bayesian Optimization	Gradient information
Prabuchandran’s FOBO	March 1, 2021	n/a	Novel First Order Bayesian Optimization with an Application to Reinforcement Learning	Gradient information
HEBO	December 7, 2020		HEBO: Heteroscedastic Evolutionary Bayesian Optimisation	Other
HD-GaBO	December 6, 2020		High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds	Structured Spaces
ALEBO	December 6, 2020		Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization	Linear embeddings
TRPBO	November 21, 2020	n/a	A Trust-Region Parallel Bayesian Optimization Method for Simulation-Driven Antenna Design	Trust regions
VAEs and weighted retraining	October 25, 2020		Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining	Non-linear embeddings
BORING	October 5, 2020	n/a	Parameter Optimization using high-dimensional Bayesian Optimization	Linear embeddings
BOSS	October 2, 2020		BOSS: Bayesian Optimization over String Spaces	Structured Spaces
Closed-form ASMs	September 22, 2020		Sequential Learning of Active Subspaces	Linear embeddings
MGPC-BO	September 1, 2020		High-dimensional Bayesian optimization using low-dimensional feature spaces	Non-linear embeddings
PCA-BO	July 2, 2020		High Dimensional Bayesian Optimization Assisted by Principal Component Analysis	Linear embeddings
CoCaBo	June 12, 2020		Bayesian Optimisation over Multiple Continuous and Categorical Inputs	Structured Spaces
RPA-GP	June 12, 2020		Randomly Projected Additive Gaussian Processes for Regression	Additive models
SILBO	May 29, 2020		Semi-supervised Embedding Learning for High-dimensional Bayesian Optimization	Linear embeddings
Amortized Bayesian Optimization	May 27, 2020		Amortized Bayesian Optimization over Discrete Spaces	Structured Spaces
G.M. & Lobato	March 1, 2020	n/a	Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes	Structured Spaces
Quantile-GP BO	February 1, 2020	n/a	High-dimensional Bayesian optimization with projections using quantile Gaussian processes	Linear embeddings
COMBO	December 8, 2019		Combinatorial Bayesian Optimization using the Graph Cartesian Product	Additive models, Structured Spaces, Variable selection
TuRBO	December 8, 2019		Scalable Global Optimization via Local Bayesian Optimization	Trust regions
REMBO 2.0	October 18, 2019	n/a	On the choice of the low-dimensional domain for global optimization via random embeddings	Linear embeddings
SIR/SDR	July 21, 2019		High Dimensional Bayesian Optimization via Supervised Dimension Reduction	Linear embeddings
LineBO	June 10, 2019		Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces	Linear embeddings, Variable selection
Active manifolds	May 24, 2019	n/a	Active Manifolds: A non-linear analogue to Active Subspaces	Non-linear embeddings
HeSBO	May 24, 2019		A Framework for Bayesian Optimization in Embedded Subspaces	Linear embeddings
Deep GPs (single objective)	May 7, 2019	n/a	BAYESIAN OPTIMIZATION USING DEEP GAUSSIAN PROCESSES	Non-linear embeddings
MercBO	February 2, 2019		Mercer Features for Efficient Combinatorial Bayesian Optimization	Structured Spaces
SOLID	January 23, 2019	n/a	Sequential Optimization in Locally Important Dimensions	Variable selection
Deep GPs (multiobjective)	January 1, 2019	n/a	Multi-objective optimization using Deep Gaussian Processes: Application to Aerospace Vehicle Design	Non-linear embeddings
Quadrature Fourier Features	December 3, 2018	n/a	Efficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features	Additive models
D-SKIP	December 3, 2018		Scaling Gaussian Process Regression with Derivatives	Gradient information, Linear embeddings
Attribute Adjustment	August 6, 2018	n/a	Bayesian optimization and attribute adjustment	Non-linear embeddings
BOCS	July 10, 2018		Bayesian Optimization of Combinatorial Structures	Structured Spaces
SG-VAE	July 3, 2018	n/a	Structured Variationally Auto-encoded Optimization	Non-linear embeddings, Structured Spaces
G-Add-GP-UCB	April 9, 2018	n/a	High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups	Additive models
Ensemble BO	March 31, 2018		Batched Large-scale Bayesian Optimization in High-dimensional Spaces	Additive models
BOCK: cylindrical kernels	March 3, 2018		BOCK : Bayesian Optimization with Cylindrical Kernels	Structured Spaces
LSBO	February 28, 2018		Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules	Non-linear embeddings
FOBO	December 6, 2017	n/a	Do we need “Harmless” Bayesian Optimization and “First-Order” Bayesian Optimization?	Gradient information
d-KG	December 4, 2017		Bayesian Optimization with Gradients	Gradient information
HDBO using dropout	August 19, 2017	n/a	High dimensional Bayesian optimization using dropout	Variable selection
Elastic Gaussian Processes	August 7, 2017	n/a	High Dimensional Bayesian Optimization with Elastic Gaussian Process	Other
Active subspaces (ASM)	July 15, 2017	n/a	Exploiting Active Subspaces in Global Optimization: How Complex is your Problem?	Linear embeddings
BO + MCMC (Finding additive substructure)	April 10, 2017		Discovering and Exploiting Additive Structure for Bayesian Optimization	Additive models
BOHAMIANN	December 5, 2016		Bayesian Optimization with Robust Bayesian Neural Networks	Other
G-STAR	December 1, 2016	n/a	G-STAR: A new Kriging-based trust region method for global optimization	Trust regions
SRE-IMGPO	July 9, 2016	n/a	Derivative-Free Optimization of High-Dimensional Non-Convex Functions by Sequential Random Embeddings	Linear embeddings
Restricted Projection Pursuit Models	May 9, 2016	n/a	High Dimensional Bayesian Optimization via Restricted Projection Pursuit Models	Additive models
DSA	November 18, 2015		Bayesian Optimization with Dimension Scheduling: Application to Biological Systems	Variable selection
TRIKE	August 8, 2015	n/a	Trust regions in Kriging-based optimization with expected improvement	Trust regions
Add-GP-UCB	May 13, 2015		High Dimensional Bayesian Optimisation and Bandits via Additive Models	Additive models
Warped REMBO	January 1, 2015	n/a	A Warped Kernel Improving Robustness in Bayesian Optimization Via Random Embeddings	Linear embeddings
SI-BO	December 5, 2013	n/a	High-Dimensional Gaussian Process Bandits	Linear embeddings
Active learning of linear embeddings	October 24, 2013		Active Learning of Linear Embeddings for Gaussian Processes	Linear embeddings
REMBO	January 9, 2013		Bayesian Optimization in a Billion Dimensions via Random Embeddings	Linear embeddings
Hierarchical diagonal sampling	June 27, 2012	n/a	Joint Optimization and Variable Selection of High-dimensional Gaussian Processes	Variable selection

MLLS

This project is brought to you by Center for Basic Machine Learning Research in Life Science