Adaptive Replication Strategies in Trust-Region-Based Bayesian Optimization of Stochastic Functions

Abstract

We develop and analyze a method for stochastic simulation optimization based on Gaussian process models within a trust-region framework. We focus on settings where the variance of the objective function is large, making accurate estimation challenging and often requiring many evaluations. To address this regime, we combine local modeling with adaptive replication, allowing the method to allocate repeated evaluations where they are most beneficial. We introduce several mechanisms to promote and adapt replication, including modifications to the acquisition function and cost-aware evaluation strategies. These components enable our approach to scale effectively when high levels of sampling are required to reduce noise. We refer to the resulting method as OGPIT, for Optimization by Gaussian Processes In Trust regions. Numerical experiments show that adaptive replication can substantially improve computational efficiency while preserving solution accuracy compared to baseline methods, in particular when evaluation costs are taken into account.