Distributed Derivative-Free Optimization Using Inexact ADMM and Trust-Region Methods

Abstract

To reduce complexity and achieve scalable performance in high-dimensional black-box settings, we propose a distributed method for nonconvex derivative-free optimization of continuous variables with an additively separable objective, subject to linear equality constraints. The approach is built upon the alternating direction method of multipliers (ADMM) as the distributed optimization framework. To handle general, potentially complicating linear equality constraints beyond the standard ADMM formulation, we employ a two-level ADMM structure: an inner layer that performs sequential ADMM updates, and an outer layer that drives an introduced slack variable to zero via the method of multipliers. In addition, each subproblem is solved inexactly using a derivative-free trust-region solver, ensuring suboptimality within a decreasing, theoretically controlled error tolerance. This inexactness is critical for both computational efficiency and practical applicability in black-box settings, where exact solutions are impractical or overly expensive. We establish theoretical convergence of the proposed approach to an approximate solution, and demonstrate improved computational efficiency over monolithic derivative-free optimization approaches on challenging high-dimensional benchmarks, as well as effective performance on a distributed learning problem.