Regularized extragradient method for structured bilevel optimization in continuous and discrete time

Abstract

In a real Hilbert space, we study a bilevel optimization problem that consists in minimizing an outer convex function over the zero set of a maximally monotone operator. In the smooth setting, where the outer objective is convex and Fréchet differentiable and the inner operator is single-valued, continuous and monotone, we associate with the problem a first-order dynamical system that can be viewed as a monotone flow applied to a dynamically regularized operator. Under suitable geometric conditions on the inner problem --- either a weak Attouch-Czarnecki-type integrability condition or the stronger assumption of sharpness --- we establish last-iterate convergence rates for both the outer and inner residuals, together with weak convergence of the trajectories to optimal solutions of the bilevel problem. In the smooth+nonsmooth setting, we enrich the outer objective with a proper, convex, and lower semicontinuous function, while the inner operator is augmented by the subdifferential of a function with the same properties. We propose a regularized proximal-extragradient algorithm in which both the forward and backward steps are performed with respect to dynamically regularized operators and functions, respectively. Under geometric assumptions on the inner problem analogous to those in the smooth setting, we establish last-iterate convergence rates for both the outer and inner residuals, together with weak convergence of the iterates to optimal solutions of the bilevel problem.