A physics-inspired nonlinear momentum method for gradient descent with applications to inverse photonic design

Abstract

In this work, a nonlinear momentum method is introduced to enhance the convergence performance of momentum-based gradient optimization algorithms. Classical momentum methods, such as the Heavy Ball method, can be viewed as a dynamical system with quadratic kinetic energy and linear damping. By extending this analogy to non-Newtonian dynamical systems, we construct a Hamiltonian framework for optimization problems. In this framework, nonlinear kinetic energy and nonlinear damping effects naturally emerge. It provides a more flexible and physically interpretable mechanism for optimization algorithms. Specifically, we employ an anharmonic kinetic energy function to capture the inertial effects of accumulated gradient information during the optimization process, while the nonlinear damping mechanism effectively regulates the contribution of momentum during convergence. Numerical experiments show that the proposed method achieves faster convergence compared to classical momentum algorithms, making it particularly suitable for inverse design tasks. Moreover, the Hamiltonian based algorithmic framework may offer physical insights for the development of efficient physics-inspired optimization algorithms.