arXiv subjects

Valentin Duruisseaux

Publications and source records attributed to Valentin Duruisseaux.

27 records · Page 2Linked to original sources

arXiv2023

Practical Perspectives on Symplectic Accelerated Optimization

Geometric numerical integration has recently been exploited to design symplectic accelerated optimization algorithms by simulating the Lagrangian and Hamiltonian systems from the variational framework introduced in Wibisono et al. In this paper, we discuss practical considerations which can significantly boost the computational performance of these optimization algorithms, and considerably simplify the tuning process. In particular, we investigate how momentum restarting schemes ameliorate computational efficiency and robustness by reducing the undesirable effect of oscillations, and ease the tuning process by making time-adaptivity superfluous. We also discuss how temporal looping helps avoiding instability issues caused by numerical precision, without harming the computational efficiency of the algorithms. Finally, we compare the efficiency and robustness of different geometric integration techniques, and study the effects of the different parameters in the algorithms to inform and simplify tuning in practice. From this paper emerge symplectic accelerated optimization algorithms whose computational efficiency, stability and robustness have been improved, and which are now much simpler to use and tune for practical applications.

math.OC↗

arXiv2023

Lie Group Forced Variational Integrator Networks for Learning and Control of Robot Systems

Incorporating prior knowledge of physics laws and structural properties of dynamical systems into the design of deep learning architectures has proven to be a powerful technique for improving their computational efficiency and generalization capacity. Learning accurate models of robot dynamics is critical for safe and stable control. Autonomous mobile robots, including wheeled, aerial, and underwater vehicles, can be modeled as controlled Lagrangian or Hamiltonian rigid-body systems evolving on matrix Lie groups. In this paper, we introduce a new structure-preserving deep learning architecture, the Lie group Forced Variational Integrator Network (LieFVIN), capable of learning controlled Lagrangian or Hamiltonian dynamics on Lie groups, either from position-velocity or position-only data. By design, LieFVINs preserve both the Lie group structure on which the dynamics evolve and the symplectic structure underlying the Hamiltonian or Lagrangian systems of interest. The proposed architecture learns surrogate discrete-time flow maps allowing accurate and fast prediction without numerical-integrator, neural-ODE, or adjoint techniques, which are needed for vector fields. Furthermore, the learnt discrete-time dynamics can be utilized with computationally scalable discrete-time (optimal) control strategies.

cs.RO↗

arXiv2023

Time-adaptive Lagrangian Variational Integrators for Accelerated Optimization on Manifolds

A variational framework for accelerated optimization was recently introduced on normed vector spaces and Riemannian manifolds in Wibisono et al. (2016) and Duruisseaux and Leok (2021). It was observed that a careful combination of timeadaptivity and symplecticity in the numerical integration can result in a significant gain in computational efficiency. It is however well known that symplectic integrators lose their near energy preservation properties when variable time-steps are used. The most common approach to circumvent this problem involves the Poincare transformation on the Hamiltonian side, and was used in Duruisseaux et al. (2021) to construct efficient explicit algorithms for symplectic accelerated optimization. However, the current formulations of Hamiltonian variational integrators do not make intrinsic sense on more general spaces such as Riemannian manifolds and Lie groups. In contrast, Lagrangian variational integrators are well-defined on manifolds, so we develop here a framework for time-adaptivity in Lagrangian variational integrators and use the resulting geometric integrators to solve optimization problems on normed vector spaces and Lie groups.

math.OC↗

arXiv2023

Approximation of nearly-periodic symplectic maps via structure-preserving neural networks

A continuous-time dynamical system with parameter $\varepsilon$ is nearly-periodic if all its trajectories are periodic with nowhere-vanishing angular frequency as $\varepsilon$ approaches 0. Nearly-periodic maps are discrete-time analogues of nearly-periodic systems, defined as parameter-dependent diffeomorphisms that limit to rotations along a circle action, and they admit formal $U(1)$ symmetries to all orders when the limiting rotation is non-resonant. For Hamiltonian nearly-periodic maps on exact presymplectic manifolds, the formal $U(1)$ symmetry gives rise to a discrete-time adiabatic invariant. In this paper, we construct a novel structure-preserving neural network to approximate nearly-periodic symplectic maps. This neural network architecture, which we call symplectic gyroceptron, ensures that the resulting surrogate map is nearly-periodic and symplectic, and that it gives rise to a discrete-time adiabatic invariant and a long-time stability. This new structure-preserving neural network provides a promising architecture for surrogate modeling of non-dissipative dynamical systems that automatically steps over short timescales without introducing spurious instabilities.

cs.LG↗

arXiv2022

Bistability, Bifurcations and Chaos in the Mackey-Glass Equation

Numerical bifurcation analysis, and in particular two-parameter continuation, is used in consort with numerical simulation to reveal complicated dynamics in the Mackey-Glass equation for moderate values of the delay close to the onset of chaos. In particular a cusp bifurcation of periodic orbits and resulting branches of folds of periodic orbits effectively partition the parameter space into regions where different behaviours are seen. The cusp bifurcation leads directly to bistability between periodic orbits, and subsequently to bistability between a periodic orbit and a chaotic attractor. This leads to two different mechanisms by which the chaotic attractor is destroyed in a global bifurcation with a periodic orbit in either an interior crisis or a boundary crisis. In another part of parameter space a sequence of subcritical period-doublings is found to give rise to bistability between a periodic orbit and a chaotic attractor. Torus bifurcations, and a codimension-two fold-flip bifurcation are also identified, and Lyapunov exponent computations are used to determine chaotic regions and attractor dimension.

nlin.CD↗

arXiv2022

Accelerated Optimization on Riemannian Manifolds via Discrete Constrained Variational Integrators

A variational formulation for accelerated optimization on normed vector spaces was recently introduced in Wibisono et al., and later generalized to the Riemannian manifold setting in Duruisseaux and Leok. This variational framework was exploited on normed vector spaces in Duruisseaux et al. using time-adaptive geometric integrators to design efficient explicit algorithms for symplectic accelerated optimization, and it was observed that geometric discretizations which respect the time-rescaling invariance and symplecticity of the Lagrangian and Hamiltonian flows were substantially less prone to stability issues, and were therefore more robust, reliable, and computationally efficient. As such, it is natural to develop time-adaptive Hamiltonian variational integrators for accelerated optimization on Riemannian manifolds. In this paper, we consider the case of Riemannian manifolds embedded in a Euclidean space that can be characterized as the level set of a submersion. We will explore how holonomic constraints can be incorporated in discrete variational integrators to constrain the numerical discretization of the Riemannian Hamiltonian system to the Riemannian manifold, and we will test the performance of the resulting algorithms by solving eigenvalue and Procrustes problems formulated as optimization problems on the unit sphere and Stiefel manifold.

math.NA↗

arXiv2022

A Variational Formulation of Accelerated Optimization on Riemannian Manifolds

It was shown recently by Su et al. (2016) that Nesterov's accelerated gradient method for minimizing a smooth convex function $f$ can be thought of as the time discretization of a second-order ODE, and that $f(x(t))$ converges to its optimal value at a rate of $\mathcal{O}(1/t^2)$ along any trajectory $x(t)$ of this ODE. A variational formulation was introduced in Wibisono et al. (2016) which allowed for accelerated convergence at a rate of $\mathcal{O}(1/t^p)$, for arbitrary $p>0$, in normed vector spaces. This framework was exploited in Duruisseaux et al. (2021) to design efficient explicit algorithms for symplectic accelerated optimization. In Alimisis et al. (2020), a second-order ODE was proposed as the continuous-time limit of a Riemannian accelerated algorithm, and it was shown that the objective function $f(x(t))$ converges to its optimal value at a rate of $\mathcal{O}(1/t^2)$ along solutions of this ODE. In this paper, we show that on Riemannian manifolds, the convergence rate of $f(x(t))$ to its optimal value can also be accelerated to an arbitrary convergence rate $\mathcal{O}(1/t^p)$, by considering a family of time-dependent Bregman Lagrangian and Hamiltonian systems on Riemannian manifolds. This generalizes the results of Wibisono et al. (2016) to Riemannian manifolds and also provides a variational framework for accelerated optimization on Riemannian manifolds. An approach based on the time-invariance property of the family of Bregman Lagrangians and Hamiltonians was used to construct very efficient optimization algorithms in Duruisseaux et al. (2021), and we establish a similar time-invariance property in the Riemannian setting. One expects that a geometric numerical integrator that is time-adaptive, symplectic, and Riemannian manifold preserving will yield a class of promising optimization algorithms on manifolds.

math.OC↗

arXiv2022

Accelerated Optimization on Riemannian Manifolds via Projected Variational Integrators

A variational formulation of accelerated optimization on normed spaces was recently introduced by considering a specific family of time-dependent Bregman Lagrangian and Hamiltonian systems whose corresponding trajectories converge to the minimizer of the given convex function at an arbitrary accelerated rate of $\mathcal{O}(1/t^p)$. This framework has been exploited using time-adaptive geometric integrators to design efficient explicit algorithms for symplectic accelerated optimization. It was observed that geometric discretizations were substantially less prone to stability issues, and were therefore more robust, reliable, and computationally efficient. More recently, this variational framework has been extended to the Riemannian manifold setting by considering a more general family of time-dependent Bregman Lagrangian and Hamiltonian systems on Riemannian manifolds. It is thus natural to develop time-adaptive Hamiltonian variational integrators for accelerated optimization on Riemannian manifolds. In the past, Hamiltonian variational integrators have been constructed with holonomic constraints, but the resulting algorithms were implicit in nature, which significantly increased their cost per iteration. In this paper, we will test the performance of explicit methods based on Hamiltonian variational integrators combined with projections that constrain the numerical solution to remain on the constraint manifold.

math.OC↗

arXiv2021

Adaptive Hamiltonian Variational Integrators and Symplectic Accelerated Optimization

It is well known that symplectic integrators lose their near energy preservation properties when variable step sizes are used. The most common approach to combine adaptive step sizes and symplectic integrators involves the Poincaré transformation of the original Hamiltonian. In this article, we provide a framework for the construction of variational integrators using the Poincaré transformation. Since the transformed Hamiltonian is typically degenerate, the use of Hamiltonian variational integrators based on Type II or Type III generating functions is required instead of the more traditional Lagrangian variational integrators based on Type I generating functions. Error analysis is provided and numerical tests based on the Taylor variational integrator approach of Schmitt, Shingel, Leok (2018) to time-adaptive variational integration of Kepler's 2-Body problem are presented. Finally, we use our adaptive framework together with the variational approach to accelerated optimization presented in Wibisono, Wilson, Jordan (2016) to design efficient variational and non-variational explicit integrators for symplectic accelerated optimization.

math.NA↗