Decoupling actions of finite-dimensional Lie groups and of groups of diffeomorphisms in the large deformation framework

Abstract

In computational anatomy, the Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework has become a central tool for modeling smooth, invertible transformations between shapes such as curves or landmarks. In this paper, we extend this framework by enriching diffeomorphic deformations with transformations induced by finite-dimensional Lie groups (e.g. isometries, scalings), and we develop a registration model that decouples the actions of these two types of deformation on the shape during the matching process. To achieve this, we consider semidirect products between finite-dimensional groups and groups of diffeomorphisms, endowed with a right-invariant sub-Riemannian structure that give rise to new variational problems for shape registration. By exploiting symmetries and reduction theory, we decouple the contributions of each group throughout the matching process. We further extend the framework to incoroporate anisotropic deformations that preferentially favor certain directions during registration. On the numerical side, we propose an algorithm based on a joint optimization over both deformation groups, in contrast to the standard twostage approach that optimizes first over the finite-dimensional component and then over the diffeomorphic one. Experiments on curves and landmarks demonstrate that the proposed joint optimization improves registration accuracy and more effectively disentangles the contributions of the two deformation groups.