Globally aligned Principal Component Analysis for multi-group data

Abstract

We propose a novel principal component analysis (PCA) for multi-group datasets, where the same numerical variables are measured across different groups of observations. Existing approaches either ignore group structure entirely by working with global (pooled) data, focus exclusively on local structure (group-wise PCA), or impose restrictive assumptions of common principal components. Our approach respects the multi-group nature of data while improving global comparability of components. We combine group-specific principal components with global ones through an explicit alignment mechanism based on regularized optimization. We introduce the notion of globally aligned covariance matrix, incorporating weighted contributions from global principal directions in the group-wise covariance matrix. The alignment strength is controlled by regularization parameters that can be tuned to achieve the desired trade-off. Through a comprehensive simulation study, we demonstrate that the proposed aligned PCA achieves a favorable compromise between capturing local variation within groups and maintaining interpretability and stability across groups. Furthermore, in an application to the 2021 Canadian Census socioeconomic data, the proposed aligned PCA yields more comparable and stable region-specific components than pooled or region-wise PCA.