A New Perspective on Clustering: A Mixed-norm Model and its Solution by Progressive Integer Programming

Abstract

Extending the classical $K$-means and $K$-medians models, this paper introduces an $\ell_{p,q}$ mixed-norm clustering model where the centroid updates and cluster assignments are under the $\ell_p$ and $\ell_q$ norms, respectively. The model is formulated as a mixed-integer program (MIP) with Heaviside composite constraints that describe the nearest-center assignments. The framework recovers $K$-means and $K$-medians when $p=q=2$ and $p=q=1$, respectively, and yields new models when $p\ne q$. To address the computational challenges, we develop a progressive integer programming (PIP) method that adaptively fixes confident assignments and solves restricted mixed-integer subproblems. For $q=1$, we develop a convex inner approximation of the difference-of-convex constraints in the restricted subproblems, for which a global solution can be computed. Importantly, we establish the connection between the local minimizer and the strong center-local minimizer of the mixed-norm clustering problem and the global optimal solution of the restricted subproblems under certain assumptions. This connection provides a practical certificate of a local minimizer of the nonconvex mixed-norm clustering model. We further develop techniques for constructing adaptive fixing sets and working sets for $q=1$. Extensive numerical experiments demonstrate the superior performance of the mixed-norm clustering model and the efficiency of PIP for solving the MIP model, which may be intractable otherwise. In particular, the $\ell_{2,1}$ mixed-norm clustering model is effective under coordinate-sparse, mean-balanced contamination, whereas the $\ell_{1,2}$ model is preferred under dense coordinatewise Cauchy contamination. The numerical results also show that PIP can escape poor alternating solutions and obtain substantially better feasible clustering, while preserving strong warm starts when no improvement is found.