Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1902.00151

A dual Newton based preconditioned proximal point algorithm for exclusive lasso models

Meixia Lin·Defeng Sun·Kim-Chuan Toh·Yancheng Yuan
View original source ↗Read source document ↗Export metadata ↓

Abstract

The exclusive lasso (also known as elitist lasso) regularization has become popular recently due to its superior performance on group sparsity. Compared to the group lasso regularization which enforces the competition on variables among different groups, the exclusive lasso regularization also enforces the competition within each group. In this paper, we propose a highly efficient dual Newton based preconditioned proximal point algorithm (PPDNA) to solve machine learning models involving the exclusive lasso regularizer. As an important ingredient, we provide a rigorous proof for deriving the closed-form solution to the proximal mapping of the weighted exclusive lasso regularizer. In addition, we derive the corresponding HS-Jacobian to the proximal mapping and analyze its structure --- which plays an essential role in the efficient computation of the PPA subproblem via applying a semismooth Newton method on its dual. Various numerical experiments in this paper demonstrate the superior performance of the proposed PPDNA against other state-of-the-art numerical algorithms.

Explore related subjects

math.OCcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Meixia Lin, Defeng Sun, Kim-Chuan Toh, Yancheng Yuan. 2019-12-06. A dual Newton based preconditioned proximal point algorithm for exclusive lasso models. https://arxiv.org/abs/1902.00151

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

On the stability of proximal operators in Wasserstein spaces under different notions of convexity

The proximal operator is a fundamental tool in variational analysis and optimization. In the setting of a Hilbert space, given a proper, lower semicontinuous convex functional, its proximal operator is non-expansive, that is, 1-Lipschitz continuous. In the Wasserstein setting, the contraction properties of this operator have been investigated from different perspectives by Carlen and Craig and by Adve and Mészáros, among others, and are not completely understood. In this paper, we study the stability properties of proximal maps, with a particular focus on non-expansivity, under various notions of convexity of the functional that can be considered in the Wasserstein space.

math.OC↗
arXiv2026

Symmetry-dependence in Rounding of a Convex Body

The symmetry measure of a convex body $S\subset\mathbb{R}^n$ is given by: $\mathrm{sym}(S):=\max\{α\ge0:\text{ there exists }x\in S\text{ such that }-α(S-x)\subseteq S-x\}$, where such an $x$ is called a Minkowski center. We prove that every convex body $S$ admits a $\sqrt{\frac{n}{\mathrm{sym}(S)}}$-rounding of $S$, namely, there exists an origin-centered ellipsoid $E$ and a center $c$ such that $E\subseteq S-c\subseteq\sqrt{\frac{n}{\mathrm{sym}(S)}}\,E$. This result was conjectured in 2005 by Belloni and Freund. As special cases, this recovers an $n$-rounding of $S$ (since $\mathrm{sym}(S)\ge\frac{1}{n}$), and a $\sqrt{n}$-rounding when $\mathrm{sym}(S)=1$. In the case when $S$ is a polytope given as the convex hull of points, the desired rounding is produced by a regularized minimum-volume covering ellipsoid problem where the regularization is with respect to the Minkowski center. Similarly, when $S$ is a polytope given as the intersection of halfspaces, such a rounding is produced by a regularized maximum-volume inscribed ellipsoid problem. In both of these cases, the rounding can be computed by first solving a linear optimization problem (to compute $\mathrm{sym}(S)$ and a Minkowski center), and then solving a convex optimization problem with a logarithmic determinant objective, second-order cone constraints, and one semidefinite cone constraint. We also show that the factor $\sqrt{\frac{n}{\mathrm{sym}(S)}}$ is nearly tight in its dependence on dimension and symmetry. When $\frac{n+1}{1+\mathrm{sym}(S)}$ is an integer, we show by explicit construction that the factor $\sqrt{\frac{n}{\mathrm{sym}(S)}}$ is tight. In the more general case, for every dimension $n$ and every admissible symmetry value, we construct a polytope $S$ for which every rounding factor is at least $\sqrt{\frac{2}{3}}\sqrt{\frac{n}{\mathrm{sym}(S)}}$.

math.OC↗
arXiv2026

Improving the Last-Iterate Guarantees of Anytime Algorithms for Stochastic Monotone Variational Inequalities

We analyze a stochastic algorithm with Halpern-type anchoring for constrained convex-concave problems and monotone variational inequalities. This single-loop and single-call algorithm uses one unbiased sample of the gradient operator at every iteration, to be applicable to monotone games with noisy feedback. With $t$ denoting the iteration counter, we prove an anytime last-iterate convergence rate of $O(t^{-1/4})$ for both the gradient-mapping norm and restricted gap, bypassing the $O(t^{-1/5})$ constrained-anytime bottleneck in the literature. Specializing then to multi-point oracles, we use variance reduction to achieve the $O(t^{-1/2})$ rate with an anytime single-loop algorithm using $2$ samples per iteration. Our results allow constrained problems with a potentially unbounded feasible set; as well as a structured class of stochastic oracles whose variance need not be uniformly bounded.

math.OC↗