Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2305.08051

Prox-DBRO-VR: A Unified Analysis on Byzantine-Resilient Decentralized Stochastic Composite Optimization with Variance Reduction and Non-Asymptotic Convergence Rates

Jinhui Hu·Guo Chen·Huaqing Li·Xiaoyu Guo·Liang Ran·Tingwen Huang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Decentralized stochastic gradient algorithms efficiently solve large-scale finite-sum optimization problems when all agents in the network are reliable. However, most of these algorithms are not resilient to adverse conditions, such as malfunctioning agents, software bugs, and cyber attacks. This paper aims to handle a class of general composite optimization problems over multi-agent systems (MASs) in the presence of an unknown number of Byzantine agents. Building on a resilient aggregation mechanism and the proximal-gradient mapping method, a Byzantine-resilient decentralized stochastic proximal-gradient algorithmic framework is proposed, dubbed Prox-DBRO-VR, which achieves an optimization and control goal using only local computations and communications. To asymptotically reduce the noise variance arising from local gradient estimation and accelerate the convergence, we incorporate two localized variance-reduced (VR) techniques (SAGA and LSVRG) into Prox-DBRO-VR to design Prox-DBRO-SAGA and Prox-DBRO-LSVRG. By analyzing the contraction relationships among the gradient-learning error, resilient consensus condition, and convergence error in a unified theoretical framework, it is proved that both Prox-DBRO-SAGA and Prox-DBRO-LSVRG, with a well-designed constant (resp., decaying) step-size, converge linearly (resp., sub-linearly) inside an error ball around the optimal solution to the original problem under standard assumptions. A trade-off between convergence accuracy and Byzantine resilience in both linear and sub-linear cases is also characterized. In numerical experiments, the effectiveness and practicability of the proposed algorithms are manifested via resolving a decentralized sparse machine-learning problem under various Byzantine attacks.

Explore related subjects

math.OCcs.SYeess.SY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jinhui Hu, Guo Chen, Huaqing Li, Xiaoyu Guo, Liang Ran, Tingwen Huang. 2026-07-13. Prox-DBRO-VR: A Unified Analysis on Byzantine-Resilient Decentralized Stochastic Composite Optimization with Variance Reduction and Non-Asymptotic Convergence Rates. https://doi.org/10.1109/tsmc.2025.3565568

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

AdGT: Decentralized Gradient Tracking with Adaptive Per-Agent Stepsizes

In decentralized optimization, gradient-tracking methods typically rely on a single global stepsize. This choice can be conservative when agents have local objectives with different smoothness constants, since the stepsize must remain stable for the agent with the largest smoothness constant. This paper proposes AdGT, a decentralized gradient-tracking method in which each agent adapts its own stepsize using local gradient variation and a single global safety factor. The method reduces fixed-stepsize tuning effort and allows agents to exploit local smoothness information during the iterations. For smooth and strongly convex local objectives over undirected networks, we prove that the analyzed AdGT update converges linearly to the exact consensus optimizer. We also study two adaptive stepsize updates that use changes in the gradient-tracking direction. We characterize when the corresponding candidate determines the stepsize and prove conditional lower and upper stepsize bounds and linear convergence under an additional relative tracking-disagreement condition. Experiments on logistic regression, ridge regression, synthetic quadratic problems, and a linear-regression benchmark against state-of-the-art decentralized solvers show that AdGT often reaches a given accuracy in fewer iterations or gradient evaluations than tuned fixed-stepsize GT and the tested baselines, especially under heterogeneous local smoothness. In the topology experiments, each tested AdGT update uses one common safety factor across all graphs, whereas the fixed GT stepsize is tuned separately for each graph and seed.

math.OC↗
arXiv2026

Brockett cost function for symplectic eigenvalues

The sum of symplectic eigenvalues and corresponding eigenvectors of symmetric positive-definite matrices in the sense of Williamson's theorem can be computed via minimization of a trace cost function under the symplecticity constraint. Optimal solutions to this problem only offer a symplectic basis for the symplectic eigenspace corresponding to the sought symplectic eigenvalues. In this note, we introduce a Brockett cost function and investigate its properties and the connection with the symplectic eigenvalues and eigenvectors of the considered matrix. Specifically, we prove that any stationary point consists of symplectic eigenvectors, characterize the saddle points and global minimizers based on which the trace minimization theorem for the symplectic eigenvalues is re-established, and the nonexistence of local nonglobal minimizers is justified.

math.OC↗
arXiv2026

Riemannian Bilevel Optimization with Gradient Aggregation

We study bilevel optimization on Riemannian manifolds when the lower-level solution set is a positive-dimensional submanifold, so that implicit differentiation fails. We propose Riemannian Bilevel Descent Aggregation (RBDA), which extends bilevel descent aggregation to manifolds. Its inner loop aggregates the lower-level descent direction with the upper-level gradient under a decaying multiplier, and its hypergradient is the reverse-mode derivative of the unrolled loop. Under geodesic convexity and quadratic growth of the lower level, the inner iterates converge to a point of the optimistic solution set at a polynomial rate. Approximate minimizers of the objective with a finite number of inner iterations converge to minimizers of the optimistic value. In the experiments RBDA selects the optimistic solution where the implicit and unrolled estimators remain at the initial point or stop at a larger query loss. While each of its outer steps costs more than that of the unrolled estimator, it attains the highest test accuracy in data hyper-cleaning.

math.OC↗