arXiv subjects

Runze You

Publications and source records attributed to Runze You.

6 recordsLinked to original sources

arXiv2026

Optimal Network Dependence in Distributed Stochastic Optimization via Tree Routing

Communication is a central bottleneck in distributed optimization, but its effect is often summarized by the spectral gap of a chosen mixing matrix. Since this gap depends on link weights as well as topology, it can obscure the intrinsic effect of the network. We clarify this distinction by relating graph diameter to the inverse spectral gap. We establish a universal one-sided bound that is tight up to constant factors, while constructions across several graph families show that no topology-only converse or universal two-sided scaling law exists. This motivates a class-based view of network optimality: diameter-dependent guarantees can be assessed uniformly over graph classes, whereas spectral-gap optimality requires both the graph--matrix pair class and the admissible algorithm class to be specified. Based on the resulting diameter-based minimax benchmark, we introduce Tree-Routed Gradient Tracking (Tree-RGT), a distributed method for stochastic nonconvex optimization. The method pipelines model dissemination and gradient aggregation over a rooted shortest-path spanning tree. Each iteration uses one round of one-hop communication and one stochastic-gradient evaluation per agent, without inner consensus or multi-gossip. Under smoothness and unbiased bounded-variance oracles, Tree-RGT achieves uniformly optimal network dependence over connected graphs of prescribed size and diameter. It recovers centralized stochastic scaling after $\mathcal{O}(nD_{\mathcal{G}}^2)$ transient iterations, where $n$ is the number of agents and $D_{\mathcal{G}}$ is the graph diameter. This network-dependent transient-iteration bound improves upon or matches those reported for representative distributed methods. Thus, topology-aware routing attains diameter-optimal guarantees without relying on a prescribed mixing matrix.

math.OC↗

arXiv2025

FedSUM Family: Efficient Federated Learning Methods under Arbitrary Client Participation

Federated Learning (FL) methods are often designed for specific client participation patterns, limiting their applicability in practical deployments. We introduce the FedSUM family of algorithms, which supports arbitrary client participation without additional assumptions on data heterogeneity. Our framework models participation variability with two delay metrics, the maximum delay $τ_{\max}$ and the average delay $τ_{\text{avg}}$. The FedSUM family comprises three variants: FedSUM-B (basic version), FedSUM (standard version), and FedSUM-CR (communication-reduced version). We provide unified convergence guarantees demonstrating the effectiveness of our approach across diverse participation patterns, thereby broadening the applicability of FL in real-world scenarios.

cs.LG↗

arXiv2025

Distributed Learning over Arbitrary Topology: Linear Speed-Up with Polynomial Transient Time

We study a distributed learning problem in which $n$ agents, each with potentially heterogeneous local data, collaboratively minimize the sum of their local cost functions via peer-to-peer communication. We propose a novel algorithm, \emph{Spanning Tree Push-Pull} (STPP), which employs two spanning trees extracted from a general communication graph to distribute both model parameters and stochastic gradients. Unlike prior approaches that rely heavily on spectral gap properties, STPP leverages a more flexible topological characterization, enabling robust information flow and efficient updates. Theoretically, we prove that STPP achieves linear speedup and polynomial transient iteration complexity -- up to $\mathcal{O}(n^7)$ for smooth nonconvex objectives and $\tilde{\mathcal{O}}(n^3)$ for smooth strongly convex objectives -- under arbitrary network topologies. Moreover, compared with existing methods, STPP achieves faster convergence rates on sparse and non-regular topologies (e.g., directed rings) and reduces communication overhead on dense networks (e.g., static exponential graphs). Numerical experiments further demonstrate the strong performance of STPP across various graph architectures.

math.OC↗

arXiv2025

Stochastic Push-Pull for Decentralized Nonconvex Optimization

To understand the convergence behavior of the Push-Pull method for decentralized optimization with stochastic gradients (Stochastic Push-Pull), this paper presents a comprehensive analysis. Specifically, we first clarify the algorithm's underlying assumptions, particularly those regarding the network structure and weight matrices. Then, to establish the convergence rate under smooth nonconvex objectives, we introduce a general analytical framework that not only encompasses a broad class of decentralized optimization algorithms, but also recovers or enhances several state-of-the-art results for distributed stochastic gradient tracking methods. A key highlight is the derivation of a sufficient condition under which the Stochastic Push-Pull algorithm achieves linear speedup, matching the scalability of centralized stochastic gradient methods -- a result not previously reported. Extensive numerical experiments validate our theoretical findings, demonstrating the algorithm's effectiveness and robustness across various decentralized optimization scenarios.

math.OC↗

arXiv2025

Decentralized Min-Max Optimization with Gradient Tracking

This paper presents a novel distributed formulation of the min-max optimization problem. Such a formulation enables enhanced flexibility among agents when optimizing their maximization variables. To address the problem, we propose two distributed gradient methods over networks, termed Distributed Gradient Tracking Ascent (DGTA) and Distributed Stochastic Gradient Tracking Ascent (DSGTA). We demonstrate that DGTA achieves an iteration complexity of $\mathcal{O}(κ^2\varepsilon^{-2})$, and DSGTA attains a sample complexity of $\mathcal{O}(κ^3\varepsilon^{-4})$ for nonconvex strongly concave (NC-SC) objective functions. Both results match those of their centralized counterparts up to constant factors related to the communication network. Numerical experiments further demonstrate the superior empirical performance of the proposed algorithms compared to existing methods.

math.OC↗

arXiv2024

B-ary Tree Push-Pull Method is Provably Efficient for Distributed Learning on Heterogeneous Data

This paper considers the distributed learning problem where a group of agents cooperatively minimizes the summation of their local cost functions based on peer-to-peer communication. Particularly, we propose a highly efficient algorithm, termed ``B-ary Tree Push-Pull'' (BTPP), that employs two B-ary spanning trees for distributing the information related to the parameters and stochastic gradients across the network. The simple method is efficient in communication since each agent interacts with at most $(B+1)$ neighbors per iteration. More importantly, BTPP achieves linear speedup for smooth nonconvex and strongly convex objective functions with only $\tilde{O}(n)$ and $\tilde{O}(1)$ transient iterations, respectively, significantly outperforming the state-of-the-art results to the best of our knowledge. Our code is available at https://github.com/ryou98/BTPP.

math.OC↗