Decentralized Optimization over Time-Varying Row-Stochastic Digraphs

Abstract

Decentralized optimization over directed graphs is essential for applications such as robotic swarms, sensor networks, and distributed learning. In many practical scenarios, the underlying network takes the form of a Time-Varying Broadcast Network (TVBN), where only row-stochastic mixing matrices can be constructed due to the unavailability of out-degree information. Achieving exact convergence for decentralized optimization over TVBNs has remained a long-standing open problem, as the limiting distribution of time-varying row-stochastic mixing matrices depends on unpredictable future graph realizations, rendering standard bias-correction techniques infeasible. This paper develops the first decentralized optimization algorithm that achieves exact convergence using only time-varying row-stochastic matrices. We first propose PULM (Pull-with-Memory), a gossip protocol that achieves average consensus with exponential convergence by alternating between row-stochastic mixing and local adjustment steps. Building on PULM, we develop PULM-DGD, which converges to a stationary solution at a rate of $\mathcal{O}(\ln(T)/T)$ for smooth nonconvex objectives, where $T$ denotes the communication round. Our results significantly broaden the applicability of decentralized optimization to highly dynamic communication environments.