Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.04117

Sharp Convergence and Sample Complexity of Policy Mirror Descent for Average-Reward MDPs

Enes Arda·Atilla Eryilmaz
View original source ↗Read source document ↗Export metadata ↓

Abstract

Policy mirror descent (PMD) has a mature finite-time theory in discounted Markov decision processes (MDPs), but less is known in the average-reward setting, a more natural objective for many control applications. We give a finite-time, finite-sample analysis of PMD in ergodic average-reward MDPs built around a single master recursion that governs convergence for any critic, without external regularization. Its specializations yield linear rates for exact, inexact-tabular, and linear function approximation (LFA) updates, with a superlinear regime for exact PMD. We complement these convergence results with end-to-end sample complexities of order $t_{\mathrm{mix}}^3/\varepsilon^2$ in both tabular ($|S||A|$-dependent) and LFA ($d$-dependent) settings. Our LFA sample complexity sharpens the prior best $t_{\mathrm{mix}}^5$ mixing dependence to $t_{\mathrm{mix}}^3$, and matching information-theoretic lower bounds establish that the critic's $t_{\mathrm{mix}}^3/\varepsilon^2$ sample complexity is unimprovable in both settings.

Explore related subjects

cs.LGmath.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Enes Arda, Atilla Eryilmaz. 2026-10-02. Sharp Convergence and Sample Complexity of Policy Mirror Descent for Average-Reward MDPs. https://arxiv.org/abs/2610.04117

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Cooperative Sheaf Neural Networks

Sheaf diffusion has recently emerged as a promising design pattern for graph representation learning due to its inherent ability to handle heterophilic data and avoid oversmoothing. Meanwhile, cooperative message passing has also been proposed as a way to enhance the flexibility of information diffusion by allowing nodes to independently choose whether to propagate/gather information from/to neighbors. A natural question ensues: is sheaf diffusion capable of exhibiting this cooperative behavior? Here, we provide a negative answer to this question. In particular, we show that existing sheaf diffusion methods fail to achieve cooperative behavior due to the lack of message directionality. To circumvent this limitation, we introduce the notion of cellular sheaves over directed graphs and characterize their in- and out-degree Laplacians. We leverage our construction to propose Cooperative Sheaf Neural Networks (CSNNs). Theoretically, we characterize the receptive field of CSNN and show it allows nodes to selectively attend (listen) to arbitrarily far nodes while ignoring all others in their path, potentially mitigating oversquashing. Our experiments show that CSNN presents overall better performance compared to prior art on sheaf diffusion as well as cooperative graph neural networks.

cs.LG↗
arXiv2026

GeoFunFlow: Geometric function flow matching for joint probabilistic inference of physical fields and complex geometries

Inverse problems governed by partial differential equations (PDEs) arise widely in science and engineering, but are often ill-posed and limited by sparse, noisy observations. In many applications, measurements reveal only part of the physical state, while the domain geometry may also be unknown even though it shapes the observed response. Joint field and geometry inference across varying computational domains and discretizations remains challenging, whereas many existing machine learning approaches are designed for known geometries and deterministic field reconstruction. Here, we introduce GeoFunFlow, a probabilistic framework that unifies field reconstruction on known domains and joint field and geometry inference on unknown domains. GeoFunFlow combines a geometric function autoencoder (GeoFAE) with flow matching in the latent space to model a joint distribution over physical fields and geometries. GeoFAE establishes a common representation across spatial discretizations that captures the relationship between physical fields and domain geometries, with unknown geometry represented by a signed distance function. The resulting representation allows observations to guide both field reconstruction and geometry recovery, while latent rectified flow enables efficient conditional sampling and spatially resolved uncertainty quantification. A calibration procedure further provides geometry uncertainty estimates with interpretable empirical coverage. Across seven benchmarks spanning porous media flow, fluid mechanics, and optical tomography, GeoFunFlow accurately recovers fields and geometries across complex, variable, and unknown domains while quantifying spatially resolved conditional uncertainty.

cs.LG↗
arXiv2026

Truncated Kernel Stochastic Gradient Descent with General Losses and Spherical Radial Basis Functions

In this paper, we propose a novel kernel stochastic gradient descent (SGD) algorithm for large-scale supervised learning with general losses. Compared to traditional kernel SGD, our algorithm improves efficiency and scalability through an adaptive regularization strategy. By leveraging the infinite series expansion of spherical radial basis functions, this strategy projects the stochastic gradient onto a finite-dimensional hypothesis space, which is adaptively scaled according to the bias-variance trade-off, thereby enhancing generalization performance. To handle the gradient nonlinearity arising from general losses, we develop a new generalization framework combining an inequality-based characterization of the kernel-induced covariance operator with optimization techniques. We prove that both the last iterate and the suffix average converge at minimax-optimal rates, and we further establish optimal strong convergence in the reproducing kernel Hilbert space. Our framework accommodates a broad class of classical loss functions, including least-squares, Huber, and logistic losses. Moreover, the proposed algorithm significantly reduces computational complexity and achieves optimal storage complexity by incorporating coordinate-wise updates from linear SGD, thereby avoiding the costly pairwise operations typical of kernel SGD and enabling efficient processing of streaming data. Finally, extensive numerical experiments provide empirical support for the theoretical results and the computational advantages of our algorithm.

cs.LG↗