Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2601.13045

Markovian restless bandits and index policies: A review

José Niño-Mora
View original source ↗Read source document ↗Export metadata ↓

Abstract

The restless multi-armed bandit problem is a paradigmatic modeling framework for optimal dynamic priority allocation in stochastic models of wide-ranging applications that has been widely investigated and applied since its inception in a seminal paper by Whittle in the late 1980s. The problem has generated a vast and fast-growing literature from which a significant sample is thematically organized and reviewed in this paper. While the main focus is on priority-index policies due to their intuitive appeal, tractability, asymptotic optimality properties, and often strong empirical performance, other lines of work are also reviewed. Theoretical and algorithmic developments are discussed, along with diverse applications. The main goals are to highlight the remarkable breadth of work that has been carried out on the topic and to stimulate further research in the field.

Explore related subjects

math.GM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

José Niño-Mora. 2026-01-19. Markovian restless bandits and index policies: A review. https://doi.org/10.3390/math11071639

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Proof of Liu's Conjecture on the Fundamental Triangle Inequality

Let $a,b,c$ be the side lengths of a triangle, and let $R$ and $r$ denote its circumradius and inradius, respectively. Liu proposed the conjecture \[ \sum_{\text{cyc}} \left(\frac{a(b+c-a)}{bc}\right)^k \ge 2+\left(\frac{2r}{R}\right)^k,\qquad k>1, \] with the reverse inequality for $k<1$. We prove this conjecture by reducing it to an algebraic inequality for three positive variables with prescribed sum and product. We also determine the equality cases.

math.GM↗
arXiv2026

A quadratic critical-value conjecture for the fifth Bessel moment

We conjecture an explicit evaluation of the pure fifth Bessel moment $\int_0^\infty K_0(t)^5\,dt$ as a quadratic expression in the critical value $L(f,2)$ of the weight-three, level-60 newform $f$ (LMFDB orbit 60.3.b.a) identified in the twisted fifth-moment modularity theorem of Lim, Tu and Yu, with coefficients in $\mathbb{Q}(\sqrt{5})$ and the square taken before the real and imaginary parts. Directed interval computations, using no stored Bessel or $L$-values, bound the absolute discrepancy by $10^{-358}$. We prove three exact modular identities for $f$: the Petersson-norm formula $\langle f,f\rangle_{60} = \frac{3(5-\sqrt{5})}{2π^4}|L(f,2)|^2$, the coefficient-conjugation relation $L(f^σ,2) = κL(f,2)$ with explicit $κ\in \mathbb{Q}(\sqrt{5},i)$, and the twisted symmetric-square evaluation $L(χ_{-4}\mathrm{Sym}^2 f,2) = \sqrt{15}\,π^2 \langle f,f\rangle_{60}$, together with $L(χ_{-4}\mathrm{Sym}^2 f,3) = π^4\langle f,f\rangle_{60}/8$, in the full Euler-factor normalization of Lim, Tu and Yu. The last identity shows that the companion norm conjecture $D_{5,\mathrm{odd}} = \frac{3\sqrt{15}(5-\sqrt{5})}{2}|L(f,2)|^2$ is equivalent to the symmetric-square conjecture $D_{5,\mathrm{odd}} = π^2 L(χ_{-4}\mathrm{Sym}^2 f,2)$ of Lim, Tu and Yu, while the exact relation $D_{5,\mathrm{even}} = π^2 D_{5,\mathrm{odd}}/(2\sqrt{15})$ follows from Chuang's period formulas. Every Bessel-to-modular equality, including the individual-period formula, remains conjectural. Complete proofs, exact rational certificates and verification programs are included as ancillary files.

math.GM↗