Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2207.13550

Solving Poisson's equation for birth-death chains: Structure, instability, and accurate approximation

José Niño-Mora
View original source ↗Read source document ↗Export metadata ↓

Abstract

Poisson's equation plays a fundamental role as a tool for performance evaluation and optimization of Markov chains. For continuous-time birth-death chains with possibly unbounded transition and cost rates as addressed herein, when analytical solutions are unavailable its numerical solution can in theory be obtained by a simple forward recurrence. Yet, this may suffer from numerical instability, which can hide the structure of exact solutions. This paper presents three main contributions: (1) it establishes a structural result (convexity of the relative cost function) under mild conditions on transition and cost rates, which is relevant for proving structural properties of optimal policies in Markov decision models; (2) it elucidates the root cause, extent and prevalence of instability in numerical solutions by standard forward recurrence; and (3) it presents a novel forward-backward recurrence scheme to compute accurate numerical solutions. The results are applied to the accurate evaluation of the bias and the asymptotic variance, and are illustrated in an example.

Explore related subjects

math.PRmath.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

José Niño-Mora. 2022-07-27. Solving Poisson's equation for birth-death chains: Structure, instability, and accurate approximation. https://doi.org/10.1016/j.peva.2020.102163

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Villani's conjecture for Kac's walk

We prove Villani's conjecture on entropy production for Kac's walk with uniform collision angles. For every $N\geq2$, with total collision rate $N$, the optimal entropy production constant is $2/(N-1)$. We show that the known lower bound is sharp by constructing two families of smooth strictly positive probability densities, invariant under coordinate permutations and sign changes. The first consists of normalized products of inverse powers; the second is obtained by conditioning products of Gaussian mixtures on the energy sphere. With $N$ fixed, we compute the leading terms of entropy and entropy production. Successive parameter limits yield two independent proofs of sharpness.

math.PR↗
arXiv2026

Scaling limit and tail bounds for a random walk model of SOS level lines

This paper analyzes a random walk model for the level lines appearing in the entropic repulsion phenomena of three-dimensional discrete random interfaces above a hard wall; we are particularly motivated by the low-temperature (2+1)D solid-on-solid (SOS) model, where the emergence of these level lines has been rigorously established. The model we consider is a line ensemble of non-crossing random walk bridges above a wall with geometrically growing area tilts. Our main result, which in particular resolves a question of Caputo, Ioffe, and Wachtel (2019), is an edge 1:2:3 scaling limit for this ensemble as the domain size diverges, with a growing number of walks (including the number of level lines of the SOS model) and high boundary conditions (covering the maximum upper deviation of the SOS level lines). As a key input, we establish Tracy--Widom-type upper tail bounds for each of the relevant curves in the line ensemble. An ingredient which may be of independent interest is a ballot theorem for random walk bridges under a broader range of boundary values than available in the literature.

math.PR↗
arXiv2026

One-dimensional particle clouds with elastic collisions

We study an interacting particle system of a finite number of labelled particles on the integer lattice, in which particles have intrinsic masses and left/right jump rates. If a particle is the minimal-label particle at its site when it tries to jump left, the jump is executed. If not, 'momentum' is transferred to increase the rate of jumping left of the minimal-label particle. Similarly for jumps to the right. The collision rule is 'elastic' in the sense that the net rate of flow of mass is independent of the present configuration, in contrast to the exclusion process, for example. We show that the particle masses and jump rates determine explicitly, via a concave majorant of a simple `potential' function associated to the masses and jump rates, a unique partition of the system into maximal stable subsystems. The internal configuration of each stable subsystem remains tight, while the location of each stable subsystem obeys a strong law of large numbers with an explicit speed. We indicate connections to adjacent models, including diffusions with rank-based coefficients, where, to the authors' knowledge, the problem of identifying the analogous decomposition into stable sub-systems is yet to be fully solved.

math.PR↗