Steady-state diffusion approximations of Markov chains: error analysis via the discrete Poisson equation

Abstract

This paper uses the generator approach of Stein's method to analyze the gap between steady-state distributions of Markov chains and diffusion processes. Until now, the standard way to invoke Stein's method for this problem was to use the Poisson equation for the diffusion as a starting point. The main technical difficulty with this approach is obtaining bounds on the derivatives of the solution to the Poisson equation, also known as Stein factor bounds. In this paper we propose starting with the discrete Poisson equation of the Markov chain. An important step in our approach is extending the discrete Poisson equation to be defined on the continuum where the diffusion is defined, and we achieve this by using interpolation. Although there are still Stein factor bounds to prove, these now correspond to the finite differences of the discrete Poisson equation solution, as opposed to the derivatives of the solution to the continuous one. Discrete Stein factor bounds can be easier to obtain, for instance when the drift is not everywhere differentiable, when the diffusion has a state-dependent diffusion coefficient, or in the presence of a reflecting boundary condition. We use the join the shortest queue model in the Halfin-Whitt regime as a working example to illustrate the methodology. We show that the steady-state approximation error of the diffusion limit converges to zero at a rate of $1/\sqrt{n}$, where $n$ is the number of servers in the system.