Invariant Distributions in Nonlinear Markov Chains with Aggregators: Theory, Computation, and Applications

Abstract

We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion dynamics, and wealth dynamics. In these chains, the next period's distribution depends on both the current state and a real-valued function of the current distribution. For these chains, we provide conditions for the uniqueness of an invariant distribution that do not rely on typical contraction arguments. Instead, our approach leverages flexible monotonicity properties imposed on the nonlinear Markov kernel. We demonstrate the necessity of these monotonicity conditions for proving the uniqueness of an invariant distribution through simple examples. We also provide existence results and introduce an iterative computational method that solves a simpler, tractable subproblem in each iteration and converges to the invariant distribution of the nonlinear Markov chain, even in cases where uniqueness does not hold. We leverage our findings to analyze invariant distributions in strategic queueing systems, study inventory dynamics when retailers optimize pricing and inventory decisions, establish conditions ensuring the uniqueness of solutions for a class of nonlinear equations in $\mathbb{R}^{n}$, and investigate the properties of stationary wealth distributions in large dynamic economies.