Robust Optimization in Stochastic Games

Abstract

A player's impatient valuation of an infinite stream of stage payoffs is a weighted average with nonincreasing weights. Such valuations generalize geometric discounting and allow imprecise or heterogeneous time preferences. For a stochastic game \(Γ\) and a profile \(u\) of impatient valuations, let \(Γ_u\) be the induced game. We ask when approximately optimal or equilibrium behavior can be chosen to vary continuously and stably with \(u\). We show that this robustness is governed by the corresponding uniform long-run payoff property: the uniform value in the adversary case, and uniform approximate equilibrium payoffs in the equilibrium case. In finite state spaces, the latter is equivalent to the usual existence of a uniform equilibrium payoff. In the two-player zero-sum case, if \(Γ\) has a uniform value, then the value of \(Γ_u\) varies uniformly continuously with \(u\), and for every \(\varepsilon>0\) there is a uniformly continuous mapping assigning to each \(u\) an \(\varepsilon\)-optimal strategy in \(Γ_u\). Hence this holds for every finite two-player zero-sum stochastic game. In the non-zero-sum case, uniform approximate equilibrium payoffs imply a finite cover of the valuation-profile space by neighborhoods on each of which a single strategy profile is a robust \(\varepsilon\)-equilibrium. For sunspot, correlated, and extensive-form correlated equilibria, uniform approximate equilibrium payoffs are equivalent to the existence, for every \(\varepsilon>0\), of a uniformly continuous mapping assigning to each valuation profile a robust \(\varepsilon\)-equilibrium on a neighborhood of that profile. Consequently, every finite stochastic game admits uniformly continuous robust extensive-form correlated \(\varepsilon\)-equilibrium selections.