Computing stable limit cycles of learning in games

Abstract

Many well-studied learning dynamics, such as fictitious play and the replicator, are known to not converge in general $N$-player games. The simplest mode of non-convergence is cyclical or periodic behavior. Such cycles are fundamental objects, and have inspired a number of significant insights in the field, beginning with the pioneering work of Shapley (1964). However a central question remains unanswered: which cycles are stable under game dynamics? In this paper we give a complete and computational answer to this question for the two best-studied dynamics, fictitious play/best-response dynamics and the replicator dynamic. We show (1) that a periodic sequence of profiles is stable under one of these dynamics if and only it is stable under the other, and (2) we provide a polynomial-time spectral stability test to determine whether a given periodic sequence is stable under either dynamic. Finally, we give an entirely `structural' sufficient condition for stability: every cycle that is a sink equilibrium of the preference graph of the game is stable, and moreover it is an attractor of the replicator dynamic. This result generalizes the famous theorems of Shapley (1964) and Jordan (1993), and extends the frontier of recent work relating the preference graph to the replicator attractors.