Sure-almost-sure and Sure-limit-sure Window Mean Payoff in Markov Decision Processes

Abstract

Given rationals $α$ and $β$, the sure-almost-sure problem for a threshold Boolean objective $φ$ in a Markov decision process (MDP) asks if one can simultaneously ensure that all outcomes of the MDP have $φ$-value at least $α$ (i.e. sure $α$ satisfaction) and with probability $1$ the outcome has $φ$-value at least $β$ (i.e. almost-sure $β$ satisfaction). The sure-limit-sure problem asks if for all $\varepsilon > 0$ one can simultaneously ensure that all outcomes have $φ$-value at least $α$ and with probability at least $1 - \varepsilon$ the outcome has $φ$-value at least $β$. Moreover, if simultaneous satisfaction of objectives is possible, then one would also like to construct a strategy (for sure-almost-sure) or a family of strategies (for sure-limit-sure) that achieves this. In this paper, we solve the sure-almost-sure and sure-limit-sure problems for window mean-payoff objectives. The window mean-payoff objective strengthens the standard mean-payoff objective by requiring that eventually, from every point in the infinite run, the average payoff becomes greater than a given threshold within a finite window length. We study two variants of window mean payoff: in the fixed variant, the window length $\ell$ is given, while in the bounded variant, the length is not given but is required to be bounded throughout the run. We show that the sure-almost-sure problem and the sure-limit-sure problem are both in P for the fixed variant (if $\ell$ is given in unary) and are both in NP $\cap$ coNP for the bounded variant, matching the computational complexity of sure satisfaction and almost-sure satisfaction when considered separately for these objectives. We also give bounds for the memory requirement of winning strategies for all considered problems.