Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2205.03243

Alternating Good-for-MDP Automata

Ernst Moritz Hahn·Mateo Perez·Sven Schewe·Fabio Somenzi·Ashutosh Trivedi·Dominik Wojtczak
View original source ↗Read source document ↗Export metadata ↓

Abstract

When omega-regular objectives were first proposed in model-free reinforcement learning (RL) for controlling MDPs, deterministic Rabin automata were used in an attempt to provide a direct translation from their transitions to scalar values. While these translations failed, it has turned out that it is possible to repair them by using good-for-MDPs (GFM) Büchi automata instead. These are nondeterministic Büchi automata with a restricted type of nondeterminism, albeit not as restricted as in good-for-games automata. Indeed, deterministic Rabin automata have a pretty straightforward translation to such GFM automata, which is bi-linear in the number of states and pairs. Interestingly, the same cannot be said for deterministic Streett automata: a translation to nondeterministic Rabin or Büchi automata comes at an exponential cost, even without requiring the target automaton to be good-for-MDPs. Do we have to pay more than that to obtain a good-for-MDP automaton? The surprising answer is that we have to pay significantly less when we instead expand the good-for-MDP property to alternating automata: like the nondeterministic GFM automata obtained from deterministic Rabin automata, the alternating good-for-MDP automata we produce from deterministic Streett automata are bi-linear in the the size of the deterministic automaton and its index, and can therefore be exponentially more succinct than minimal nondeterministic Büchi automata.

Explore related subjects

cs.FLcs.AIcs.LO

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh Trivedi, Dominik Wojtczak. 2022-05-06. Alternating Good-for-MDP Automata. https://arxiv.org/abs/2205.03243

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Adding Reconfiguration to Zielonka's Asynchronous Automata

We study an extension of Zielonka's (fixed) asynchronous automata called reconfigurable asynchronous automata where processes can dynamically change who they communicate with. We show that reconfigurable asynchronous automata are not more expressive than fixed asynchronous automata by giving translations from one to the other. However, going from reconfigurable to fixed comes at the cost of disseminating communication (and knowledge) to all processes in the system. We then show that this is unavoidable by describing a language accepted by a reconfigurable automaton such that in every equivalent fixed automaton, every process must either be aware of all communication or be irrelevant.

cs.FL↗
arXiv2026

Certificates for short extending words in a finite automaton

Let $\mathcal A$ be a complete deterministic finite automaton on a state set $Q$ of size $n$ with $k$ letters, and for a proper nonempty subset $S$ of $Q$ let $\mathrm{minext}(S)$ be the length of a shortest word $u$ with $|Su^{-1}|>|S|$, where $Su^{-1}=\{q: q\cdot u\in S\}$. To each state $q$ attach the integer $β^{\ast}_q=\sum_{t=1}^{n-1}k^{\,n-1-t}(\mathrm{indeg}_t(q)-k^{t})$, where $\mathrm{indeg}_t(q)$ counts the pairs $(p,u)$ with $|u|=t$ and $p\cdot u=q$, and let $B(S)=\sum_{q\in S}β^{\ast}_q$. On every synchronizing automaton, $B(S)\ge0$ implies $\mathrm{minext}(S)\le n-1$, so, as $B(Q)=0$, one of $S$ and $Q\setminus S$ extends within $n-1$; when $B(S)>0$ no hypothesis is needed. Kari's Eulerian extension lemma is the case $β^{\ast}=0$, and $β^{\ast}$, like every member of the family $\sum_{t=1}^{n-1}c_tσ_t$, $c_t>0$, vanishes identically if and only if the automaton is Eulerian, where $σ_t(S)=\sum_{q\in S}(\mathrm{indeg}_t(q)-k^{t})$. On strongly connected automata $σ_t(S)/k^{t}$ has Cesàro limit $n\,e(S)/e(Q)-|S|$ for Friedman's weight $e$; that limit certifies singletons but no larger subset in general. The hypothesis $B(S)\ge0$ cannot be relaxed by one integer unit, nor can the constant $n-1$ be improved. A second-moment test on the sizes $|Su^{-1}|$ certifies 60 to 95 percent of the subsets with $B(S)<0$ at $n\le7$. Along non-Eulerian automata whose words of length $n-1$ merge a fraction of the state pairs bounded below, with $\max_q\mathrm{indeg}_{n-1}(q)=o(nk^{n-1})$, it certifies all but a vanishing share of them. The functional $B$ certifies half of the subsets outside $\{B=0\}$. At each subset size coprime to $n$ ($n\ge4$) some synchronizing Eulerian binary automaton attains the constant $n-1$; whether only there is open. No reset bound follows: Černý's automata have subsets not extending within $n-1$.

cs.FL↗
arXiv2026

Quadratic Word Equations with a Linear Side: Polynomial Nielsen Graph Diameter and NP-Completeness

The satisfiability problem for word equations asks whether variables can be replaced by words so that the two sides become equal. For regular word equations, in which each variable occurs at most once on each side, satisfiability is NP-complete. For general quadratic word equations, in which each variable occurs at most twice in total, satisfiability is NP-hard, but its membership in NP remains open. We consider an intermediate class: quadratic word equations with a linear side, where each variable occurs at most once on one designated side. We show that the Nielsen graph of an equation $U=V$ in this class, with total length $N=|U|+|V|$, has diameter $O(N^{12})$, measured over reachable pairs of vertices. Together with the known NP-hardness for regular word equations, this result establishes NP-completeness of satisfiability for this class.

cs.FL↗