Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1611.05796

Imprecise Continuous-Time Markov Chains

Thomas Krak·Jasper De Bock·Arno Siebes
View original source ↗Read source document ↗Export metadata ↓

Abstract

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models computationally tractable, they rely on a number of assumptions that may not be realistic for the domain of application; in particular, the ability to provide exact numerical parameter assessments, and the applicability of time-homogeneity and the eponymous Markov property. In this work, we extend these models to imprecise continuous-time Markov chains (ICTMC's), which are a robust generalisation that relaxes these assumptions while remaining computationally tractable. More technically, an ICTMC is a set of "precise" continuous-time finite-state stochastic processes, and rather than computing expected values of functions, we seek to compute lower expectations, which are tight lower bounds on the expectations that correspond to such a set of "precise" models. Note that, in contrast to e.g. Bayesian methods, all the elements of such a set are treated on equal grounds; we do not consider a distribution over this set. The first part of this paper develops a formalism for describing continuous-time finite-state stochastic processes that does not require the aforementioned simplifying assumptions. Next, this formalism is used to characterise ICTMC's and to investigate their properties. The concept of lower expectation is then given an alternative operator-theoretic characterisation, by means of a lower transition operator, and the properties of this operator are investigated as well. Finally, we use this lower transition operator to derive tractable algorithms (with polynomial runtime complexity w.r.t. the maximum numerical error) for computing the lower expectation of functions that depend on the state at any finite number of time points.

Explore related subjects

math.PR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Thomas Krak, Jasper De Bock, Arno Siebes. 2017-06-20. Imprecise Continuous-Time Markov Chains. https://arxiv.org/abs/1611.05796

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Local well-posedness of general mean field game master equations

This paper presents a generic approach for establishing mean field game master equations, applicable whenever the mean field equilibrium can be characterized by a McKean-Vlasov forward-backward stochastic differential equation system. The core of our approach is a representation formula for the first-order Lions derivative of the decoupling field of this forward-backward SDE system. We then employ a bootstrap argument to recursively compute its higher-order derivatives. To demonstrate the method's versatility, we establish the local well-posedness for master equations in three distinct models: extended mean field games, mean field games with volatility control, and mean field games with a major player.

math.PR↗
arXiv2026

Uniqueness for nonlinear Fokker-Planck equations with general diffusion terms and their associated nonlinear Markov processes

This work is concerned with the uniqueness of distributional solutions to nonlinear Fokker-Planck equations with non-diagonal diffusion terms of type \begin{equation} u_{t}-\sum_{i,j=1}^{d} D^{2}_{ij}(a_{ij}(x)β(x,u))+ \text{div}(b(x,u)u)=0 \quad \text{in}\; (0, \infty) \times \mathbb{R}^{d} ,\notag \end{equation} with initial condition $u(0,x)\equiv u_{0}(x)$, where $a_{ij}$, $β$, and $b$ are suitable functions. Under suitable assumptions, this equation generates a continuous contraction semigroup $S(t): L^{1}(\mathbb{R}^{d}) \rightarrow L^{1}(\mathbb{R}^{d})$, and $u(t)=S(t)u_{0}$ is a mild solution to the equation. Our main contribution is to prove that this mild solution is unique in the much larger class of distributional solutions. This extends previous uniqueness results for the diagonal (also called isotropic) diffusion case $a_{ij} \equiv δ_{ij}$. Another key analytical result of this paper is the uniqueness for distributional solutions of the associated linearized equation. As a main application, we prove weak uniqueness for the corresponding McKean-Vlasov SDEs. Moreover, we prove that, the probabilistically weak solution to the McKean-Vlasov SDEs is also the unique probabilistically strong solution. Furthermore, we establish a new $L^{\infty}$ estimate for mild solutions starting from data in $L^{1}\cap L^{\infty}$ and this estimate is used in the construction of nonlinear Markov processes. Finally, we prove that the path laws of the solutions to the McKean-Vlasov SDEs form a nonlinear Markov process in the sense of McKean.

math.PR↗
arXiv2026

Small-time annealed large deviations principle for one-dimensional diffusions in a random environment

In this paper, we establish a small-time annealed path large deviation principle for one-dimensional diffusions in a random environment associated with the generator ${\mathcal L}_W f(x)=e^{-ρ(x,W)}(e^{a(x,W)}f'(x))'$. The coefficients $\{ρ(x,\cdot):x\in\mathbb R\}$ and $\{a(x,\cdot):x\in\mathbb R\}$ are random. We assume that for each fixed realization of the environment, $ρ$ and $a$ are continuous and locally exponentially integrable, and that the support of the associated intrinsic coordinates is compact and non-collapsing. This framework includes the extensively studied Brox diffusion $dX_t=dB_t-\frac12\dot W(X_t)\,dt$, where $B$ is a standard Brownian motion and $W$ is an independent two-sided Brownian motion representing the environment. The Itô--McKean representation of the diffusions and the estimates of the first exit probabilities derived via Moser iteration play a crucial role.

math.PR↗