Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2505.02479

Reach-avoid semi-Markov decision processes with time-varying obstacles

Yanyun Li·Xianping Guo
View original source ↗Read source document ↗Export metadata ↓

Abstract

We consider the maximal reach-avoid probability to a target in finite horizon for semi-Markov decision processes with time-varying obstacles. Since the variance of the obstacle set, the model \eqref{Model} is non-homogeneous. To overcome such difficulty, we construct a related two-dimensional model \eqref{newModel}, and then prove the equivalence between such reach-avoid probability of the original model and that of the related two-dimensional one. For the related two-dimensional model, we analyze some special characteristics of the equivalent reach-avoid probability. On this basis, we provide a special improved value-type algorithm to obtain the equivalent maximal reach-avoid probability and its $ε$-optimal policy. Then, at the last step of the algorithm, by the equivalence between these two models, we obtain the original maximal reach-avoid probability and its $ε$-optimal policy for the original model.

Explore related subjects

math.PR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yanyun Li, Xianping Guo. 2025-05-05. Reach-avoid semi-Markov decision processes with time-varying obstacles. https://arxiv.org/abs/2505.02479

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Selection in the Zero-Noise Limit for High-Dimensional Diffusions with Measurable Drift

This paper investigates the zero-noise limit of high-dimensional small-noise diffusion processes governed by the stochastic differential equation (SDE) \begin{equation*} \mathrm{d}X_{t}^{\varepsilon }=b(X_{t}^{\varepsilon })\,\mathrm{d}t+\varepsilon \,\mathrm{d}W_{t},\quad X_{0}^{\varepsilon }=0,\quad \varepsilon >0, \end{equation*} where the drift coefficient $b$ is assumed to be measurable and bounded. Under this condition, the associated deterministic ordinary differential equation (ODE) $\dot{x}_{t}=b(x_{t})$ may admit multiple Filippov solutions, i.e., solutions in the sense of differential inclusions, due to the lack of Lipschitz continuity or uniqueness criteria. However, the introduction of non-degenerate additive noise restores well-posedness: the perturbed system admits a unique strong (pathwise) solution for each $\varepsilon >0$. Our analysis is based on the radial-spherical decomposition. We prove that, under certain assumptions, the radius dominates a one-dimensional comparison process and escapes to infinity at a linear rate, and the extrinsic angular martingale converges. Under an explicit separable gradient structure, expressed in terms of a radial potential $V(r,ω)=\langle b(rω),ω\rangle $, the angle converges, for every fixed noise, almost surely to the set of global angular maximizers of $V$, and its distance from that set tends to zero in probability as $\varepsilon\rightarrow0$. The limiting measure is compact, carried by immediate-departure Filippov paths, and, under a Lipschitz radial-graph condition, purely singular with Hausdorff dimension at most $d-1$. A continuous-mapping criterion for exit-time convergence is also established.

math.PR↗
arXiv2026

Asymptotic optimality of dynamic first-fit packing on the half-axis

We revisit a classical problem in dynamic storage allocation. Items arrive in a linear storage medium, modeled as a half-axis, at a Poisson rate $r$ and depart after an independent exponentially distributed unit mean service time. The arriving item sizes (lengths) are assumed to be independent and identically distributed (i.i.d.) from a common distribution $H$. A widely employed algorithm for allocating the items is the "first-fit" discipline, namely, each arriving item is placed in the left-most vacant interval large enough to accommodate it. In a seminal 1985 paper, Coffman, Kadota, and Shepp ([6]) proved that in the special case of unit length items (i.e. degenerate $H$), as $r$ tends towards infinity, the first-fit algorithm is asymptotically optimal in the following sense: the steady-state ratio of expected "empty space" (gaps between items) to expected occupied space tends towards $0$. In a sequel to [6], Coffman, Kadota, and Shepp ([5]) conjectured that the first-fit discipline is also asymptotically optimal for non-degenerate $H$. In this paper we provide the first proof of first-fit asymptotic optimality for non-degenerate distributions $H$ of item sizes. Our main result is for the case when $H$ is concentrated on countably many positive real sizes forming an increasing sequence that is either finite or goes to infinity, with the average item size being finite. We prove that under the first-fit discipline, as $r$ tends towards infinity, the steady-state packing configuration (scaled down by $r$) converges in distribution to the limiting packing configuration with smaller items on the left, larger items on the right, and with no gaps between. In particular, this proves asymptotic optimality of first-fit in the following sense: if $P$ is the expected occupied space, then in steady-state the empty space (scaled down by $r$) in $[0,P]$ vanishes.

math.PR↗
arXiv2026

Mind the jumps: well-posedness of semi-martingale 2BSDEs

We develop a well-posedness theory for second-order backward stochastic differential equations (2BSDEs) with jumps. This work covers two complementary notions of solution: an extrinsic and an intrinsic one. We also discuss the obstruction to aggregating jump integrands. The chosen framework allows for controlled diffusions with jumps, pure-jump processes, and discrete-time processes in a unified setting.

math.PR↗