Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2101.00546

Optimal stopping time on discounted semi-Markov processes

Fang Chen·Xianping Guo·Zhong-Wei Liao
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper attempts to study the optimal stopping time for semi-Markov processes (SMPs) under the discount optimization criteria with unbounded cost rates. In our work, we introduce an explicit construction of the equivalent semi-Markov decision processes (SMDPs). The equivalence is embodied in the value functions of SMPs and SMDPs, that is, every stopping time of SMPs can induce a policy of SMDPs such that the value functions are equal, and vice versa. The existence of the optimal stopping time of SMPs is proved by this equivalence relation. Next, we give the optimality equation of the value function and develop an effective iterative algorithm for computing it. Moreover, we show that the optimal and ε-optimal stopping time can be characterized by the hitting time of the special sets. Finally, to illustrate the validity of our results, an example of a maintenance system is presented in the end.

Explore related subjects

math.PRmath.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Fang Chen, Xianping Guo, Zhong-Wei Liao. 2021-01-03. Optimal stopping time on discounted semi-Markov processes. https://arxiv.org/abs/2101.00546

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Villani's conjecture for Kac's walk

We prove Villani's conjecture on entropy production for Kac's walk with uniform collision angles. For every $N\geq2$, with total collision rate $N$, the optimal entropy production constant is $2/(N-1)$. We show that the known lower bound is sharp by constructing two families of smooth strictly positive probability densities, invariant under coordinate permutations and sign changes. The first consists of normalized products of inverse powers; the second is obtained by conditioning products of Gaussian mixtures on the energy sphere. With $N$ fixed, we compute the leading terms of entropy and entropy production. Successive parameter limits yield two independent proofs of sharpness.

math.PR↗
arXiv2026

Scaling limit and tail bounds for a random walk model of SOS level lines

This paper analyzes a random walk model for the level lines appearing in the entropic repulsion phenomena of three-dimensional discrete random interfaces above a hard wall; we are particularly motivated by the low-temperature (2+1)D solid-on-solid (SOS) model, where the emergence of these level lines has been rigorously established. The model we consider is a line ensemble of non-crossing random walk bridges above a wall with geometrically growing area tilts. Our main result, which in particular resolves a question of Caputo, Ioffe, and Wachtel (2019), is an edge 1:2:3 scaling limit for this ensemble as the domain size diverges, with a growing number of walks (including the number of level lines of the SOS model) and high boundary conditions (covering the maximum upper deviation of the SOS level lines). As a key input, we establish Tracy--Widom-type upper tail bounds for each of the relevant curves in the line ensemble. An ingredient which may be of independent interest is a ballot theorem for random walk bridges under a broader range of boundary values than available in the literature.

math.PR↗
arXiv2026

One-dimensional particle clouds with elastic collisions

We study an interacting particle system of a finite number of labelled particles on the integer lattice, in which particles have intrinsic masses and left/right jump rates. If a particle is the minimal-label particle at its site when it tries to jump left, the jump is executed. If not, 'momentum' is transferred to increase the rate of jumping left of the minimal-label particle. Similarly for jumps to the right. The collision rule is 'elastic' in the sense that the net rate of flow of mass is independent of the present configuration, in contrast to the exclusion process, for example. We show that the particle masses and jump rates determine explicitly, via a concave majorant of a simple `potential' function associated to the masses and jump rates, a unique partition of the system into maximal stable subsystems. The internal configuration of each stable subsystem remains tight, while the location of each stable subsystem obeys a strong law of large numbers with an explicit speed. We indicate connections to adjacent models, including diffusions with rank-based coefficients, where, to the authors' knowledge, the problem of identifying the analogous decomposition into stable sub-systems is yet to be fully solved.

math.PR↗