A Pontryagin Maximum Principle on the Belief Space for Continuous-Time Optimal Control with Discrete Observations

Abstract

We study a continuous time stochastic optimal control problem under partial observations that are available only at discrete time instants. This hybrid setting, with continuous dynamics and intermittent noisy measurements, arises in applications ranging from robotic exploration and target tracking to epidemic control. We formulate the problem on the space of beliefs (information states), treating the controller's posterior distribution of the state as the state variable for decision making. On this belief space we derive a Pontryagin maximum principle that provides necessary conditions for optimality. The analysis carefully tracks both the continuous evolution of the state between observation times and the Bayesian jump updates of the belief at observation instants. A key insight is a relationship between the adjoint process in our maximum principle and the gradient of the value functional on the belief space, which links the optimality conditions to the dynamic programming approach on the space of probability measures. The resulting optimality system has a prediction and update structure that is closely related to the unnormalised Zakai equation and the normalised Kushner-Stratonovich equation in nonlinear filtering. Building on this analysis, we design a particle based numerical scheme to approximate the coupled forward (filter) and backward (adjoint) system. The scheme uses particle filtering to represent the evolving belief and regression techniques to approximate the adjoint, which yields a practical algorithm for computing near optimal controls under partial information. The effectiveness of the approach is illustrated on both linear and nonlinear examples and highlights in particular the benefits of actively controlling the observation process.