A dynamic programming principle for multiperiod control problems with bicausal constraints

Abstract

We consider multiperiod stochastic control problems with non-parametric uncertainty on the underlying probabilistic model. We derive a new metric on the space of probability measures, called the adapted $(p, \infty)$--Wasserstein distance $\mathcal{AW}_p^\infty$ with the following properties: (1) the adapted $(p, \infty)$--Wasserstein distance generates a topology that guarantees continuity of stochastic control problems and (2) the corresponding $\mathcal{AW}_p^\infty$-distributionally robust optimization (DRO) problem can be computed via a dynamic programming principle involving one-step Wasserstein-DRO problems. If the cost function is semi-separable, then we further show that a minimax theorem holds, even though balls with respect to $\mathcal{AW}_p^\infty$ are neither convex nor compact in general. We also derive first-order sensitivity results.