Innovation-Based Sampling for New Information

Abstract

This work introduces innovation-based sampling of continuous stochastic processes, in which a sample is generated only when the process reveals ``sufficiently new'' information relative to all previous samples. The resulting innovation process admits a lattice structure and a three-dimensional state representation. For a Wiener process, we characterize the direction, timing, and frequency of innovations. We show that direction reversals become increasingly rare and establish limit theorems for their number. Innovations also become progressively sparser: the expected number of innovations grows only as the square root of time, and hence the sampling rate vanishes asymptotically. We then study remote estimation from sparsely received innovations. Notably, the absence of an innovation carries information: the minimum mean-square error (MMSE) estimate evolves with the age of information (AoI) and achieves a substantial MSE reduction over the conventional silence-ignorant zero-order hold (ZOH) estimator. Finally, we develop tractable affine-age and exponential-age approximations for practical use. Overall, information is conveyed not only by the content of innovations, but also by their direction and timing.