arXiv subjects

Tryphon T. Georgiou

Publications and source records attributed to Tryphon T. Georgiou.

At least 91 records · Page 5Linked to original sources

arXiv2017

Gradient Flows in Filtering and Fisher-Rao Geometry

Uncertainty propagation and filtering can be interpreted as gradient flows with respect to suitable metrics in the infinite dimensional manifold of probability density functions. Such a viewpoint has been put forth in recent literature, and a systematic way to formulate and solve the same for linear Gaussian systems has appeared in our previous work where the gradient flows were realized via proximal operators with respect to Wasserstein metric arising in optimal mass transport. In this paper, we derive the evolution equations as proximal operators with respect to Fisher-Rao metric arising in information geometry. We develop the linear Gaussian case in detail and show that a template two step optimization procedure proposed earlier by the authors still applies. Our objective is to provide new geometric interpretations of known equations in filtering, and to clarify the implication of different choices of metric.

math.OC↗

arXiv2017

Gradient Flows in Uncertainty Propagation and Filtering of Linear Gaussian Systems

The purpose of this work is mostly expository and aims to elucidate the Jordan-Kinderlehrer-Otto (JKO) scheme for uncertainty propagation, and a variant, the Laugesen-Mehta-Meyn-Raginsky (LMMR) scheme for filtering. We point out that these variational schemes can be understood as proximal operators in the space of density functions, realizing gradient flows. These schemes hold the promise of leading to efficient ways for solving the Fokker-Planck equation as well as the equations of non-linear filtering. Our aim in this paper is to develop in detail the underlying ideas in the setting of linear stochastic systems with Gaussian noise and recover known results.

math.OC↗

arXiv2017

Efficient-robust routing for single commodity network flows

We study single commodity network flows with suitable robustness and efficiency specs. An original use of a maximum entropy problem for distributions on the paths of the graph turns this problem into a steering problem for Markov chains with prescribed initial and final marginals. From a computational standpoint, viewing scheduling this way is especially attractive in light of the existence of an iterative algorithm to compute the solution. The present paper builds on [13] by introducing an index of efficiency of a transportation plan and points, accordingly, to efficient-robust transport policies. In developing the theory, we establish two new invariance properties of the solution (called bridge) - an iterated bridge invariance property and the invariance of the most probable paths. These properties, which were tangentially mentioned in our previous work, are fully developed here. We also show that the distribution on paths of the optimal transport policy, which depends on a "temperature" parameter, tends to the solution of the "most economical" but possibly less robust optimal mass transport problem as the temperature goes to zero. The relevance of all of these properties for transport over networks is illustrated in an example.

math-ph↗

arXiv2017

An Efficient Algorithm for Matrix-Valued and Vector-Valued Optimal Mass Transport

We present an efficient algorithm for recent generalizations of optimal mass transport theory to matrix-valued and vector-valued densities. These generalizations lead to several applications including diffusion tensor imaging, color images processing, and multi-modality imaging. The algorithm is based on sequential quadratic programming (SQP). By approximating the Hessian of the cost and solving each iteration in an inexact manner, we are able to solve each iteration with relatively low cost while still maintaining a fast convergent rate. The core of the algorithm is solving a weighted Poisson equation, where different efficient preconditioners may be employed. We utilize incomplete Cholesky factorization, which yields an efficient and straightforward solver for our problem. Several illustrative examples are presented for both the matrix and vector-valued cases.

math.NA↗

arXiv2017

Vector-Valued Optimal Mass Transport

We introduce the problem of transporting vector-valued distributions. In this, a salient feature is that mass may flow between vectorial entries as well as across space (discrete or continuous). The theory relies on a first step taken to define an appropriate notion of optimal transport on a graph. The corresponding distance between distributions is readily computable via convex optimization and provides a suitable generalization of Wasserstein-type metrics. Building on this, we define Wasserstein-type metrics on vector-valued distributions supported on continuous spaces as well as graphs. Motivation for developing vector-valued mass transport is provided by applications such as multi-color image processing, polarimetric radar, as well as network problems where resources may be vectorial.

math.OC↗

arXiv2017

Matricial Wasserstein-1 Distance

In this note, we propose an extension of the Wasserstein 1-metric ($W_1$) for matrix probability densities, matrix-valued density measures, and an unbalanced interpretation of mass transport. The key is using duality theory, in particular, a "dual of the dual" formulation of $W_1$. This matrix analogue of the Earth Mover's Distance has several attractive features including ease of computation.

math.FA↗

arXiv2017

Interpolation of Density Matrices and Matrix-Valued Measures: The Unbalanced Case

In this note, we propose an unbalanced version of the quantum mechanical version of optimal mass transport that was based on the Lindblad equation. We formulate a natural interpolation framework between density matrices and matrix-valued measures via a quantum mechanical formulation of Fisher-Rao information and the matricial Wasserstein distance.

math.FA↗

arXiv2016

Regularization and Interpolation of Positive Matrices

We consider certain matricial analogues of optimal mass transport of positive definite matrices of equal trace. The framework is motivated by the need to devise a suitable geometry for interpolating positive definite matrices in ways that allow controlling the apparent tradeoff between "aligning up their eigenstructure" and "scaling the corresponding eigenvalues". Indeed, motivation for this work is provided by power spectral analysis of multivariate time series where, linear interpolation between matrix-valued power spectra generates push-pop artifacts. Push-pop of power distribuion is objectionable as it corresponds to unrealistic response of scatterers.

math.OC↗

arXiv2016

Color of turbulence

In this paper, we address the problem of how to account for second-order statistics of turbulent flows using low-complexity stochastic dynamical models based on the linearized Navier-Stokes equations. The complexity is quantified by the number of degrees of freedom in the linearized evolution model that are directly influenced by stochastic excitation sources. For the case where only a subset of velocity correlations are known, we develop a framework to complete unavailable second-order statistics in a way that is consistent with linearization around turbulent mean velocity. In general, white-in-time stochastic forcing is not sufficient to explain turbulent flow statistics. We develop models for colored-in-time forcing using a maximum entropy formulation together with a regularization that serves as a proxy for rank minimization. We show that colored-in-time excitation of the Navier-Stokes equations can also be interpreted as a low-rank modification to the generator of the linearized dynamics. Our method provides a data-driven refinement of models that originate from first principles and captures complex dynamics of turbulent flows in a way that is tractable for analysis, optimization, and control design.

physics.flu-dyn↗

arXiv2016

Matrix Optimal Mass Transport: A Quantum Mechanical Approach

In this paper, we describe a possible generalization of the Wasserstein 2-metric, originally defined on the space of scalar probability densities, to the space of Hermitian matrices with trace one, and to the space of matrix-valued probability densities. Our approach follows a computational fluid dynamical formulation of the Wasserstein-2 metric and utilizes certain results from the quantum mechanics of open systems, in particular the Lindblad equation. It allows determining the gradient flow for the quantum entropy relative to this matricial Wasserstein metric. This may have implications to some key issues in quantum information theory.

math-ph↗

arXiv2016

Optimal steering of a linear stochastic system to a final probability distribution, Part III

The subject of this work has its roots in the so called Schroedginer Bridge Problem (SBP) which asks for the most likely distribution of Brownian particles in their passage between observed empirical marginal distributions at two distinct points in time. Renewed interest in this problem was sparked by a reformulation in the language of stochastic control. In earlier works, presented as Part I and Part II, we explored a generalization of the original SBP that amounts to optimal steering of linear stochastic dynamical systems between state-distributions, at two points in time, under full state feedback. In these works the cost was quadratic in the control input. The purpose of the present work is to detail the technical steps in extending the framework to the case where a quadratic cost in the state is also present. In the zero-noise limit, we obtain the solution of a (deterministic) mass transport problem with general quadratic cost.

eess.SY↗

arXiv2016

Likelihood Analysis of Power Spectra and Generalized Moment Problems

We develop an approach to spectral estimation that has been advocated by Ferrante, Masiero and Pavon and, in the context of the scalar-valued covariance extension problem, by Enqvist and Karlsson. The aim is to determine the power spectrum that is consistent with given moments and minimizes the relative entropy between the probability law of the underlying Gaussian stochastic process to that of a prior. The approach is analogous to the framework of earlier work by Byrnes, Georgiou and Lindquist and can also be viewed as a generalization of the classical work by Burg and Jaynes on the maximum entropy method. In the present paper we present a new fast algorithm in the general case (i.e., for general Gaussian priors) and show that for priors with a specific structure the solution can be given in closed form.

eess.SY↗

arXiv2016

Low-complexity modeling of partially available second-order statistics: theory and an efficient matrix completion algorithm

State statistics of linear systems satisfy certain structural constraints that arise from the underlying dynamics and the directionality of input disturbances. In the present paper we study the problem of completing partially known state statistics. Our aim is to develop tools that can be used in the context of control-oriented modeling of large-scale dynamical systems. For the type of applications we have in mind, the dynamical interaction between state variables is known while the directionality and dynamics of input excitation is often uncertain. Thus, the goal of the mathematical problem that we formulate is to identify the dynamics and directionality of input excitation in order to explain and complete observed sample statistics. More specifically, we seek to explain correlation data with the least number of possible input disturbance channels. We formulate this inverse problem as rank minimization, and for its solution, we employ a convex relaxation based on the nuclear norm. The resulting optimization problem is cast as a semidefinite program and can be solved using general-purpose solvers. For problem sizes that these solvers cannot handle, we develop a customized alternating minimization algorithm (AMA). We interpret AMA as a proximal gradient for the dual problem and prove sub-linear convergence for the algorithm with fixed step-size. We conclude with an example that illustrates the utility of our modeling and optimization framework and draw contrast between AMA and the commonly used alternating direction method of multipliers (ADMM) algorithm.

math.OC↗

arXiv2016

Robust transport over networks

We consider transport over a strongly connected, directed graph. The scheduling amounts to selecting transition probabilities for a discrete-time Markov evolution which is designed to be consistent with certain initial and final marginals. The random evolution is selected to be closest to a prior measure on paths in the relative entropy sense, i.e., a Schroedinger bridge between the two marginals. This is an atypical stochastic control problem where the control consists in suitably modifying the transition mechanism. The prior can incorporate cost of traversing edges or allocate equal probability to all paths of equal length connecting any two given nodes, i.e., a uniform measure on paths. This latter choice relies on the so-called Ruelle-Bowen random walk and gives rise to a scheduling that tends to utilize all paths as uniformly as the topology allows. Thus, when the Ruelle-Bowen law is taken as prior, the transportation plan tends to lessen congestion and ensure a level of robustness. We show that the Ruelle-Bowen law is itself a Schroedinger bridge albeit with a prior that is not a probability measure. The paradigm of Schroedinger bridges as a mechanism for scheduling transport on networks can be adapted to graphs that are not strongly connected as well as to weighted graphs. The latter leads to transportation plans that effect a compromise between robustness and transportation cost.

eess.SY↗

arXiv2015

Optimal estimation with missing observations via balanced time-symmetric stochastic models

We consider data fusion for the purpose of smoothing and interpolation based on observation records with missing data. Stochastic processes are generated by linear stochastic models. The paper begins by drawing a connection between time reversal in stochastic systems and all-pass extensions. A particular normalization (choice of basis) between the two time-directions allows the two to share the same orthonormalized state process and simplifies the mathematics of data fusion. In this framework we derive symmetric and balanced Mayne-Fraser-like formulas that apply simultaneously to smoothing and interpolation.

math.OC↗

arXiv2015

The role of the time-arrow in mean-square estimation of stochastic processes

The purpose of this paper is to explain a certain dichotomy between the information that the past and future values of a multivariate stochastic process carry about the present. More specifically, vector-valued, second-order stochastic processes may be deterministic in one time-direction and not the other. This phenomenon, which is absent in scalar-valued processes, is deeply rooted in the geometry of the shift-operator. The exposition and the examples we discuss are based on the work of Douglas, Shapiro and Shields on cyclic vectors of the backward shift and relate to classical ideas going back to Wiener and Kolmogorov. We focus on rank-one stochastic processes for which we present a characterization of all regular processes that are deterministic in the reverse time-direction. The paper builds on examples and the goal is to provide pertinent insights to a control engineering audience.

math.OC↗

arXiv2015

Entropic and displacement interpolation: a computational approach using the Hilbert metric

Monge-Kantorovich optimal mass transport (OMT) provides a blueprint for geometries in the space of positive densities -- it quantifies the cost of transporting a mass distribution into another. In particular, it provides natural options for interpolation of distributions (displacement interpolation) and for modeling flows. As such it has been the cornerstone of recent developments in physics, probability theory, image processing, time-series analysis, and several other fields. In spite of extensive work and theoretical developments, the computation of OMT for large scale problems has remained a challenging task. An alternative framework for interpolating distributions, rooted in statistical mechanics and large deviations, is that of Schroedinger bridges (entropic interpolation). This may be seen as a stochastic regularization of OMT and can be cast as the stochastic control problem of steering the probability density of the state-vector of a dynamical system between two marginals. In this approach, however, the actual computation of flows had hardly received any attention. In recent work on Schroedinger bridges for Markov chains and quantum evolutions, we noted that the solution can be efficiently obtained from the fixed-point of a map which is contractive in the Hilbert metric. Thus, the purpose of this paper is to show that a similar approach can be taken in the context of diffusion processes which i) leads to a new proof of a classical result on Schroedinger bridges and ii) provides an efficient computational scheme for both, Schroedinger bridges and OMT. We illustrate this new computational approach by obtaining interpolation of densities in representative examples such as interpolation of images.

math.OC↗

arXiv2015

Steering state statistics with output feedback

Consider a linear stochastic system whose initial state is a random vector with a specified Gaussian distribution. Such a distribution may represent a collection of particles abiding by the specified system dynamics. In recent publications, we have shown that, provided the system is controllable, it is always possible to steer the state covariance to any specified terminal Gaussian distribution using state feedback. The purpose of the present work is to show that, in the case where only partial state observation is available, a necessary and sufficient condition for being able to steer the system to a specified terminal Gaussian distribution for the state vector is that the terminal state covariance be greater (in the positive-definite sense) than the error covariance of a corresponding Kalman filter.

math.OC↗