arXiv subjects

Yongxin Chen

Publications and source records attributed to Yongxin Chen.

At least 163 records · Page 9Linked to original sources

arXiv2021

Stein particle filtering

We present a new particle filtering algorithm for nonlinear systems in the discrete-time setting. Our algorithm is based on the Stein variational gradient descent (SVGD) framework, which is a general approach to sample from a target distribution. We merge the standard two-step paradigm in particle filtering into one step so that SVGD can be used. A distinguishing feature of the proposed algorithm is that, unlike most particle filtering methods, all the particles at any time step are equally weighted and thus no update on the weights is needed. We further extended our algorithm to allow for updating previous particles within a sliding window. This strategy may improve the reliability of the algorithm with respect to unexpected disturbance in the dynamics or outlier-measurements. The efficacy of the proposed algorithms is illustrated through several numerical examples in comparison with a standard particle filtering method.

cs.CE↗

arXiv2021

Learning High Dimensional Wasserstein Geodesics

We propose a new formulation and learning strategy for computing the Wasserstein geodesic between two probability distributions in high dimensions. By applying the method of Lagrange multipliers to the dynamic formulation of the optimal transport (OT) problem, we derive a minimax problem whose saddle point is the Wasserstein geodesic. We then parametrize the functions by deep neural networks and design a sample based bidirectional learning algorithm for training. The trained networks enable sampling from the Wasserstein geodesic. As by-products, the algorithm also computes the Wasserstein distance and OT map between the marginal distributions. We demonstrate the performance of our algorithms through a series of experiments with both synthetic and realistic data.

cs.LG↗

arXiv2021

Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications

The min-max problem, also known as the saddle point problem, is a class of optimization problems which minimizes and maximizes two subsets of variables simultaneously. This class of problems can be used to formulate a wide range of signal processing and communication (SPCOM) problems. Despite its popularity, most existing theory for this class has been mainly developed for problems with certain special convex-concave structure. Therefore, it cannot be used to guide the algorithm design for many interesting problems in SPCOM, where various kinds of non-convexity arise. In this work, we consider a block-wise one-sided non-convex min-max problem, in which the minimization problem consists of multiple blocks and is non-convex, while the maximization problem is (strongly) concave. We propose a class of simple algorithms named Hybrid Block Successive Approximation (HiBSA), which alternatingly perform gradient descent-type steps for the minimization blocks and gradient ascent-type steps for the maximization problem. A key element in the proposed algorithm is the use of certain regularization and penalty sequences, which stabilize the algorithm and ensure convergence. We show that HiBSA converges to some properly defined first-order stationary solutions with quantifiable global rates. To validate the efficiency of the proposed algorithms, we conduct numerical tests on a number of problems, including the robust learning problem, the non-convex min-utility maximization problems, and certain wireless jamming problem arising in interfering channels.

math.OC↗

arXiv2021

Multi-marginal Optimal Transport with a Tree-structured cost and the Schrödinger Bridge Problem

The optimal transport problem has recently developed into a powerful framework for various applications in estimation and control. Many of the recent advances in the theory and application of optimal transport are based on regularizing the problem with an entropy term, which connects it to the Schrödinger bridge problem and thus to stochastic optimal control. Moreover, the entropy regularization makes the otherwise computationally demanding optimal transport problem feasible even for large scale settings. This has lead to an accelerated development of optimal transport based methods in a broad range of fields. Many of these applications have a underlying graph structure, for instance information fusion and tracking problems can be described by trees. In this work we consider multi-marginal optimal transport problems with a cost function that decouples according to a tree structure. The entropy regularized multi-marginal optimal transport problem can be viewed as a generalization of the Schrödinger bridge problem with the same tree-structure, and by utilizing these connections we extend the computational methods for the classical optimal transport problem in order to solve structured multi-marginal optimal transport problems in an efficient manner. In particular, the algorithm requires only matrix-vector multiplications of relatively small dimensions. We show that the multi-marginal regularization introduces less diffusion, compared to the commonly used pairwise regularization, and is therefore more suitable for many applications. Numerical examples illustrate this, and we finally apply the proposed framework for tracking of an ensemble of indistinguishable agents.

math.OC↗

arXiv2021

On the relation between information and power in stochastic thermodynamic engines

The common saying, that information is power, takes a rigorous form in stochastic thermodynamics, where a quantitative equivalence between the two helps explain the paradox of Maxwell's demon in its ability to reduce entropy. In the present paper, we build on earlier work on the interplay between the relative cost and benefits of information in producing work in cyclic operation of thermodynamic engines (by Sandberg etal. 2014). Specifically, we study the general case of overdamped particles in a time-varying potential (control action) in feedback that utilizes continuous measurements (nonlinear filtering) of a thermodynamic ensemble, to produce suitable adaptations of the second law of thermodynamics that involve information.

cond-mat.stat-mech↗

arXiv2021

Optimal steering to invariant distributions for networks flows

We derive novel results on the ergodic theory of irreducible, aperiodic Markov chains. We show how to optimally steer the network flow to a stationary distribution over a finite or infinite time horizon. Optimality is with respect to an entropic distance between distributions on feasible paths. When the prior is reversible, it shown that solutions to this discrete time and space steering problem are reversible as well. A notion of temperature is defined for Boltzmann distributions on networks, and problems analogous to cooling (in this case, for evolutions in discrete space and time) are discussed.

eess.SY↗

arXiv2021

Feedback Particle Filter for Collective Inference

The purpose of this paper is to describe the feedback particle filter algorithm for problems where there are a large number ($M$) of non-interacting agents (targets) with a large number ($M$) of non-agent specific observations (measurements) that originate from these agents. In its basic form, the problem is characterized by data association uncertainty whereby the association between the observations and agents must be deduced in addition to the agent state. In this paper, the large-$M$ limit is interpreted as a problem of collective inference. This viewpoint is used to derive the equation for the empirical distribution of the hidden agent states. A feedback particle filter (FPF) algorithm for this problem is presented and illustrated via numerical simulations. Results are presented for the Euclidean and the finite state-space cases, both in continuous-time settings. The classical FPF algorithm is shown to be the special case (with $M=1$) of these more general results. The simulations help show that the algorithm well approximates the empirical distribution of the hidden states for large $M$.

math.OC↗

arXiv2021

Harvesting energy from a periodic heat bath

The context of the present paper is stochastic thermodynamics - an approach to nonequilibrium thermodynamics rooted within the broader framework of stochastic control. In contrast to the classical paradigm of Carnot engines, we herein propose to consider thermodynamic processes with periodic continuously varying temperature of a heat bath and study questions of maximal power and efficiency for two idealized cases, overdamped (first-order) and underdamped (second-order) stochastic models. We highlight properties of optimal periodic control, derive and numerically validate approximate formulae for the optimal performance (power and efficiency).

eess.SY↗

arXiv2020

Stochastic control liaisons: Richard Sinkhorn meets Gaspard Monge on a Schroedinger bridge

In 1931/32, Schroedinger studied a hot gas Gedankenexperiment, an instance of large deviations of the empirical distribution and an early example of the so-called maximum entropy inference method. This so-called Schroedinger bridge problem (SBP) was recently recognized as a regularization of the Monge-Kantorovich Optimal Mass Transport (OMT), leading to effective computation of the latter. Specifically, OMT with quadratic cost may be viewed as a zero-temperature limit of SBP, which amounts to minimization of the Helmholtz's free energy over probability distributions constrained to possess given marginals. The problem features a delicate compromise, mediated by a temperature parameter, between minimizing the internal energy and maximizing the entropy. These concepts are central to a rapidly expanding area of modern science dealing with the so-called {\em Sinkhorn algorithm} which appears as a special case of an algorithm first studied by the French analyst Robert Fortet in 1938/40 specifically for Schroedinger bridges. Due to the constraint on end-point distributions, dynamic programming is not a suitable tool to attack these problems. Instead, Fortet's iterative algorithm and its discrete counterpart, the Sinkhorn iteration, permit computation by iteratively solving the so-called {\em Schroedinger system}. In both the continuous as well as the discrete-time and space settings, {\em stochastic control} provides a reformulation and dynamic versions of these problems. The formalism behind these control problems have attracted attention as they lead to a variety of new applications in spacecraft guidance, control of robot or biological swarms, sensing, active cooling, network routing as well as in computer and data science. This multifacet and versatile framework, intertwining SBP and OMT, provides the substrate for a historical and technical overview of the field taken up in this paper.

math.OC↗

arXiv2020

Filtering for Aggregate Hidden Markov Models with Continuous Observations

We consider a class of filtering problems for large populations where each individual is modeled by the same hidden Markov model (HMM). In this paper, we focus on aggregate inference problems in HMMs with discrete state space and continuous observation space. The continuous observations are aggregated in a way such that the individuals are indistinguishable from measurements. We propose an aggregate inference algorithm called continuous observation collective forward-backward algorithm. It extends the recently proposed collective forward-backward algorithm for aggregate inference in HMMs with discrete observations to the case of continuous observations. The efficacy of this algorithm is illustrated through several numerical experiments.

stat.ML↗

arXiv2020

Inference with Aggregate Data: An Optimal Transport Approach

We consider inference (filtering) problems over probabilistic graphical models with aggregate data generated by a large population of individuals. We propose a new efficient belief propagation type algorithm over tree-structured graphs with polynomial computational complexity as well as a global convergence guarantee. This is in contrast to previous methods that either exhibit prohibitive complexity as the population grows or do not guarantee convergence. Our method is based on optimal transport, or more specifically, multi-marginal optimal transport theory. In particular, we consider an inference problem with aggregate observations, that can be seen as a structured multi-marginal optimal transport problem where the cost function decomposes according to the underlying graph. Consequently, the celebrated Sinkhorn/iterative scaling algorithm for multi-marginal optimal transport can be leveraged together with the standard belief propagation algorithm to establish an efficient inference scheme which we call Sinkhorn belief propagation (SBP). We further specialize the SBP algorithm to cases associated with hidden Markov models due to their significance in control and estimation. We demonstrate the performance of our algorithm on applications such as inferring population flow from aggregate observations. We also show that in the special case where the observations are generated by a single individual, our algorithm naturally reduces to the standard belief propagation algorithm.

cs.LG↗

arXiv2020

Macroscopic network circulation for planar graphs

The analysis of networks, aimed at suitably defined functionality, often focuses on partitions into subnetworks that capture desired features. Chief among the relevant concepts is a 2-partition, that underlies the classical Cheeger inequality, and highlights a constriction (bottleneck) that limits accessibility between the respective parts of the network. In a similar spirit, the purpose of the present work is to introduce a new concept of maximal global circulation and to explore 3-partitions that expose this type of macroscopic feature of networks. Herein, graph circulation is motivated by transportation networks and probabilistic flows (Markov chains) on graphs. Our goal is to quantify the large-scale imbalance of network flows and delineate key parts that mediate such global features. While we introduce and propose these notions in a general setting, in this paper, we only work out the case of planar graphs. We explain that a scalar potential can be identified to encapsulate the concept of circulation, quite similarly as in the case of the curl of planar vector fields. Beyond planar graphs, in the general case, the problem to determine global circulation remains at present a combinatorial problem.

cs.DM↗

arXiv2020

On Spectral Properties of Signed Laplacians with Connections to Eventual Positivity

Signed graphs have appeared in a broad variety of applications, ranging from social networks to biological networks, from distributed control and computation to power systems. In this paper, we investigate spectral properties of signed Laplacians for undirected signed graphs. We find conditions on the negative weights under which a signed Laplacian is positive semidefinite via the Kron reduction and multiport network theory. For signed Laplacians that are indefinite, we characterize their inertias with the same framework. Furthermore, we build connections between signed Laplacians, generalized M-matrices, and eventually exponentially positive matrices.

eess.SY↗

arXiv2020

A convex data-driven approach for nonlinear control synthesis

We consider a class of nonlinear control synthesis problems where the underlying mathematical models are not explicitly known. We propose a data-driven approach to stabilize the systems when only sample trajectories of the dynamics are accessible. Our method is founded on the density function based almost everywhere stability certificate that is dual to the Lyapunov function for dynamic systems. Unlike Lyapunov based methods, density functions lead to a convex formulation for a joint search of the control strategy and the stability certificate. This type of convex problem can be solved efficiently by invoking the machinery of the sum of squares (SOS). For the data-driven part, we exploit the fact that the duality results in the stability theory of the dynamical system can be understood using linear Perron-Frobenius and Koopman operators. This connection allows us to use data-driven methods developed to approximate these operators combined with the SOS techniques for the convex formulation of control synthesis. The efficacy of the proposed approach is demonstrated through several examples.

eess.SY↗

arXiv2020

Incremental inference of collective graphical models

We consider incremental inference problems from aggregate data for collective dynamics. In particular, we address the problem of estimating the aggregate marginals of a Markov chain from noisy aggregate observations in an incremental (online) fashion. We propose a sliding window Sinkhorn belief propagation (SW-SBP) algorithm that utilizes a sliding window filter of the most recent noisy aggregate observations along with encoded information from discarded observations. Our algorithm is built upon the recently proposed multi-marginal optimal transport based SBP algorithm that leverages standard belief propagation and Sinkhorn algorithm to solve inference problems from aggregate data. We demonstrate the performance of our algorithm on applications such as inferring population flow from aggregate observations.

stat.ML↗

arXiv2020

Multi-marginal optimal transport and probabilistic graphical models

We study multi-marginal optimal transport problems from a probabilistic graphical model perspective. We point out an elegant connection between the two when the underlying cost for optimal transport allows a graph structure. In particular, an entropy regularized multi-marginal optimal transport is equivalent to a Bayesian marginal inference problem for probabilistic graphical models with the additional requirement that some of the marginal distributions are specified. This relation on the one hand extends the optimal transport as well as the probabilistic graphical model theories, and on the other hand leads to fast algorithms for multi-marginal optimal transport by leveraging the well-developed algorithms in Bayesian inference. Several numerical examples are provided to highlight the results.

math.OC↗

arXiv2020

Regularized transport between singular covariance matrices

We consider the problem of steering a linear stochastic system between two end-point degenerate Gaussian distributions in finite time. This accounts for those situations in which some but not all of the state entries are uncertain at the initial, t = 0, and final time, t = T . This problem entails non-trivial technical challenges as the singularity of terminal state-covariance causes the control to grow unbounded at the final time T. Consequently, the entropic interpolation (Schroedinger Bridge) is provided by a diffusion process which is not finite-energy, thereby placing this case outside of most of the current theory. In this paper, we show that a feasible interpolation can be derived as a limiting case of earlier results for non-degenerate cases, and that it can be expressed in closed form. Moreover, we show that such interpolation belongs to the same reciprocal class of the uncontrolled evolution. By doing so we also highlight a time-symmetry of the problem, contrasting dual formulations in the forward and reverse time-directions, where in each the control grows unbounded as time approaches the end-point (in the forward and reverse time-direction, respectively).

math.OC↗

arXiv2020

Improving Robustness via Risk Averse Distributional Reinforcement Learning

One major obstacle that precludes the success of reinforcement learning in real-world applications is the lack of robustness, either to model uncertainties or external disturbances, of the trained policies. Robustness is critical when the policies are trained in simulations instead of real world environment. In this work, we propose a risk-aware algorithm to learn robust policies in order to bridge the gap between simulation training and real-world implementation. Our algorithm is based on recently discovered distributional RL framework. We incorporate CVaR risk measure in sample based distributional policy gradients (SDPG) for learning risk-averse policies to achieve robustness against a range of system disturbances. We validate the robustness of risk-aware SDPG on multiple environments.

cs.LG↗