arXiv subjects

Alexei Novikov

Publications and source records attributed to Alexei Novikov.

At least 19 recordsLinked to original sources

arXiv2026

Iterative Atom Refinement: A Monotonicity Principle for Dictionary Learning

Dictionary learning seeks to recover an unknown dictionary $A$ from observations ${\bf y}_i = A{\bf x}_i$ with sparse coefficient vectors ${\bf x}_i$. We introduce the \emph{Iterative Atom Refinement} (IAR) algorithm, a simple procedure for recovering individual dictionary atoms. Starting from a random direction, IAR repeatedly selects the observations most strongly correlated with the current iterate and updates the direction by averaging the selected data. Our main contribution is a rigorous convergence theory of IAR. Using high-dimensional probabilistic estimates and a novel monotonicity principle for atom-selection probabilities, we show that a small initial advantage of one atom is amplified until that atom is isolated. Under our model assumptions, IAR identifies a generating atom after only three refinement steps. Numerical experiments support the theory and show that the resulting dynamics accurately capture the behavior observed in dictionary refinement.

cs.LG↗

arXiv2026

Imaging with super-resolution in changing random media

We develop an imaging algorithm that exploits strong scattering to achieve super-resolution in changing random media. The method processes large and diverse array datasets using sparse dictionary learning, clustering, and multidimensional scaling. Starting from random initializations, the algorithm reliably extracts the unknown medium properties necessary for accurate imaging using back-propagation, $\ell_2$ or $\ell_1$ methods. Remarkably, scattering enhances resolution beyond homogeneous medium limits. When abundant data are available, the algorithm allows the realization of super-resolution in imaging.

physics.optics↗

arXiv2024

Super-resolution in disordered media using neural networks

We propose a methodology that exploits large and diverse data sets to accurately estimate the ambient medium's Green's functions in strongly scattering media. Given these estimates, obtained with and without the use of neural networks, excellent imaging results are achieved, with a resolution that is better than that of a homogeneous medium. This phenomenon, also known as super-resolution, occurs because the ambient scattering medium effectively enhances the physical imaging aperture. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.

cs.LG↗

arXiv2024

Diffusion of knowledge and the lottery society

The Lucas-Moll system is a mean-field game type model describing the growth of an economy by means of diffusion of knowledge. The individual agents in the economy advance their knowledge by learning from each other and via internal innovation. Their cumulative distribution function satisfies a forward in time nonlinear non-local reaction-diffusion type equation. On the other hand, the learning strategy of the agents is based on the solution to a backward in time nonlocal Hamilton-Jacobi-Bellman equation that is coupled to the aforementioned equation for the agents density. Together, these equations form a system of the mean-field game type. When the learning rate is sufficiently large, existence of balanced growth path solutions to the Lucas-Moll system was proved in~\cite{PRV,Porretta-Rossi}. Here, we analyze a complementary regime where the balanced growth paths do not exist. The main result is a long time convergence theorem. Namely, the solution to the initial-terminal value problem behaves in such a way that at large times an overwhelming majority of the agents spend no time producing at all and are only learning. In particular, the agents density propagates at the Fisher-KPP speed. We name this type of solutions a lottery society.

math.AP↗

arXiv2023

Wave-informed dictionary learning for high-resolution imaging in complex media

We propose an approach for imaging in scattering media when large and diverse data sets are available. It has two steps. Using a dictionary learning algorithm the first step estimates the true Green's function vectors as columns in an unordered sensing matrix. The array data comes from many sparse sets of sources whose location and strength are not known to us. In the second step, the columns of the estimated sensing matrix are ordered for imaging using Multi-Dimensional Scaling with connectivity information derived from cross-correlations of its columns, as in time reversal. For these two steps to work together we need data from large arrays of receivers so the columns of the sensing matrix are incoherent for the first step, as well as from sub-arrays so that they are coherent enough to obtain the connectivity needed in the second step. Through simulation experiments, we show that the proposed approach is able to provide images in complex media whose resolution is that of a homogeneous medium.

cs.CV↗

arXiv2023

Speeding up Langevin Dynamics by Mixing

We study an overdamped Langevin equation on the $d$-dimensional torus with stationary distribution proportional to~$p = e^{-U / κ}$. When~$U$ has multiple wells the mixing time of the associated process is exponentially large (of size~$e^{O(1/κ)}$). We add a drift to the Langevin dynamics (without changing the stationary distribution) and obtain quantitative estimates on the mixing time. We show that an exponentially mixing drift can be rescaled to make the mixing time of the Langevin system arbitrarily small. For numerical purposes, it is useful to keep the size of the imposed drift small, and we show that the smallest allowable rescaling ensures that the mixing time is $O( d/κ^2)$, which is an order of magnitude smaller than~$e^{O(1/κ)}$. We provide one construction of an exponentially mixing drift, although with rate constants whose~$κ$-dependence is unknown.Heuristics (from discrete time) suggest that $κ$-dependence of the mixing rate is such that the imposed drift is of size~$O(d / κ^3)$. The large amplitude of the imposed drift increases the numerical complexity, and thus we expect this method will be most useful in the initial phase of Monte Carlo methods to rapidly explore the state space.

math.PR↗

arXiv2023

An oscillator driven by algebraically decorrelating noise

We consider a stochastically forced nonlinear oscillator driven by a stationary Gaussian noise that has an algebraically decaying covariance function. It is well known that such noise processes can be renormalized to converge to fractional Brownian motion, a process that has memory. In contrast, we show that the renormalized limit of the nonlinear oscillator driven by this noise converges to diffusion driven by standard (not fractional) Brownian motion, and thus retains no memory in the scaling limit. The proof is based on the study of a fast-slow system using the perturbed test function method.

math.PR↗

arXiv2023

Dictionary Learning for the Almost-Linear Sparsity Regime

Dictionary learning, the problem of recovering a sparsely used matrix $\mathbf{D} \in \mathbb{R}^{M \times K}$ and $N$ $s$-sparse vectors $\mathbf{x}_i \in \mathbb{R}^{K}$ from samples of the form $\mathbf{y}_i = \mathbf{D}\mathbf{x}_i$, is of increasing importance to applications in signal processing and data science. When the dictionary is known, recovery of $\mathbf{x}_i$ is possible even for sparsity linear in dimension $M$, yet to date, the only algorithms which provably succeed in the linear sparsity regime are Riemannian trust-region methods, which are limited to orthogonal dictionaries, and methods based on the sum-of-squares hierarchy, which requires super-polynomial time in order to obtain an error which decays in $M$. In this work, we introduce SPORADIC (SPectral ORAcle DICtionary Learning), an efficient spectral method on family of reweighted covariance matrices. We prove that in high enough dimensions, SPORADIC can recover overcomplete ($K > M$) dictionaries satisfying the well-known restricted isometry property (RIP) even when sparsity is linear in dimension up to logarithmic factors. Moreover, these accuracy guarantees have an ``oracle property" that the support and signs of the unknown sparse vectors $\mathbf{x}_i$ can be recovered exactly with high probability, allowing for arbitrarily close estimation of $\mathbf{D}$ with enough samples in polynomial time. To the author's knowledge, SPORADIC is the first polynomial-time algorithm which provably enjoys such convergence guarantees for overcomplete RIP matrices in the near-linear sparsity regime.

cs.LG↗

arXiv2022

Quantitative phase and absorption contrast imaging

We present an algorithm for coherent diffractive imaging with phaseless measurements. It treats the forward model as a combination of coherent and incoherent waves. The algorithm reconstructs absorption and phase contrast that quantifies the attenuation and the refraction of the waves propagating through an object. It requires coherent or partially coherent illuminations, and several detectors to record the intensity of the distorted wave that passes through the object under inspection. The diversity of illuminations, obtained by putting masks between the source and the object, provides enough information for imaging. The computational cost of our algorithm is linear in the number of pixels of the image. Therefore, it is efficient for high-resolution imaging. Our algorithm guarantees exact recovery if the image is sparse for a given basis. Numerical experiments in the setting of phaseless diffraction imaging of sparse objects validate the efficiency and the precision of the suggested algorithm.

physics.comp-ph↗

arXiv2021

Support Recovery for Sparse Multidimensional Phase Retrieval

We consider the \textit{phase retrieval} problem of recovering a sparse signal $\mathbf{x}$ in $\mathbb{R}^d$ from intensity-only measurements in dimension $d \geq 2$. Phase retrieval can be equivalently formulated as the problem of recovering a signal from its autocorrelation, which is in turn directly related to the combinatorial problem of recovering a set from its pairwise differences. In one spatial dimension, this problem is well studied and known as the \textit{turnpike problem}. In this work, we present MISTR (Multidimensional Intersection Sparse supporT Recovery), an algorithm which exploits this formulation to recover the support of a multidimensional signal from magnitude-only measurements. MISTR takes advantage of the structure of multiple dimensions to provably achieve the same accuracy as the best one-dimensional algorithms in dramatically less time. We prove theoretically that MISTR correctly recovers the support of signals distributed as a Gaussian point process with high probability as long as sparsity is at most $\mathcal{O}\left(n^{dθ}\right)$ for any $θ< 1/2$, where $n^d$ represents pixel size in a fixed image window. In the case that magnitude measurements are corrupted by noise, we provide a thresholding scheme with theoretical guarantees for sparsity at most $\mathcal{O}\left(n^{dθ}\right)$ for $θ< 1/4$ that obviates the need for MISTR to explicitly handle noisy autocorrelation data. Detailed and reproducible numerical experiments demonstrate the effectiveness of our algorithm, showing that in practice MISTR enjoys time complexity which is nearly linear in the size of the input.

math.CO↗

arXiv2021

Thresholding Greedy Pursuit for Sparse Recovery Problems

We study here sparse recovery problems in the presence of additive noise. We analyze a thresholding version of the CoSaMP algorithm, named Thresholding Greedy Pursuit (TGP). We demonstrate that an appropriate choice of thresholding parameter, even without the knowledge of sparsity level of the signal and strength of the noise, can result in exact recovery with no false discoveries as the dimension of the data increases to infinity.

eess.SP↗

arXiv2020

Fast signal recovery from quadratic measurements

We present a novel approach for recovering a sparse signal from cross-correlated data. Cross-correlations naturally arise in many fields of imaging, such as optics, holography and seismic interferometry. Compared to the sparse signal recovery problem that uses linear measurements, the unknown is now a matrix formed by the cross correlation of the unknown signal. Hence, the bottleneck for inversion is the number of unknowns that grows quadratically. The main idea of our proposed approach is to reduce the dimensionality of the problem by recovering only the diagonal of the unknown matrix, whose dimension grows linearly with the size of the problem. The keystone of the methodology is the use of an efficient {\em Noise Collector} that absorbs the data that come from the off-diagonal elements of the unknown matrix and that do not carry extra information about the support of the signal. This results in a linear problem whose cost is similar to the one that uses linear measurements. Our theory shows that the proposed approach provides exact support recovery when the data is not too noisy, and that there are no false positives for any level of noise. Moreover, our theory also demonstrates that when using cross-correlated data, the level of sparsity that can be recovered increases, scaling almost linearly with the number of data. The numerical experiments presented in the paper corroborate these findings.

eess.SP↗

arXiv2019

The Noise Collector for sparse recovery in high dimensions

The ability to detect sparse signals from noisy high-dimensional data is a top priority in modern science and engineering. A sparse solution of the linear system $A ρ= b_0$ can be found efficiently with an $l_1$-norm minimization approach if the data is noiseless. Detection of the signal's support from data corrupted by noise is still a challenging problem, especially if the level of noise must be estimated. We propose a new efficient approach that does not require any parameter estimation. We introduce the Noise Collector (NC) matrix $C$ and solve an augmented system $A ρ+ C η= b_0 + e$, where $ e$ is the noise. We show that the $l_1$-norm minimal solution of the augmented system has zero false discovery rate for any level of noise and with probability that tends to one as the dimension of $ b_0$ increases to infinity. We also obtain exact support recovery if the noise is not too large, and develop a Fast Noise Collector Algorithm which makes the computational cost of solving the augmented system comparable to that of the original one. Finally, we demonstrate the effectiveness of the method in applications to passive array imaging.

eess.SP↗

arXiv2019

Imaging with highly incomplete and corrupted data

We consider the problem of imaging sparse scenes from a few noisy data using an $l_1$-minimization approach. This problem can be cast as a linear system of the form $A \, ρ=b$, where $A$ is an $N\times K$ measurement matrix. We assume that the dimension of the unknown sparse vector $ρ\in {\mathbb{C}}^K$ is much larger than the dimension of the data vector $b \in {\mathbb{C}}^N$, i.e, $K \gg N$. We provide a theoretical framework that allows us to examine under what conditions the $\ell_1$-minimization problem admits a solution that is close to the exact one in the presence of noise. Our analysis shows that $l_1$-minimization is not robust for imaging with noisy data when high resolution is required. To improve the performance of $l_1$-minimization we propose to solve instead the augmented linear system $ [A \, | \, C] ρ=b$, where the $N \times Σ$ matrix $C$ is a noise collector. It is constructed so as its column vectors provide a frame on which the noise of the data, a vector of dimension $N$, can be well approximated. Theoretically, the dimension $Σ$ of the noise collector should be $e^N$ which would make its use not practical. However, our numerical results illustrate that robust results in the presence of noise can be obtained with a large enough number of columns $Σ\approx 10 K$.

eess.IV↗

arXiv2019

Synthetic aperture imaging with intensity-only data

We consider imaging the reflectivity of scatterers from intensity-only data recorded by a single moving transducer that both emits and receives signals, forming a synthetic aperture. By exploiting frequency illumination diversity, we obtain multiple intensity measurements at each location, from which we determine field cross-correlations using an appropriate phase controlled illumination strategy and the inner product polarization identity. The field cross-correlations obtained this way do not, however, provide all the missing phase information because they are determined up to a phase that depends on the receiver's location. The main result of this paper is an algorithm with which we recover the field cross-correlations up to a single phase that is common to all the data measured over the synthetic aperture, so all the data are synchronized. Thus, we can image coherently with data over all frequencies and measurement locations as if full phase information was recorded.

physics.comp-ph↗

arXiv2019

Feeble fish in time-dependent waters and homogenization of the G-equation

We study the following control problem. A fish with bounded aquatic locomotion speed swims in fast waters. Can this fish, under reasonable assumptions, get to a desired destination? It can, even if the flow is time-dependent. Moreover, given a prescribed sufficiently large time $t$, it can be there at exactly the time $t$. The major difference from our previous work is the time-dependence of the flow. We also give an application to homogenization of the G-equation.

math.OC↗

arXiv2019

Robust multifrequency imaging with MUSIC

In this paper, we study the MUltiple SIgnal Classification (MUSIC) algorithm often used to image small targets when multiple measurement vectors are available. We show that this algorithm may be used when the imaging problem can be cast as a linear system that admits a special factorization. We discuss several active array imaging configurations where this factorization is exact, as well as other configurations where the factorization only holds approximately and, hence, the results provided by MUSIC deteriorate. We give special attention to the most general setting where an active array with an arbitrary number of transmitters and receivers uses signals of multiple frequencies to image the targets. This setting provides all the possible diversity of information that can be obtained from the illuminations. We give a theorem that shows that MUSIC is robust with respect to additive noise provided that the targets are well separated. The theorem also shows the relevance of using appropriate sets of controlled parameters, such as excitations, to form the images with MUSIC robustly. We present numerical experiments that support our theoretical results.

math.NA↗

arXiv2017

A Stochastic Lagrangian particle system for the Navier-Stokes equations

This work is based on a formulation of the incompressible Navier-Stokes equations developed by P. Constantin and G.Iyer, where the velocity field of a viscous incompressible fluid is written as the expected value of a stochastic process. If we take $N$ copies of the above process (each based on independent Wiener processes), and replace the expected value with the empirical mean, then it was shown that the particle system for the Navier-Stokes equations does not dissipate all its energy as $t \to \infty$. In contrast to the true (unforced) Navier-Stokes equations, which dissipate all of its energy as $t \to \infty$. The objective of this short note is to describe a resetting procedure that removes this deficiency. We prove that if we repeat this resetting procedure often enough, then the new particle system for the Navier-Stokes equations dissipates all its energy.

math.PR↗