arXiv subjects

Haizhao Yang

Publications and source records attributed to Haizhao Yang.

At least 127 records · Page 7Linked to original sources

arXiv2020

Multidimensional Phase Recovery and Interpolative Decomposition Butterfly Factorization

This paper focuses on the fast evaluation of the matvec $g=Kf$ for $K\in \mathbb{C}^{N\times N}$, which is the discretization of a multidimensional oscillatory integral transform $g(x) = \int K(x,ξ) f(ξ)dξ$ with a kernel function $K(x,ξ)=e^{2\piıΦ(x,ξ)}$, where $Φ(x,ξ)$ is a piecewise smooth phase function with $x$ and $ξ$ in $\mathbb{R}^d$ for $d=2$ or $3$. A new framework is introduced to compute $Kf$ with $O(N\log N)$ time and memory complexity in the case that only indirect access to the phase function $Φ$ is available. This framework consists of two main steps: 1) an $O(N\log N)$ algorithm for recovering the multidimensional phase function $Φ$ from indirect access is proposed; 2) a multidimensional interpolative decomposition butterfly factorization (MIDBF) is designed to evaluate the matvec $Kf$ with an $O(N\log N)$ complexity once $Φ$ is available. Numerical results are provided to demonstrate the effectiveness of the proposed framework.

math.NA↗

arXiv2019

A Hierarchical Butterfly LU Preconditioner for Two-Dimensional Electromagnetic Scattering Problems Involving Open Surfaces

This paper introduces a hierarchical interpolative decomposition butterfly-LU factorization (H-IDBF-LU) preconditioner for solving two-dimensional electric-field integral equations (EFIEs) in electromagnetic scattering problems of perfect electrically conducting objects with open surfaces. H-IDBF-LU leverages the interpolative decomposition butterfly factorization (IDBF) to compress dense blocks of the discretized EFIE operator to expedite its application; this compressed operator also serves as an approximate LU factorization of the EFIE operator leading to an efficient preconditioner in iterative solvers. Both the memory requirement and computational cost of the H-IDBF-LU solver scale as $O(N\log^2 N)$ in one iteration; the total number of iterations required for a reasonably good accuracy scales as $O(1)$ to $O(\log^2N)$ in all of our numerical tests. The efficacy and accuracy of the proposed preconditioned iterative solver are demonstrated via its application to a broad range of scatterers involving up to $100$ million unknowns.

math.NA↗

arXiv2019

DIANet: Dense-and-Implicit Attention Network

Attention networks have successfully boosted the performance in various vision problems. Previous works lay emphasis on designing a new attention module and individually plug them into the networks. Our paper proposes a novel-and-simple framework that shares an attention module throughout different network layers to encourage the integration of layer-wise information and this parameter-sharing module is referred as Dense-and-Implicit-Attention (DIA) unit. Many choices of modules can be used in the DIA unit. Since Long Short Term Memory (LSTM) has a capacity of capturing long-distance dependency, we focus on the case when the DIA unit is the modified LSTM (refer as DIA-LSTM). Experiments on benchmark datasets show that the DIA-LSTM unit is capable of emphasizing layer-wise feature interrelation and leads to significant improvement of image classification accuracy. We further empirically show that the DIA-LSTM has a strong regularization ability on stabilizing the training of deep networks by the experiments with the removal of skip connections or Batch Normalization in the whole residual network. The code is released at https://github.com/gbup-group/DIANet.

cs.CV↗

arXiv2019

Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise

Batch Normalization (BN)(Ioffe and Szegedy 2015) normalizes the features of an input image via statistics of a batch of images and hence BN will bring the noise to the gradient of the training loss. Previous works indicate that the noise is important for the optimization and generalization of deep neural networks, but too much noise will harm the performance of networks. In our paper, we offer a new point of view that self-attention mechanism can help to regulate the noise by enhancing instance-specific information to obtain a better regularization effect. Therefore, we propose an attention-based BN called Instance Enhancement Batch Normalization (IEBN) that recalibrates the information of each channel by a simple linear transformation. IEBN has a good capacity of regulating noise and stabilizing network training to improve generalization even in the presence of two kinds of noise attacks during training. Finally, IEBN outperforms BN with only a light parameter increment in image classification tasks for different network structures and benchmark datasets.

cs.LG↗

arXiv2019

Fast Algorithms for the Multi-dimensional Jacobi Polynomial Transform

We use the well-known observation that the solutions of Jacobi's differential equation can be represented via non-oscillatory phase and amplitude functions to develop a fast algorithm for computing multi-dimensional Jacobi polynomial transforms. More explicitly, it follows from this observation that the matrix corresponding to the discrete Jacobi transform is the Hadamard product of a numerically low-rank matrix and a multi-dimensional discrete Fourier transform (DFT) matrix. The application of the Hadamard product can be carried out via $O(1)$ fast Fourier transforms (FFTs), resulting in a nearly optimal algorithm to compute the multidimensional Jacobi polynomial transform.

math.NA↗

arXiv2019

Multiresolution Mode Decomposition for Adaptive Time Series Analysis

This paper proposes the \emph{multiresolution mode decomposition} as a novel model for adaptive time series analysis. The main conceptual innovation is the introduction of the \emph{multiresolution intrinsic mode function} (MIMF) of the form \[ \sum_{n=-N/2}^{N/2-1} a_n\cos(2πnϕ(t))s_{cn}(2πNϕ(t))+\sum_{n=-N/2}^{N/2-1}b_n \sin(2πnϕ(t))s_{sn}(2πNϕ(t))\] to model nonlinear and non-stationary data with time-dependent amplitudes, frequencies, and waveforms. %The MIMF explains the intrinsic difficulty in concentrating time-frequency representation of nonlinear and non-stationary data and provides a new direction for mode decomposition. The multiresolution expansion coefficients $\{a_n\}$, $\{b_n\}$, and the shape function series $\{s_{cn}(t)\}$ and $\{s_{sn}(t)\}$ provide innovative features for adaptive time series analysis. For complex signals that are a superposition of several MIMFs with well-differentiated phase functions $ϕ(t)$, a new recursive scheme based on Gauss-Seidel iteration and diffeomorphisms is proposed to identify these MIMFs, their multiresolution expansion coefficients, and shape function series. Numerical examples from synthetic data and natural phenomena are given to demonstrate the power of this new method.

math.NA↗

arXiv2019

SelectNet: Learning to Sample from the Wild for Imbalanced Data Training

Supervised learning from training data with imbalanced class sizes, a commonly encountered scenario in real applications such as anomaly/fraud detection, has long been considered a significant challenge in machine learning. Motivated by recent progress in curriculum and self-paced learning, we propose to adopt a semi-supervised learning paradigm by training a deep neural network, referred to as SelectNet, to selectively add unlabelled data together with their predicted labels to the training dataset. Unlike existing techniques designed to tackle the difficulty in dealing with class imbalanced training data such as resampling, cost-sensitive learning, and margin-based learning, SelectNet provides an end-to-end approach for learning from important unlabelled data "in the wild" that most likely belong to the under-sampled classes in the training data, thus gradually mitigates the imbalance in the data used for training the classifier. We demonstrate the efficacy of SelectNet through extensive numerical experiments on standard datasets in computer vision.

cs.LG↗

arXiv2019

CASS: Cross Adversarial Source Separation via Autoencoder

This paper introduces a cross adversarial source separation (CASS) framework via autoencoder, a new model that aims at separating an input signal consisting of a mixture of multiple components into individual components defined via adversarial learning and autoencoder fitting. CASS unifies popular generative networks like auto-encoders (AEs) and generative adversarial networks (GANs) in a single framework. The basic building block that filters the input signal and reconstructs the $i$-th target component is a pair of deep neural networks $\mathcal{EN}_i$ and $\mathcal{DE}_i$ as an encoder for dimension reduction and a decoder for component reconstruction, respectively. The decoder $\mathcal{DE}_i$ as a generator is enhanced by a discriminator network $\mathcal{D}_i$ that favors signal structures of the $i$-th component in the $i$-th given dataset as guidance through adversarial learning. In contrast with existing practices in AEs which trains each Auto-Encoder independently, or in GANs that share the same generator, we introduce cross adversarial training that emphasizes adversarial relation between any arbitrary network pairs $(\mathcal{DE}_i,\mathcal{D}_j)$, achieving state-of-the-art performance especially when target components share similar data structures.

cs.LG↗

arXiv2019

Generative Imaging and Image Processing via Generative Encoder

This paper introduces a novel generative encoder (GE) model for generative imaging and image processing with applications in compressed sensing and imaging, image compression, denoising, inpainting, deblurring, and super-resolution. The GE model consists of a pre-training phase and a solving phase. In the pre-training phase, we separately train two deep neural networks: a generative adversarial network (GAN) with a generator $\G$ that captures the data distribution of a given image set, and an auto-encoder (AE) network with an encoder $\EN$ that compresses images following the estimated distribution by GAN. In the solving phase, given a noisy image $x=\mathcal{P}(x^*)$, where $x^*$ is the target unknown image, $\mathcal{P}$ is an operator adding an addictive, or multiplicative, or convolutional noise, or equivalently given such an image $x$ in the compressed domain, i.e., given $m=\EN(x)$, we solve the optimization problem \[ z^*=\underset{z}{\mathrm{argmin}} \|\EN(\G(z))-m\|_2^2+λ\|z\|_2^2 \] to recover the image $x^*$ in a generative way via $\hat{x}:=\G(z^*)\approx x^*$, where $λ>0$ is a hyperparameter. The GE model unifies the generative capacity of GANs and the stability of AEs in an optimization framework above instead of stacking GANs and AEs into a single network or combining their loss functions into one as in existing literature. Numerical experiments show that the proposed model outperforms several state-of-the-art algorithms.

eess.IV↗

arXiv2018

$3D$ Crystal Image Analysis based on Fast Synchrosqueezed Transforms

We propose an efficient algorithm to analyze $3D$ atomic resolution crystal images based on a fast $3D$ synchrosqueezed wave packet transform. The proposed algorithm can automatically extract microscopic information from $3D$ atomic resolution crystal images, e.g., crystal orientation, defects, and deformation, which are important information for characterizing material properties. The effectiveness of our algorithms is illustrated by experiments of synthetic datasets and real $3$D microscopic colloidal images.

math.NA↗

arXiv2018

A Fast Algorithm for Multiresolution Mode Decomposition

\emph{Multiresolution mode decomposition} (MMD) is an adaptive tool to analyze a time series $f(t)=\sum_{k=1}^K f_k(t)$, where $f_k(t)$ is a \emph{multiresolution intrinsic mode function} (MIMF) of the form \begin{eqnarray*} f_k(t)&=&\sum_{n=-N/2}^{N/2-1} a_{n,k}\cos(2πnϕ_k(t))s_{cn,k}(2πN_kϕ_k(t))\\&&+\sum_{n=-N/2}^{N/2-1}b_{n,k} \sin(2πnϕ_k(t))s_{sn,k}(2πN_kϕ_k(t)) \end{eqnarray*} with time-dependent amplitudes, frequencies, and waveforms. The multiresolution expansion coefficients $\{a_{n,k}\}$, $\{b_{n,k}\}$, and the shape function series $\{s_{cn,k}(t)\}$ and $\{s_{sn,k}(t)\}$ provide innovative features for adaptive time series analysis. The MMD aims at identifying these MIMF's (including their multiresolution expansion coefficients and shape functions series) from their superposition. This paper proposes a fast algorithm for solving the MMD problem based on recursive diffeomorphism-based spectral analysis (RDSA). RDSA admits highly efficient numerical implementation via the nonuniform fast Fourier transform (NUFFT); its convergence and accuracy can be guaranteed theoretically. Numerical examples from synthetic data and natural phenomena are given to demonstrate the efficiency of the proposed method.

math.NA↗

arXiv2018

A Unified Framework for Oscillatory Integral Transform: When to use NUFFT or Butterfly Factorization?

This paper concerns the fast evaluation of the matvec $g=Kf$ for $K\in \mathbb{C}^{N\times N}$, which is the discretization of the oscillatory integral transform $g(x) = \int K(x,ξ) f(ξ)dξ$ with a kernel function $K(x,ξ)=α(x,ξ)e^{2\piıΦ(x,ξ)}$, where $α(x,ξ)$ is a smooth amplitude function, and $Φ(x,ξ)$ is a piecewise smooth phase function with $O(1)$ discontinuous points in $x$ and $ξ$. A unified framework is proposed to compute $Kf$ with $O(N\log N)$ time and memory complexity via the non-uniform fast Fourier transform (NUFFT) or the butterfly factorization (BF), together with an $O(N)$ fast algorithm to determine whether NUFFT or BF is more suitable. This framework works for two cases: 1) explicit formulas for the amplitude and phase functions are known, 2) only indirect access of the amplitude and phase functions are available. Especially in the case of indirect access, our main contributions are: 1) an $O(N\log N)$ algorithm for recovering the amplitude and phase functions is proposed based on a new low-rank matrix recovery algorithm, 2) a new stable and nearly optimal BF with amplitude and phase functions in a form of a low-rank factorization (IBF-MAT) is proposed to evaluate the matvec $Kf$. Numerical results are provided to demonstrate the effectiveness of the proposed framework.

math.NA↗

arXiv2018

Interpolative Decomposition Butterfly Factorization

This paper introduces a "kernel-independent" interpolative decomposition butterfly factorization (IDBF) as a data-sparse approximation for matrices that satisfy a complementary low-rank property. The IDBF can be constructed in $O(N\log N)$ operations for an $N\times N$ matrix via hierarchical interpolative decompositions (IDs), if matrix entries can be sampled individually and each sample takes $O(1)$ operations. The resulting factorization is a product of $O(\log N)$ sparse matrices, each with $O(N)$ non-zero entries. Hence, it can be applied to a vector rapidly in $O(N\log N)$ operations. IDBF is a general framework for nearly optimal fast matvec useful in a wide range of applications, e.g., special function transformation, Fourier integral operators, high-frequency wave computation. Numerical results are provided to demonstrate the effectiveness of the butterfly factorization and its construction algorithms.

math.NA↗

arXiv2018

Phase Space Sketching for Crystal Image Analysis based on Synchrosqueezed Transforms

Recent developments of imaging techniques enable researchers to visualize materials at the atomic resolution to better understand the microscopic structures of materials. This paper aims at automatic and quantitative characterization of potentially complicated microscopic crystal images, providing feedback to tweak theories and improve synthesis in materials science. As such, an efficient phase-space sketching method is proposed to encode microscopic crystal images in a translation, rotation, illumination, and scale invariant representation, which is also stable with respect to small deformations. Based on the phase-space sketching, we generalize our previous analysis framework for crystal images with simple structures to those with complicated geometry.

cond-mat.mtrl-sci↗

arXiv2018

Fast algorithms for Jacobi expansions via nonoscillatory phase functions

We describe a suite of fast algorithms for evaluating Jacobi polynomials, applying the corresponding discrete Sturm-Liouville eigentransforms and calculating Gauss-Jacobi quadrature rules. Our approach is based on the well-known fact that Jacobi's differential equation admits a nonoscillatory phase function which can be loosely approximated via an affine function over much of its domain. Our algorithms perform better than currently available methods in most respects. We illustrate this with several numerical experiments, the source code for which is publicly available.

math.NA↗

arXiv2017

Diffusion forecasting model with basis functions from QR decomposition

The diffusion forecasting is a nonparametric approach that provably solves the Fokker-Planck PDE corresponding to Itô diffusion without knowing the underlying equation. The key idea of this method is to approximate the solution of the Fokker-Planck equation with a discrete representation of the shift (Koopman) operator on a set of basis functions generated via the diffusion maps algorithm. While the choice of these basis functions is provably optimal under appropriate conditions, computing these basis functions is quite expensive since it requires the eigendecomposition of an $N\times N$ diffusion matrix, where $N$ denotes the data size and could be very large. For large-scale forecasting problems, only a few leading eigenvectors are computationally achievable. To overcome this computational bottleneck, a new set of basis functions constructed by orthonormalizing selected columns of the diffusion matrix and its leading eigenvectors is proposed. This computation can be carried out efficiently via the unpivoted Householder QR factorization. The efficiency and effectiveness of the proposed algorithm will be shown in both deterministically chaotic and stochastic dynamical systems; in the former case, the superiority of the proposed basis functions over purely eigenvectors is significant, while in the latter case forecasting accuracy is improved relative to using a purely small number of eigenvectors. Supporting arguments will be provided on three- and six-dimensional chaotic ODEs, a three-dimensional SDE that mimics turbulent systems, and also on the two spatial modes associated with the boreal winter Madden-Julian Oscillation obtained from applying the Nonlinear Laplacian Spectral Analysis on the measured Outgoing Longwave Radiation (OLR).

math.NA↗

arXiv2017

Recursive Diffeomorphism-Based Regression for Shape Functions

This paper proposes a recursive diffeomorphism based regression method for one-dimensional generalized mode decomposition problem that aims at extracting generalized modes $α_k(t)s_k(2πN_kϕ_k(t))$ from their superposition $\sum_{k=1}^K α_k(t)s_k(2πN_kϕ_k(t))$. First, a one-dimensional synchrosqueezed transform is applied to estimate instantaneous information, e.g., $α_k(t)$ and $N_kϕ_k(t)$. Second, a novel approach based on diffeomorphisms and nonparametric regression is proposed to estimate wave shape functions $s_k(t)$. These two methods lead to a framework for the generalized mode decomposition problem under a weak well-separation condition. Numerical examples of synthetic and real data are provided to demonstrate the fruitful applications of these methods.

math.NA↗

arXiv2017

ELSI: A Unified Software Interface for Kohn-Sham Electronic Structure Solvers

Solving the electronic structure from a generalized or standard eigenproblem is often the bottleneck in large scale calculations based on Kohn-Sham density-functional theory. This problem must be addressed by essentially all current electronic structure codes, based on similar matrix expressions, and by high-performance computation. We here present a unified software interface, ELSI, to access different strategies that address the Kohn-Sham eigenvalue problem. Currently supported algorithms include the dense generalized eigensolver library ELPA, the orbital minimization method implemented in libOMM, and the pole expansion and selected inversion (PEXSI) approach with lower computational complexity for semilocal density functionals. The ELSI interface aims to simplify the implementation and optimal use of the different strategies, by offering (a) a unified software framework designed for the electronic structure solvers in Kohn-Sham density-functional theory; (b) reasonable default parameters for a chosen solver; (c) automatic conversion between input and internal working matrix formats, and in the future (d) recommendation of the optimal solver depending on the specific problem. Comparative benchmarks are shown for system sizes up to 11,520 atoms (172,800 basis functions) on distributed memory supercomputing architectures.

physics.comp-ph↗