arXiv subjects

Meng Li

Publications and source records attributed to Meng Li.

At least 289 records · Page 16Linked to original sources

arXiv2022

Multipartite High-dimensional Quantum State Engineering via Discrete Time Quantum Walk

Quantum state engineering, namely the generation and control of arbitrary quantum states, is drawing more and more attention due to its wide applications in quantum information and computation. However, there is no general method in theory, and the existing schemes also depend heavily on the selected experimental platform. In this manuscript, we give two schemes for the engineering task of arbitrary quantum state in $c$-partite $d$-dimensional system, both of which are based on discrete-time quantum walk with a $2^c$-dimensional time- and position-dependent coin. The first procedure is a $d$-step quantum walk where all the $d$ coins are non-identity, while the second procedure is an $O(d)$-step quantum walk where only $O(\log d)$ coins are non-identity. A concrete example of preparing generalized Bell states is given to demonstrate the first scheme we proposed. We also show how these schemes can be used to reduce the cost of long-distance quantum communication when the particles involved in the system are far away from each other. Furthermore, the first scheme can be applied to give an alternative approach to the quantum state preparation problem which is one of the fundamental tasks of quantum information processing. We design circuits for quantum state preparation with the help of our quantum state engineering scheme that match the best current result in both size and depth of the circuit asymptotically.

quant-ph↗

arXiv2022

End to End Generative Meta Curriculum Learning For Medical Data Augmentation

Current medical image synthetic augmentation techniques rely on intensive use of generative adversarial networks (GANs). However, the nature of GAN architecture leads to heavy computational resources to produce synthetic images and the augmentation process requires multiple stages to complete. To address these challenges, we introduce a novel generative meta curriculum learning method that trains the task-specific model (student) end-to-end with only one additional teacher model. The teacher learns to generate curriculum to feed into the student model for data augmentation and guides the student to improve performance in a meta-learning style. In contrast to the generator and discriminator in GAN, which compete with each other, the teacher and student collaborate to improve the student's performance on the target tasks. Extensive experiments on the histopathology datasets show that leveraging our framework results in significant and consistent improvements in classification performance.

eess.IV↗

arXiv2022

Bayesian Image-on-Scalar Regression with a Spatial Global-Local Spike-and-Slab Prior

In this article, we propose a novel spatial global-local spike-and-slab selection prior for image-on-scalar regression. We consider a Bayesian hierarchical Gaussian process model for image smoothing, that uses a flexible Inverse-Wishart process prior to handle within-image dependency, and propose a general global-local spatial selection prior that extends a rich class of well-studied selection priors. Unlike existing constructions, we achieve simultaneous global (i.e, at covariate-level) and local (i.e., at pixel/voxel-level) selection by introducing `participation rate' parameters that measure the probability for the individual covariates to affect the observed images. This along with a hard-thresholding strategy leads to dependency between selections at the two levels, introduces extra sparsity at the local level, and allows the global selection to be informed by the local selection, all in a model-based manner. We design an efficient Gibbs sampler that allows inference for large image data. We show on simulated data that parameters are interpretable and lead to efficient selection. Finally, we demonstrate performance of the proposed model by using data from the Autism Brain Imaging Data Exchange (ABIDE) study. To the best of our knowledge, the proposed model construction is the first in the Bayesian literature to simultaneously achieve image smoothing, parameter estimation and a two-level variable selection for image-on-scalar regression.

stat.ME↗

arXiv2022

Multilevel Transformer For Multimodal Emotion Recognition

Multimodal emotion recognition has attracted much attention recently. Fusing multiple modalities effectively with limited labeled data is a challenging task. Considering the success of pre-trained model and fine-grained nature of emotion expression, it is reasonable to take these two aspects into consideration. Unlike previous methods that mainly focus on one aspect, we introduce a novel multi-granularity framework, which combines fine-grained representation with pre-trained utterance-level representation. Inspired by Transformer TTS, we propose a multilevel transformer model to perform fine-grained multimodal emotion recognition. Specifically, we explore different methods to incorporate phoneme-level embedding with word-level embedding. To perform multi-granularity learning, we simply combine multilevel transformer model with Albert. Extensive experimental results show that both our multilevel transformer model and multi-granularity model outperform previous state-of-the-art approaches on IEMOCAP dataset with text transcripts and speech signal.

cs.CL↗

arXiv2022

The solution set of fuzzy relation equations with addition-min composition

This paper deals with the resolutions of fuzzy relation equations with addition-min composition. When the fuzzy relation equations have a solution, we first propose an algorithm to find all minimal solutions of the fuzzy relation equations and also supply an algorithm to find all maximal solutions of the fuzzy relation equations, which will be illustrated, respectively, by numeral examples. Then we prove that every solution of the fuzzy relation equations is between a minimal solution and a maximal one, so that we describe the solution set of the fuzzy relation equations completely.

cs.AI↗

arXiv2022

Variational Matrix Product State Approach for Non-Hermitian System Based on a Companion Hermitian Hamiltonian

Non-Hermitian systems exhibiting topological properties are attracting growing interest. In this work, we propose an algorithm for solving the ground state of a non-Hermitian system in the matrix product state (MPS) formalism based on a companion Hermitian Hamiltonian. If the eigenvalues of the non-Hermitian system are known, the companion Hermitian Hamiltonian can be directly constructed and solved using Hermitian variational methods. When the eigenvalues are unknown, a gradient descent along with the companion Hermitian Hamiltonian yields both the ground state eigenenergy and the eigenstate. With the variational principle as a solid foundation, our algorithm ensures convergence and provides results in excellent agreement with the exact solutions of the non-Hermitian Su-Schrieffer-Heeger (nH-SSH) model as well as its interacting extension. The approach we present avoids solving any non-Hermitian matrix and overcomes numerical instabilities commonly encountered in large non-Hermitian systems.

quant-ph↗

arXiv2022

Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures

Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, and/or overfitting. Here, we overcome these limitations with the largest multi-site sample size to date (n=5,356) to provide a generalizable ML classification benchmark of major depressive disorder (MDD). Using brain measures from standardized ENIGMA analysis pipelines in FreeSurfer, we were able to classify MDD vs healthy controls (HC) with around 62% balanced accuracy, but when harmonizing the data using ComBat balanced accuracy dropped to approximately 52%. Similar results were observed in stratified groups according to age of onset, antidepressant use, number of episodes and sex. Future studies incorporating higher dimensional brain imaging/phenotype features, and/or using more advanced machine and deep learning methods may achieve more encouraging prospects.

q-bio.QM↗

arXiv2022

BiT: Robustly Binarized Multi-distilled Transformer

Modern pre-trained transformers have rapidly advanced the state-of-the-art in machine learning, but have also grown in parameters and computational complexity, making them increasingly difficult to deploy in resource-constrained environments. Binarization of the weights and activations of the network can significantly alleviate these issues, however, is technically challenging from an optimization perspective. In this work, we identify a series of improvements that enables binary transformers at a much higher accuracy than what was possible previously. These include a two-set binarization scheme, a novel elastic binary activation function with learned parameters, and a method to quantize a network to its limit by successively distilling higher precision models into lower precision students. These approaches allow for the first time, fully binarized transformer models that are at a practical level of accuracy, approaching a full-precision BERT baseline on the GLUE language understanding benchmark within as little as 5.9%. Code and models are available at: https://github.com/facebookresearch/bit.

cs.LG↗

arXiv2022

Incorporating VAD into ASR System by Multi-task Learning

When we use End-to-end automatic speech recognition (E2E-ASR) system for real-world applications, a voice activity detection (VAD) system is usually needed to improve the performance and to reduce the computational cost by discarding non-speech parts in the audio. Usually ASR and VAD systems are trained and utilized independently to each other. In this paper, we present a novel multi-task learning (MTL) framework that incorporates VAD into the ASR system. The proposed system learns ASR and VAD jointly in the training stage. With the assistance of VAD, the ASR performance improves as its connectionist temporal classification (CTC) loss function can leverage the VAD alignment information. In the inference stage, the proposed system removes non-speech parts at low computational cost and recognizes speech parts with high robustness. Experimental results on segmented speech data show that by utilizing VAD information, the proposed method outperforms the baseline ASR system on both English and Chinese datasets. On unsegmented speech data, we find that the system outperforms the ASR systems that build an extra GMM-based or DNN-based voice activity detector.

eess.AS↗

arXiv2022

Existence of normalized solutions for fractional coupled Hartree-Fock type system

In this paper, we consider the existence of solutions for the following fractional coupled Hartree-Fock type system \begin{align*} \left\{\begin{aligned} &(-Δ)^s u+V_1(x)u+λ_1u=μ_1(I_α\star |u|^p)|u|^{p-2}u+β(I_α\star |v|^r)|u|^{r-2}u\\ &(-Δ)^s v+V_2(x)v+λ_2v=μ_2(I_α\star |v|^q)|v|^{q-2}v+β(I_α\star |u|^r)|v|^{r-2}v \end{aligned} \right.~\quad x\in\mathbb{R}^N, \end{align*} under the constraint \begin{align*} \int_{\mathbb{R}^N}|u|^2=a^2,~\int_{\mathbb{R}^N}|v|^2=b^2. \end{align*} where $s\in(0,1),~N\ge3,~μ_1>0,~μ_2>0,~β>0,~α\in(0,N),~1+\fracα{N}<p,~q,~r<\frac{N+α}{N-2s}$ and $I_α(x)=|x|^{α-N}$. Under some restrictions of $N,α,p,q$ and $r$, we give the positivity of normalized solutions for $p,q,r\le 1+\frac{α+2s}{N}$.

math.AP↗

arXiv2022

Double spike Dirichlet priors for structured weighting

Assigning weights to a large pool of objects is a fundamental task in a wide variety of applications. In this article, we introduce the concept of structured high-dimensional probability simplexes, in which most components are zero or near zero and the remaining ones are close to each other. Such structure is well motivated by (i) high-dimensional weights that are common in modern applications, and (ii) ubiquitous examples in which equal weights -- despite their simplicity -- often achieve favorable or even state-of-the-art predictive performance. This particular structure, however, presents unique challenges partly because, unlike high-dimensional linear regression, the parameter space is a simplex and pattern switching between partial constancy and sparsity is unknown. To address these challenges, we propose a new class of double spike Dirichlet priors to shrink a probability simplex to one with the desired structure. When applied to ensemble learning, such priors lead to a Bayesian method for structured high-dimensional ensembles that is useful for forecast combination and improving random forests, while enabling uncertainty quantification. We design efficient Markov chain Monte Carlo algorithms for implementation. Posterior contraction rates are established to study large sample behaviors of the posterior distribution. We demonstrate the wide applicability and competitive performance of the proposed methods through simulations and two real data applications using the European Central Bank Survey of Professional Forecasters data set and a data set from the UC Irvine Machine Learning Repository (UCI).

stat.ME↗

arXiv2022

Magnetic-field modulation of topological electronic states and emergent magneto-transport in a magnetic Weyl semimetal

The modulation of topological electronic states by an external magnetic field is highly desired for condensed matter physics. Schemes to achieve this have been proposed theoretically, but few can be realized experimentally. Here, combining transverse transport, theoretical calculations, and scanning tunneling microscopy/spectroscopy (STM/S) investigations, we provide an observation that the topological electronic states, accompanied by an emergent magneto-transport phenomenon, were modulated by applying magnetic fields through induced non-collinear magnetism in the magnetic Weyl semimetal EuB6. A giant unconventional anomalous Hall effect (UAHE) is found during the magnetisation re-orientation from easy axis to hard ones in magnetic fields, with a UAHE peak around the low field of 5 kOe. Under the reasonable spin-canting effect, the folding of the topological anti-crossing bands occurs, generating a strong Berry curvature that accounts for the observed UAHE. Field-dependent STM/S reveals a highly synchronous evolution of electronic density of states, with a dI/dV peak around the same field of 5 kOe, which provides evidence to the folded bands and excited UAHE by external magnetic fields. This finding elucidates the connection between the real-space non-collinear magnetism and the k-space topological electronic states and establishes a novel manner to engineer the magneto-transport behaviours of correlated electrons for future topological spintronics.

cond-mat.mtrl-sci↗

arXiv2022

Inference in Functional Linear Quantile Regression

In this paper, we study statistical inference in functional quantile regression for scalar response and a functional covariate. Specifically, we consider a functional linear quantile regression model where the effect of the covariate on the quantile of the response is modeled through the inner product between the functional covariate and an unknown smooth regression parameter function that varies with the level of quantile. The objective is to test that the regression parameter is constant across several quantile levels of interest. The parameter function is estimated by combining ideas from functional principal component analysis and quantile regression. An adjusted Wald testing procedure is proposed for this hypothesis of interest, and its chi-square asymptotic null distribution is derived. The testing procedure is investigated numerically in simulations involving sparse and noisy functional covariates and in a capital bike share data application. The proposed approach is easy to implement and the {\tt R} code is published online at \url{https://github.com/xylimeng/fQR-testing}.

stat.ME↗

arXiv2022

Content-oriented learned image compression

In recent years, with the development of deep neural networks, end-to-end optimized image compression has made significant progress and exceeded the classic methods in terms of rate-distortion performance. However, most learning-based image compression methods are unlabeled and do not consider image semantics or content when optimizing the model. In fact, human eyes have different sensitivities to different content, so the image content also needs to be considered. In this paper, we propose a content-oriented image compression method, which handles different kinds of image contents with different strategies. Extensive experiments show that the proposed method achieves competitive subjective results compared with state-of-the-art end-to-end learned image compression methods or classic methods.

cs.CV↗

arXiv2022

Few-Shot Class-Incremental Learning from an Open-Set Perspective

The continual appearance of new objects in the visual world poses considerable challenges for current deep learning methods in real-world deployments. The challenge of new task learning is often exacerbated by the scarcity of data for the new categories due to rarity or cost. Here we explore the important task of Few-Shot Class-Incremental Learning (FSCIL) and its extreme data scarcity condition of one-shot. An ideal FSCIL model needs to perform well on all classes, regardless of their presentation order or paucity of data. It also needs to be robust to open-set real-world conditions and be easily adapted to the new tasks that always arise in the field. In this paper, we first reevaluate the current task setting and propose a more comprehensive and practical setting for the FSCIL task. Then, inspired by the similarity of the goals for FSCIL and modern face recognition systems, we propose our method -- Augmented Angular Loss Incremental Classification or ALICE. In ALICE, instead of the commonly used cross-entropy loss, we propose to use the angular penalty loss to obtain well-clustered features. As the obtained features not only need to be compactly clustered but also diverse enough to maintain generalization for future incremental classes, we further discuss how class augmentation, data augmentation, and data balancing affect classification performance. Experiments on benchmark datasets, including CIFAR100, miniImageNet, and CUB200, demonstrate the improved performance of ALICE over the state-of-the-art FSCIL methods.

cs.CV↗

arXiv2022

Federated Learning with Non-IID Data

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55% for neural networks trained for highly skewed non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30% for the CIFAR-10 dataset with only 5% globally shared data.

cs.LG↗

arXiv2022

Omni-sparsity DNN: Fast Sparsity Optimization for On-Device Streaming E2E ASR via Supernet

From wearables to powerful smart devices, modern automatic speech recognition (ASR) models run on a variety of edge devices with different computational budgets. To navigate the Pareto front of model accuracy vs model size, researchers are trapped in a dilemma of optimizing model accuracy by training and fine-tuning models for each individual edge device while keeping the training GPU-hours tractable. In this paper, we propose Omni-sparsity DNN, where a single neural network can be pruned to generate optimized model for a large range of model sizes. We develop training strategies for Omni-sparsity DNN that allows it to find models along the Pareto front of word-error-rate (WER) vs model size while keeping the training GPU-hours to no more than that of training one singular model. We demonstrate the Omni-sparsity DNN with streaming E2E ASR models. Our results show great saving on training time and resources with similar or better accuracy on LibriSpeech compared to individually pruned sparse models: 2%-6.6% better WER on Test-other.

cs.SD↗

arXiv2022

Horizontal Layer Constrained Attention Neural Network for Semblance Velocity Picking

Semblance velocity analysis is a crucial step in seismic data processing. To avoid the huge time-cost when performed manually, some deep learning methods are proposed for automatic semblance velocity picking. However, the application of existing deep learning methods is still restricted by the shortage of labels in practice. In this letter, we propose an attention neural network combined with a point-to-point regression velocity picking strategy to mitigate this problem. In our method, semblance patch and velocity value are served as network input and output, respectively. In this way, global and local features hidden in semblance patch can be effectively extracted by attention neural network. A down-sampling strategy based on horizontal layer extraction is also designed to improve the picking efficiency in prediction process. Tests on synthetic and field datasets demonstrate that the proposed method can produce reasonable results and maintain global velocity trend consistent with labels. Besides, robustness against random noise is also tested on the field data.

physics.geo-ph↗