arXiv subjects

Yu Zhang

Publications and source records attributed to Yu Zhang.

At least 1,405 records · Page 78Linked to original sources

arXiv2020

CODAR: A Contextual Duration-Aware Qubit Mapping for Various NISQ Devices

Quantum computing devices in the NISQ era share common features and challenges like limited connectivity between qubits. Since two-qubit gates are allowed on limited qubit pairs, quantum compilers must transform original quantum programs to fit the hardware constraints. Previous works on qubit mapping assume different gates have the same execution duration, which limits them to explore the parallelism from the program. To address this drawback, we propose a Multi-architecture Adaptive Quantum Abstract Machine (maQAM) and a COntext-sensitive and Duration-Aware Remapping algorithm (CODAR). The CODAR remapper is aware of gate duration difference and program context, enabling it to extract more parallelism from programs and speed up the quantum programs by 1.23 in simulation on average in different architectures and maintain the fidelity of circuits when running on Origin Quantum noisy simulator.

quant-ph↗

arXiv2020

High-Order Paired-ASPP Networks for Semantic Segmenation

Current semantic segmentation models only exploit first-order statistics, while rarely exploring high-order statistics. However, common first-order statistics are insufficient to support a solid unanimous representation. In this paper, we propose High-Order Paired-ASPP Network to exploit high-order statistics from various feature levels. The network first introduces a High-Order Representation module to extract the contextual high-order information from all stages of the backbone. They can provide more semantic clues and discriminative information than the first-order ones. Besides, a Paired-ASPP module is proposed to embed high-order statistics of the early stages into the last stage. It can further preserve the boundary-related and spatial context in the low-level features for final prediction. Our experiments show that the high-order statistics significantly boost the performance on confusing objects. Our method achieves competitive performance without bells and whistles on three benchmarks, i.e, Cityscapes, ADE20K and Pascal-Context with the mIoU of 81.6%, 45.3% and 52.9%.

cs.CV↗

arXiv2020

ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit

This paper introduces a new end-to-end text-to-speech (E2E-TTS) toolkit named ESPnet-TTS, which is an extension of the open-source speech processing toolkit ESPnet. The toolkit supports state-of-the-art E2E-TTS models, including Tacotron~2, Transformer TTS, and FastSpeech, and also provides recipes inspired by the Kaldi automatic speech recognition (ASR) toolkit. The recipes are based on the design unified with the ESPnet ASR recipe, providing high reproducibility. The toolkit also provides pre-trained models and samples of all of the recipes so that users can use it as a baseline. Furthermore, the unified design enables the integration of ASR functions with TTS, e.g., ASR-based objective evaluation and semi-supervised learning with both ASR and TTS models. This paper describes the design of the toolkit and experimental evaluation in comparison with other toolkits. The experimental results show that our models can achieve state-of-the-art performance comparable to the other latest toolkits, resulting in a mean opinion score (MOS) of 4.25 on the LJSpeech dataset. The toolkit is publicly available at https://github.com/espnet/espnet.

cs.CL↗

arXiv2020

MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing

Personal mobile sensing is fast permeating our daily lives to enable activity monitoring, healthcare and rehabilitation. Combined with deep learning, these applications have achieved significant success in recent years. Different from conventional cloud-based paradigms, running deep learning on devices offers several advantages including data privacy preservation and low-latency response for both model inference and update. Since data collection is costly in reality, Google's Federated Learning offers not only complete data privacy but also better model robustness based on multiple user data. However, personal mobile sensing applications are mostly user-specific and highly affected by environment. As a result, continuous local changes may seriously affect the performance of a global model generated by Federated Learning. In addition, deploying Federated Learning on a local server, e.g., edge server, may quickly reach the bottleneck due to resource constraint and serious failure by attacks. Towards pushing deep learning on devices, we present MDLdroid, a novel decentralized mobile deep learning framework to enable resource-aware on-device collaborative learning for personal mobile sensing applications. To address resource limitation, we propose a ChainSGD-reduce approach which includes a novel chain-directed Synchronous Stochastic Gradient Descent algorithm to effectively reduce overhead among multiple devices. We also design an agent-based multi-goal reinforcement learning mechanism to balance resources in a fair and efficient manner. Our evaluations show that our model training on off-the-shelf mobile devices achieves 2x to 3.5x faster than single-device training, and 1.5x faster than the master-slave approach.

cs.LG↗

arXiv2020

A Simple General Approach to Balance Task Difficulty in Multi-Task Learning

In multi-task learning, difficulty levels of different tasks are varying. There are many works to handle this situation and we classify them into five categories, including the direct sum approach, the weighted sum approach, the maximum approach, the curriculum learning approach, and the multi-objective optimization approach. Those approaches have their own limitations, for example, using manually designed rules to update task weights, non-smooth objective function, and failing to incorporate other functions than training losses. In this paper, to alleviate those limitations, we propose a Balanced Multi-Task Learning (BMTL) framework. Different from existing studies which rely on task weighting, the BMTL framework proposes to transform the training loss of each task to balance difficulty levels among tasks based on an intuitive idea that tasks with larger training losses will receive more attention during the optimization procedure. We analyze the transformation function and derive necessary conditions. The proposed BMTL framework is very simple and it can be combined with most multi-task learning models. Empirical studies show the state-of-the-art performance of the proposed BMTL framework.

cs.LG↗

arXiv2020

Deep Multi-Task Learning via Generalized Tensor Trace Norm

The trace norm is widely used in multi-task learning as it can discover low-rank structures among tasks in terms of model parameters. Nowadays, with the emerging of big datasets and the popularity of deep learning techniques, tensor trace norms have been used for deep multi-task models. However, existing tensor trace norms cannot discover all the low-rank structures and they require users to manually determine the importance of their components. To solve those two issues together, in this paper, we propose a Generalized Tensor Trace Norm (GTTN). The GTTN is defined as a convex combination of matrix trace norms of all possible tensor flattenings and hence it can discover all the possible low-rank structures. In the induced objective function, we will learn combination coefficients in the GTTN to automatically determine the importance. Experiments on real-world datasets demonstrate the effectiveness of the proposed GTTN.

cs.LG↗

arXiv2020

Deep Multi-Task Augmented Feature Learning via Hierarchical Graph Neural Network

Deep multi-task learning attracts much attention in recent years as it achieves good performance in many applications. Feature learning is important to deep multi-task learning for sharing common information among tasks. In this paper, we propose a Hierarchical Graph Neural Network (HGNN) to learn augmented features for deep multi-task learning. The HGNN consists of two-level graph neural networks. In the low level, an intra-task graph neural network is responsible of learning a powerful representation for each data point in a task by aggregating its neighbors. Based on the learned representation, a task embedding can be generated for each task in a similar way to max pooling. In the second level, an inter-task graph neural network updates task embeddings of all the tasks based on the attention mechanism to model task relations. Then the task embedding of one task is used to augment the feature representation of data points in this task. Moreover, for classification tasks, an inter-class graph neural network is introduced to conduct similar operations on a finer granularity, i.e., the class level, to generate class embeddings for each class in all the tasks use class embeddings to augment the feature representation. The proposed feature augmentation strategy can be used in many deep multi-task learning models. we analyze the HGNN in terms of training and generalization losses. Experiments on real-world datastes show the significant performance improvement when using this strategy.

cs.LG↗

arXiv2020

Effects of resonance weak decays and hadronic re-scattering on the proton number fluctuations in Au + Au collisions at $\sqrt{s_\mathrm{NN}} = 5$ GeV from JAM model

Proton number fluctuation is sensitive observable to search for the QCD critical point in heavy-ion collisions. In this paper, we studied rapidity acceptance dependence of the proton cumulants and correlation functions in most central Au+Au collisions at $\sqrt{s_\mathrm{NN}} = 5$ GeV from a microscopic hadronic transport model (JAM). At mid-rapidity, we found the effects of resonance weak decays and hadronic re-scattering on the proton cumulants and correlation functions are small, but those effects get larger when further increasing the rapidity acceptance. On the other hand, we found the baryon number conservation is a dominant background effect on the rapidity acceptance dependence of proton number fluctuations. It leads to a strong suppression of cumulants and cumulant ratios, as well as the negative proton correlation functions. We also studied those two effects on the energy dependence of cumulant ratios of net-proton distributions in most central Au+Au collisions at $\sqrt{s_\mathrm{NN}} = 5-200$ GeV from JAM model. This work can serve as a non-critical baseline for future QCD critical point search in heavy-ion collisions at high baryon density region.

nucl-ex↗

arXiv2020

Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis

This paper proposes a hierarchical, fine-grained and interpretable latent variable model for prosody based on the Tacotron 2 text-to-speech model. It achieves multi-resolution modeling of prosody by conditioning finer level representations on coarser level ones. Additionally, it imposes hierarchical conditioning across all latent dimensions using a conditional variational auto-encoder (VAE) with an auto-regressive structure. Evaluation of reconstruction performance illustrates that the new structure does not degrade the model while allowing better interpretability. Interpretations of prosody attributes are provided together with the comparison between word-level and phone-level prosody representations. Moreover, both qualitative and quantitative evaluations are used to demonstrate the improvement in the disentanglement of the latent dimensions.

eess.AS↗

arXiv2020

Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior

Recent neural text-to-speech (TTS) models with fine-grained latent features enable precise control of the prosody of synthesized speech. Such models typically incorporate a fine-grained variational autoencoder (VAE) structure, extracting latent features at each input token (e.g., phonemes). However, generating samples with the standard VAE prior often results in unnatural and discontinuous speech, with dramatic prosodic variation between tokens. This paper proposes a sequential prior in a discrete latent space which can generate more naturally sounding samples. This is accomplished by discretizing the latent features using vector quantization (VQ), and separately training an autoregressive (AR) prior model over the result. We evaluate the approach using listening tests, objective metrics of automatic speech recognition (ASR) performance, and measurements of prosody attributes. Experimental results show that the proposed model significantly improves the naturalness in random sample generation. Furthermore, initial experiments demonstrate that randomly sampling from the proposed model can be used as data augmentation to improve the ASR performance.

eess.AS↗

arXiv2020

Nanoscale Probing of Broken-Symmetry States in Graphene Induced by Individual Atomic Impurities

Inherent symmetries of a system lead to multiple degeneracies of its energy spectra. Introducing individual atomic impurities can locally break these symmetries, which is expected to lift the degenerate degrees of freedom around the impurities. Although central to our understanding of the fundamental properties of solids, the broken-symmetry states induced by individual atomic impurities have so far eluded observation. Here, we report nanoscale probing of the broken-symmetry states in graphene induced by two types of individual atomic impurities, i.e., isolated nitrogen dopants and isolated hydrogen atoms chemisorbed on graphene. Our experiments demonstrate that both types of atomic impurities can locally break sublattice symmetry of graphene and generate valley-polarized states, which extends several nanometers around the impurities. For the isolated hydrogen atom chemisorbed on graphene, the enhanced spin-orbit coupling, which arises from the sp3 distortion of graphene due to the hydrogen chemisorption, further lifts the spin degeneracy, resulting in a fully spin and valley polarized states within about 1 nm around the hydrogen atom. Our result paves the way to control various broken-symmetry states at the nanoscale by various atomic impurities.

cond-mat.mes-hall↗

arXiv2020

Distinct Topological Surface States on the Two Terminations of MnBi$_4$Te$_7$

The recent discovered intrinsic magnetic topological insulator MnBi2Te4 have been met with unusual success in hosting emergent phenomena such as the quantum anomalous Hall effect and the axion insulator states. However, the surface-bulk correspondence of the Mn-Bi-Te family, composed by the superlattice-like MnBi2Te4/(Bi2Te3)n (n = 0, 1, 2, 3 ...) layered structure, remains intriguing but elusive. Here, by using scanning tunneling microscopy (STM) and angle-resolved photoemission spectroscopy (ARPES) techniques, we unambiguously assign the two distinct surface states of MnBi4Te7 (n = 1) to the quintuple-layer (QL) Bi2Te3 termination and the septuple-layer (SL) MnBi2Te4 termination, respectively. A comparison of the experimental observations with theoretical calculations reveals the diverging topological behaviors, especially the hybridization effect between magnetic and nonmagnetic layers, on the two terminations: a gap on the QL termination originating from the topological surface states of the QL hybridizing with the bands of the beneath SL, and a gapless Dirac-cone band structure on the SL termination with time-reversal symmetry. The quasi-particle interference patterns further confirm the topological nature of the surface states for both terminations, continuing far above the Fermi energy. The QL termination carries a spin-helical Dirac state with hexagonal warping, while at the SL termination, a strongly canted helical state from the surface lies between a pair of Rashba-split states from its neighboring layer. Our work elucidates an unprecedented hybridization effect between the building blocks of the topological surface states, and also reveals the termination-dependent time-reversal symmetry breaking in a magnetic topological insulator, rendering an ideal platform to realize the half-integer quantum Hall effect and relevant quantum phenomena.

cond-mat.mtrl-sci↗

arXiv2020

Adversarial Representation Learning for Robust Patient-Independent Epileptic Seizure Detection

Objective: Epilepsy is a chronic neurological disorder characterized by the occurrence of spontaneous seizures, which affects about one percent of the world's population. Most of the current seizure detection approaches strongly rely on patient history records and thus fail in the patient-independent situation of detecting the new patients. To overcome such limitation, we propose a robust and explainable epileptic seizure detection model that effectively learns from seizure states while eliminates the inter-patient noises. Methods: A complex deep neural network model is proposed to learn the pure seizure-specific representation from the raw non-invasive electroencephalography (EEG) signals through adversarial training. Furthermore, to enhance the explainability, we develop an attention mechanism to automatically learn the importance of each EEG channels in the seizure diagnosis procedure. Results: The proposed approach is evaluated over the Temple University Hospital EEG (TUH EEG) database. The experimental results illustrate that our model outperforms the competitive state-of-the-art baselines with low latency. Moreover, the designed attention mechanism is demonstrated ables to provide fine-grained information for pathological analysis. Conclusion and significance: We propose an effective and efficient patient-independent diagnosis approach of epileptic seizure based on raw EEG signals without manually feature engineering, which is a step toward the development of large-scale deployment for real-life use.

eess.SP↗

arXiv2020

Discriminative Topic Mining via Category-Name Guided Text Embedding

Mining a set of meaningful and distinctive topics automatically from massive text corpora has broad applications. Existing topic models, however, typically work in a purely unsupervised way, which often generate topics that do not fit users' particular needs and yield suboptimal performance on downstream tasks. We propose a new task, discriminative topic mining, which leverages a set of user-provided category names to mine discriminative topics from text corpora. This new task not only helps a user understand clearly and distinctively the topics he/she is most interested in, but also benefits directly keyword-driven classification tasks. We develop CatE, a novel category-name guided text embedding method for discriminative topic mining, which effectively leverages minimal user guidance to learn a discriminative embedding space and discover category representative terms in an iterative manner. We conduct a comprehensive set of experiments to show that CatE mines high-quality set of topics guided by category names only, and benefits a variety of downstream applications including weakly-supervised classification and lexical entailment direction identification.

cs.CL↗

arXiv2020

Intelligent Reflecting Surface Assisted Secure Wireless Communications with Multiple-Transmit and Multiple-Receive Antennas

In this paper, we propose intelligent reflecting surfaces (IRS) assisted secure wireless communications with multi-input and multi-output antennas (IRS-MIMOME). The considered scenario is an access point (AP) equipped with multiple antennas communicates with a multi-antenna enabled legitimate user in the downlink at the present of an eavesdropper configured with multiple antennas. Particularly, the joint optimization of the transmit covariance matrix at the AP and the reflecting coefficients at the IRS to maximize the secrecy rate for the IRS-MIMOME system is investigated, with two different assumptions on the phase shifting capabilities at the IRS, i.e., the IRS has the continuous reflecting coefficients and the IRS has the discrete reflecting coefficients. For the former case, due to the non-convexity of the formulated problem, an alternating optimization (AO)-based algorithm is proposed, i.e., for given the reflecting coefficients at the IRS, the successive convex approximation (SCA)-based algorithm is used to solve the transmit covariance matrix optimization, while given the transmit covariance matrix at the AP, alternative optimization is used again in individually optimizing of each reflecting coefficient at the IRS with other fixed reflecting coefficients. For the individual reflecting coefficient optimization, the close-form or an interval of the optimal solution is provided. Then, the proposed algorithm is extended to the discrete reflecting coefficient model at the IRS. Finally, some numerical simulations have been done to demonstrate that the proposed algorithm outperforms other benchmark schemes.

cs.IT↗

arXiv2020

Context-Sensitive and Duration-Aware Qubit Mapping for Various NISQ Devices

Quantum computing (QC) technologies have reached a second renaissance in the last decade. Some fully programmable QC devices have been built based on superconducting or ion trap technologies. Although different quantum technologies have their own parameter indicators, QC devices in the NISQ era share common features and challenges such as limited qubits and connectivity, short coherence time and high gate error rates. Quantum programs written by programmers could hardly run on real hardware directly since two-qubit gates are usually allowed on few pairs of qubits. Therefore, quantum computing compilers must resolve the mapping problem and transform original programs to fit the hardware limitation. To address the issues mentioned above, we summarize different quantum technologies and abstractly define Quantum Abstract Machine (QAM); then propose a COntext-sensitive and Duration-Aware Remapping algorithm (Codar) based on the QAM. By introducing lock for each qubit, Codar is aware of gate duration difference and program context, which bring it abilities to extract more program's parallelism and reduce program execution time. Compared to the best-known algorithm, Codar halves the total execution time of several quantum algorithms and cut down 17.5% - 19.4% total execution time on average in different architectures.

quant-ph↗

arXiv2020

Knowledge Distillation from Internal Representations

Knowledge distillation is typically conducted by training a small model (the student) to mimic a large and cumbersome model (the teacher). The idea is to compress the knowledge from the teacher by using its output probabilities as soft-labels to optimize the student. However, when the teacher is considerably large, there is no guarantee that the internal knowledge of the teacher will be transferred into the student; even if the student closely matches the soft-labels, its internal representations may be considerably different. This internal mismatch can undermine the generalization capabilities originally intended to be transferred from the teacher to the student. In this paper, we propose to distill the internal representations of a large model such as BERT into a simplified version of it. We formulate two ways to distill such representations and various algorithms to conduct the distillation. We experiment with datasets from the GLUE benchmark and consistently show that adding knowledge distillation from internal representations is a more powerful method than only using soft-label distillation.

cs.CL↗

arXiv2020

Dirac Eigenvalue spectrum of $N_f$=2+1 QCD towards the chiral limit using HISQ fermions

We utilize the eigenvalue filtering technique combined with the stochastic estimate of the mode number to determine the eigenvalue spectrum. Simulations of (2 + 1)-flavor QCD are performed using the Highly Improved Staggered Quarks (HISQ/tree) action on $N_τ$ = 8 lattices with aspect ratios $N_σ/N_τ$ ranging from 5 to 7. The strange quark mass is fixed to its physical value $m_{s}^{\rm phy}$, and the light quark masses $m_{l}$ are varied from $m_{s}^{\rm phy}/40$ to $m_{s}^{\rm phy}/160$ which correspond to pion mass $m_π$ ranging from 110 MeV to 55 MeV in the continuum limit. We compute the chiral condensate and $χ_π - χ_δ$ through the eigenvalue spectrum obtained from the the eigenvalue filtering method. We compare these results with those obtained from a direct calculation of the observables which involves inversions of the fermion matrix using the stochastic "noise vector" method. We find that these approaches yield consistent results. Furthermore, we also investigate the quark mass and temperature dependences of the Dirac eigenvalue density at zero eigenvalues to gain more insights about the $U_A(1)$ symmetry breaking in QCD.

hep-lat↗