arXiv subjects

Yu Zhang

Publications and source records attributed to Yu Zhang.

At least 1,153 records · Page 64Linked to original sources

arXiv2022

FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech

We introduce FLEURS, the Few-shot Learning Evaluation of Universal Representations of Speech benchmark. FLEURS is an n-way parallel speech dataset in 102 languages built on top of the machine translation FLoRes-101 benchmark, with approximately 12 hours of speech supervision per language. FLEURS can be used for a variety of speech tasks, including Automatic Speech Recognition (ASR), Speech Language Identification (Speech LangID), Translation and Retrieval. In this paper, we provide baselines for the tasks based on multilingual pre-trained models like mSLAM. The goal of FLEURS is to enable speech technology in more languages and catalyze research in low-resource speech understanding.

cs.CL↗

arXiv2022

SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing

Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning. The SpeechT5 framework consists of a shared encoder-decoder network and six modal-specific (speech/text) pre/post-nets. After preprocessing the input speech/text through the pre-nets, the shared encoder-decoder network models the sequence-to-sequence transformation, and then the post-nets generate the output in the speech/text modality based on the output of the decoder. Leveraging large-scale unlabeled speech and text data, we pre-train SpeechT5 to learn a unified-modal representation, hoping to improve the modeling capability for both speech and text. To align the textual and speech information into this unified semantic space, we propose a cross-modal vector quantization approach that randomly mixes up speech/text states with latent units as the interface between encoder and decoder. Extensive evaluations show the superiority of the proposed SpeechT5 framework on a wide variety of spoken language processing tasks, including automatic speech recognition, speech synthesis, speech translation, voice conversion, speech enhancement, and speaker identification. We release our code and model at https://github.com/microsoft/SpeechT5.

eess.AS↗

arXiv2022

Co-optimization of Battery Routing and Load Restoration for Microgrids with Mobile Energy Storage Systems

Mobile energy storage systems (MESS) offer great operational flexibility to enhance the resiliency of distribution systems in an emergency condition. The optimal placement and sizing of those units are pivotal for quickly restoring the curtailed loads. In this paper, we propose a model for load restoration in a microgrid while concurrently optimizing the MESS routes required for the same. The model is formulated as a mixed integer second order cone program by considering the state of charge and evolution of the lower and upper bounds of battery capacities. Simulation results tested on the IEEE 123- bus benchmark system demonstrate the efficacy of the proposed model.

eess.SY↗

arXiv2022

Transferable Physical Attack against Object Detection with Separable Attention

Transferable adversarial attack is always in the spotlight since deep learning models have been demonstrated to be vulnerable to adversarial samples. However, existing physical attack methods do not pay enough attention on transferability to unseen models, thus leading to the poor performance of black-box attack.In this paper, we put forward a novel method of generating physically realizable adversarial camouflage to achieve transferable attack against detection models. More specifically, we first introduce multi-scale attention maps based on detection models to capture features of objects with various resolutions. Meanwhile, we adopt a sequence of composite transformations to obtain the averaged attention maps, which could curb model-specific noise in the attention and thus further boost transferability. Unlike the general visualization interpretation methods where model attention should be put on the foreground object as much as possible, we carry out attack on separable attention from the opposite perspective, i.e. suppressing attention of the foreground and enhancing that of the background. Consequently, transferable adversarial camouflage could be yielded efficiently with our novel attention-based loss function. Extensive comparison experiments verify the superiority of our method to state-of-the-art methods.

cs.CV↗

arXiv2022

Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation

Direct Speech-to-speech translation (S2ST) has drawn more and more attention recently. The task is very challenging due to data scarcity and complex speech-to-speech mapping. In this paper, we report our recent achievements in S2ST. Firstly, we build a S2ST Transformer baseline which outperforms the original Translatotron. Secondly, we utilize the external data by pseudo-labeling and obtain a new state-of-the-art result on the Fisher English-to-Spanish test set. Indeed, we exploit the pseudo data with a combination of popular techniques which are not trivial when applied to S2ST. Moreover, we evaluate our approach on both syntactically similar (Spanish-English) and distant (English-Chinese) language pairs. Our implementation is available at https://github.com/fengpeng-yue/speech-to-speech-translation.

cs.CL↗

arXiv2022

Side-aware Meta-Learning for Cross-Dataset Listener Diagnosis with Subjective Tinnitus

With the development of digital technology, machine learning has paved the way for the next generation of tinnitus diagnoses. Although machine learning has been widely applied in EEG-based tinnitus analysis, most current models are dataset-specific. Each dataset may be limited to a specific range of symptoms, overall disease severity, and demographic attributes; further, dataset formats may differ, impacting model performance. This paper proposes a side-aware meta-learning for cross-dataset tinnitus diagnosis, which can effectively classify tinnitus in subjects of divergent ages and genders from different data collection processes. Owing to the superiority of meta-learning, our method does not rely on large-scale datasets like conventional deep learning models. Moreover, we design a subject-specific training process to assist the model in fitting the data pattern of different patients or healthy people. Our method achieves a high accuracy of 73.8\% in the cross-dataset classification. We conduct an extensive analysis to show the effectiveness of side information of ears in enhancing model performance and side-aware meta-learning in improving the quality of the learned features.

eess.SP↗

arXiv2022

JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering

Existing KG-augmented models for commonsense question answering primarily focus on designing elaborate Graph Neural Networks (GNNs) to model knowledge graphs (KGs). However, they ignore (i) the effectively fusing and reasoning over question context representations and the KG representations, and (ii) automatically selecting relevant nodes from the noisy KGs during reasoning. In this paper, we propose a novel model, JointLK, which solves the above limitations through the joint reasoning of LM and GNN and the dynamic KGs pruning mechanism. Specifically, JointLK performs joint reasoning between LM and GNN through a novel dense bidirectional attention module, in which each question token attends on KG nodes and each KG node attends on question tokens, and the two modal representations fuse and update mutually by multi-step interactions. Then, the dynamic pruning module uses the attention weights generated by joint reasoning to prune irrelevant KG nodes recursively. We evaluate JointLK on the CommonsenseQA and OpenBookQA datasets, and demonstrate its improvements to the existing LM and LM+KG models, as well as its capability to perform interpretable reasoning.

cs.CL↗

arXiv2022

Variation-cognizant Probabilistic Power Flow Analysis via Multi-task Learning

With an increasing high penetration of solar photovoltaic generation in electric power grids, voltage phasors and branch power flows experience more severe fluctuations. In this context, probabilistic power flow (PPF) study aims at characterizing the statistical properties of the state of the system with respect to the random power injections. To avoid repeated power flow calculations involved in PPF study, the present paper leverages regression algorithms and neural networks to improve the estimation performance and speed up the computation. Specifically, based on the variation level of the voltage magnitude at each bus, we develop either a linear regression or a fully connected neural network to approximate the inverse AC power flow mappings. The proposed multi-task learning technique further improves the accuracy of branch flow estimation by incorporating the errors of voltage angle differences into the loss function design. Tested on IEEE-300 and IEEE-1354 bus systems with real data, the proposed methods achieve better performance in estimating voltage phasors and branch flows.

eess.SY↗

arXiv2022

Differentially Private Load Restoration for Microgrids with Distributed Energy Storage

Distributed energy storage systems (ESSs) can be efficiently leveraged for load restoration (LR) for a microgrid (MG) in island mode. When the ESSs are owned by third parties rather than the MG operator (MGO), the ESS operating setpoints may be considered as private information of their respective owners. Therefore, efforts must be put forth to avoid the disclosure through adversarial analysis of load setpoints. In his paper, we consider a scenario where LR takes place in a MG by determining load and ESS power injections through the solution of an AC optimal power flow (AC-OPF) problem. Since the charge/discharge mode at any given time is assumed to be private, we develop a differentially-private mechanism which restores load while maintaining privacy of ESS mode data. The performance of the proposed mechanism is demonstrated for a 33-bus MG.

eess.SY↗

arXiv2022

Interpretable Graph Convolutional Network of Multi-Modality Brain Imaging for Alzheimer's Disease Diagnosis

Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies. In this paper, we propose an interpretable Graph Convolutional Network (GCN) framework for the identification and classification of Alzheimer's disease (AD) using multi-modality brain imaging data. Specifically, we extended the Gradient Class Activation Mapping (Grad-CAM) technique to quantify the most discriminative features identified by GCN from brain connectivity patterns. We then utilized them to find signature regions of interest (ROIs) by detecting the difference of features between regions in healthy control (HC), mild cognitive impairment (MCI), and AD groups. We conducted the experiments on the ADNI database with imaging data from three modalities, including VBM-MRI, FDG-PET, and AV45-PET, and showed that the ROI features learned by our method were effective for enhancing the performances of both clinical score prediction and disease status identification. It also successfully identified biomarkers associated with AD and MCI.

cs.LG↗

arXiv2022

Adversarial Filtering Modeling on Long-term User Behavior Sequences for Click-Through Rate Prediction

Rich user behavior information is of great importance for capturing and understanding user interest in click-through rate (CTR) prediction. To improve the richness, collecting long-term behaviors becomes a typical approach in academy and industry but at the cost of increasing online storage and latency. Recently, researchers have proposed several approaches to shorten long-term behavior sequence and then model user interests. These approaches reduce online cost efficiently but do not well handle the noisy information in long-term user behavior, which may deteriorate the performance of CTR prediction significantly. To obtain better cost/performance trade-off, we propose a novel Adversarial Filtering Model (ADFM) to model long-term user behavior. ADFM uses a hierarchical aggregation representation to compress raw behavior sequence and then learns to remove useless behavior information with an adversarial filtering mechanism. The selected user behaviors are fed into interest extraction module for CTR prediction. Experimental results on public datasets and industrial dataset demonstrate that our method achieves significant improvements over state-of-the-art models.

cs.IR↗

arXiv2022

Modeling and Predicting Popularity Dynamics via Deep Learning Attention Mechanism

An ability to predict the popularity dynamics of individual items within a complex evolving system has important implications in a wide range of domains. Here we propose a deep learning attention mechanism to model the process through which individual items gain their popularity. We analyze the interpretability of the model with the four key phenomena confirmed independently in the previous studies of long-term popularity dynamics quantification, including the intrinsic quality, the aging effect, the recency effect and the Matthew effect. We analyze the effectiveness of introducing attention model in popularity dynamics prediction. Extensive experiments on a real-large citation data set demonstrate that the designed deep learning attention mechanism possesses remarkable power at predicting the long-term popularity dynamics. It consistently outperforms the existing methods, and achieves a significant performance improvement.

cs.SI↗

arXiv2022

Modeling and Predicting Citation Count via Recurrent Neural Network with Long Short-Term Memory

The rapid evolution of scientific research has been creating a huge volume of publications every year. Among the many quantification measures of scientific impact, citation count stands out for its frequent use in the research community. Although peer review process is the mainly reliable way of predicting a paper's future impact, the ability to foresee lasting impact on the basis of citation records is increasingly important in the scientific impact analysis in the era of big data. This paper focuses on the long-term citation count prediction for individual publications, which has become an emerging and challenging applied research topic. Based on the four key phenomena confirmed independently in previous studies of long-term scientific impact quantification, including the intrinsic quality of publications, the aging effect and the Matthew effect and the recency effect, we unify the formulations of all these observations in this paper. Building on a foundation of the above formulations, we propose a long-term citation count prediction model for individual papers via recurrent neural network with long short-term memory units. Extensive experiments on a real-large citation data set demonstrate that the proposed model consistently outperforms existing methods, and achieves a significant performance improvement.

cs.DL↗

arXiv2022

Neutrino dipole portal at electron colliders

We propose to search for a heavy neutral lepton (HNL), that is also know as sterile neutrino, in electron colliders running with the center-of-mass energies at few GeV, including BESIII, Belle II, and the proposed Super Tau Charm Factory (STCF). We consider the HNL interacting with Standard Model neutrino and photon via a transition magnetic moment, the so-called dipole portal.We use the monophoton signature at electron colliders to probe the constraints on the active-sterile neutrino transition magnetic moments $d$ as the function of the HNL's mass $m_N$.It is found that BESIII, Belle II and STCF can probe the upper limits for $d$ down to 1.3 $\times 10^{-5}\ {\rm GeV}^{-1}$, 8 $\times 10^{-6}\ {\rm GeV}^{-1}$, and 1.3 $\times 10^{-6}\ {\rm GeV}^{-1}$ with $m_N$ around GeV scale, respectively, and have sensitivity to the previously unexplored parameter space for electron- ($d_e$) and tau-neutrino ($d_τ$) dipole portal with $m_N$ from dozens to thounsands MeV. On $d_μ$ for HNL mixing with the {muon}-neutrino, Belle II and STCF can also provide leading constraints.

hep-ph↗

arXiv2022

XTREME-S: Evaluating Cross-lingual Speech Representations

We introduce XTREME-S, a new benchmark to evaluate universal cross-lingual speech representations in many languages. XTREME-S covers four task families: speech recognition, classification, speech-to-text translation and retrieval. Covering 102 languages from 10+ language families, 3 different domains and 4 task families, XTREME-S aims to simplify multilingual speech representation evaluation, as well as catalyze research in "universal" speech representation learning. This paper describes the new benchmark and establishes the first speech-only and speech-text baselines using XLS-R and mSLAM on all downstream tasks. We motivate the design choices and detail how to use the benchmark. Datasets and fine-tuning scripts are made easily accessible at https://hf.co/datasets/google/xtreme_s.

cs.CL↗

arXiv2022

Durable and Recoverable Hydrophilicity of Polyethylene Terephthalate Fabric Prepared with Plasma Selective Etching

Durable delustered PET (PET-TiO2) fabrics super hydrophilic surface has been obtained by plasma selecting etching. The aging effect of their hydrophilicity after plasma treatment has been investigated with storage time. After Ar/O2 radio frequency (RF) plasma treatment for only 7 min, PET-TiO2 fabric showed water contact angle of 0o. After 10 month storage time, it keeps its water contact angle below 75.7o. Further more, with Xenon light irradiation for 10 min, it is firstly found that it has well-recovered water contact angle to 5°. While the contact angle of PET fabric for 7 min returns to 123.0° and its hydrophilicity disappeared almost completely and showed no response to Xenon light irradiation. The water absorption rate of 7 min plasma treated PET-TiO2 fabric increased by 57.54%. By field emission scanning electron microscopy (FE-SEM), X-ray photoelectron spectroscopy (XPS) and X-ray diffraction analysis(XRD) measurement, waviness structure of humps and ridges with irregular particles or pits were found on the plasma treated PET-TiO2 fabric surface and increased Ti atomic percentage was observed. It is verified that TiO2 particles inside PET-TiO2 fiber have been exposed to its surface by plasma selective etching of its organic component. It suppresses the aging effect and is characterized with durable and recoverable hydrophilicity. This one step, quick, green and cost-resonable manufacture method has a pratical application for durable superhydrophilic surfaces.

physics.plasm-ph↗

arXiv2022

Efficient scheme for realizing a multiplex-controlled phase gate with photonic qubits in circuit quantum electrodynamics

We propose an efficient scheme to implement a multiplex-controlled phase gate with multiple photonic qubits simultaneously controlling one target photonic qubit based on circuit quantum electrodynamics (QED). For convenience, we denote this multiqubit gate as MCP gate. The gate is realized by using a two-level coupler to couple multiple cavities. The coupler here is a superconducting qubit. This scheme is simple because the gate implementation requires only \textit{one step} of operation. In addition, this scheme is quite general because the two logic states of each photonic qubit can be encoded with a vacuum state and an arbitrary non-vacuum state (e.g., a Fock state, a superposition of Fock states, a cat state, or a coherent state, etc.) which is orthogonal or quasi-orthogonal to the vacuum state. The scheme has some additional advantages: Because only two levels of the coupler are used, i.e., no auxiliary levels are utilized, decoherence from higher energy levels of the coupler is avoided; the gate operation time does not depend on the number of qubits; and the gate is implemented deterministically because no measurement is applied. As an example, we numerically analyze the circuit-QED based experimental feasibility of implementing a three-qubit MCP gate with photonic qubits each encoded via a vacuum state and a cat state. The scheme can be applied to accomplish the same task in a wide range of physical system, which consists of multiple microwave or optical cavities coupled to a two-level coupler such as a natural or artificial atom.

quant-ph↗

arXiv2022

Unsupervised Data Selection via Discrete Speech Representation for ASR

Self-supervised learning of speech representations has achieved impressive results in improving automatic speech recognition (ASR). In this paper, we show that data selection is important for self-supervised learning. We propose a simple and effective unsupervised data selection method which selects acoustically similar speech to a target domain. It takes the discrete speech representation available in common self-supervised learning frameworks as input, and applies a contrastive data selection method on the discrete tokens. Through extensive empirical studies we show that our proposed method reduces the amount of required pre-training data and improves the downstream ASR performance. Pre-training on a selected subset of 6% of the general data pool results in 11.8% relative improvements in LibriSpeech test-other compared to pre-training on the full set. On Multilingual LibriSpeech French, German, and Spanish test sets, selecting 6% data for pre-training reduces word error rate by more than 15% relatively compared to the full set, and achieves competitive results compared to current state-of-the-art performances.

eess.AS↗