arXiv subjects

Yongqiang Wang

Publications and source records attributed to Yongqiang Wang.

At least 91 records · Page 5Linked to original sources

arXiv2020

Privacy-Preserving Dynamic Average Consensus via State Decomposition: Case Study on Multi-Robot Formation Control

Dynamic average consensus is a decentralized control/estimation framework where a group of agents cooperatively track the average of local time-varying reference signals. In this paper, we develop a novel state decomposition-based privacy preservation scheme to protect the privacy of agents when sharing information with neighboring agents. Specifically, we first show that an external eavesdropper can successfully wiretap the reference signals of all agents in a conventional dynamic average consensus algorithm. To protect privacy against the eavesdropper, a state decomposition scheme is developed where the original state of each agent is decomposed into two sub-states: one succeeds the role of the original state in inter-node interactions, while the other sub-state only communicates with the first one and is invisible to other neighboring agents. Rigorous analyses are performed to show that 1) the proposed privacy scheme preserves the convergence of the average consensus; and 2) the privacy of the agents is protected such that an eavesdropper cannot discover the private reference signals with any guaranteed accuracy. The developed privacy-preserving dynamic average consensus framework is then applied to the formation control of multiple non-holonomic mobile robots, in which the efficacy of the scheme is demonstrated. Numerical simulation is provided to illustrate the effectiveness of the proposed approach.

eess.SY↗

arXiv2020

Impact of interfaces on the radiation response and underlying defect recovery mechanisms in nanostructured Cu-Fe-Ag

Newest developments in nuclear fission and fusion technology as well as planned long-distance space missions demand novel materials to withstand harsh, irradiative environments. Radiation-induced hardening and embrittlement are a concern that can lead to failure of materials deployed in these applications. Here the underlying mechanisms are accommodation and clustering of lattice defects created by the incident radiation particles. Interfaces, such as free surfaces, phase and grain boundaries, are known for trapping and annihilating defects and therefore preventing these radiation-induced defects from forming clusters. In this work, differently structured nanocomposite materials based on Cu-Fe-Ag were fabricated using a novel solid-state route, combining severe plastic deformation with thermal and electrochemical treatments. The influence of different interface types and spacings on radiation effects in these materials was investigated using nanoindentation. Interface-rich bulk nanocomposites showed a slight decrease in hardness after irradiation, whereas the properties of a nanoporous material remain mostly unchanged. An explanation for this different material behavior and its link to recovery mechanisms at interfaces is attempted in this work, paving a concept towards radiation resistant materials.

cond-mat.mtrl-sci↗

arXiv2020

Privacy-Preserved Collaborative Estimation for Networked Vehicles with Application to Road Anomaly Detection

Road information such as road profile and traffic density have been widely used in intelligent vehicle systems to improve road safety, ride comfort, and fuel economy. However, vehicle heterogeneity and parameter uncertainty make it extremely difficult for a single vehicle to accurately and reliably measure such information. In this work, we propose a unified framework for learning-based collaborative estimation to fuse local road estimation from a fleet of connected heterogeneous vehicles. The collaborative estimation scheme exploits the sequential measurements made by multiple vehicles traversing the same road segment and let these vehicles relay a learning signal to iteratively refine local estimations. Given that the privacy of individual vehicles' identity must be protected in collaborative estimation, we directly incorporate privacy-protection design into the collaborative estimation design and establish a unified framework for privacy-preserving collaborative estimation. Different from patching conventional privacy mechanisms like differential privacy which will compromise algorithmic accuracy or homomorphic encryption which will incur heavy communication/computational overhead, we leverage the dynamical properties of collective estimation to enable inherent privacy protection without sacrificing accuracy or significantly increasing communication/computation overhead. Numerical simulations confirm the effectiveness and efficiency of our proposed framework.

eess.SY↗

arXiv2020

Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces

In this work, we first show that on the widely used LibriSpeech benchmark, our transformer-based context-dependent connectionist temporal classification (CTC) system produces state-of-the-art results. We then show that using wordpieces as modeling units combined with CTC training, we can greatly simplify the engineering pipeline compared to conventional frame-based cross-entropy training by excluding all the GMM bootstrapping, decision tree building and force alignment steps, while still achieving very competitive word-error-rate. Additionally, using wordpieces as modeling units can significantly improve runtime efficiency since we can use larger stride without losing accuracy. We further confirm these findings on two internal VideoASR datasets: German, which is similar to English as a fusional language, and Turkish, which is an agglutinative language.

eess.AS↗

arXiv2020

Weak-Attention Suppression For Transformer Based Speech Recognition

Transformers, originally proposed for natural language processing (NLP) tasks, have recently achieved great success in automatic speech recognition (ASR). However, adjacent acoustic units (i.e., frames) are highly correlated, and long-distance dependencies between them are weak, unlike text units. It suggests that ASR will likely benefit from sparse and localized attention. In this paper, we propose Weak-Attention Suppression (WAS), a method that dynamically induces sparsity in attention probabilities. We demonstrate that WAS leads to consistent Word Error Rate (WER) improvement over strong transformer baselines. On the widely used LibriSpeech benchmark, our proposed method reduced WER by 10%$ on test-clean and 5% on test-other for streamable transformers, resulting in a new state-of-the-art among streaming models. Further analysis shows that WAS learns to suppress attention of non-critical and redundant continuous acoustic frames, and is more likely to suppress past frames rather than future ones. It indicates the importance of lookahead in attention-based ASR models.

eess.AS↗

arXiv2020

Streaming Transformer-based Acoustic Models Using Self-attention with Augmented Memory

Transformer-based acoustic modeling has achieved great suc-cess for both hybrid and sequence-to-sequence speech recogni-tion. However, it requires access to the full sequence, and thecomputational cost grows quadratically with respect to the in-put sequence length. These factors limit its adoption for stream-ing applications. In this work, we proposed a novel augmentedmemory self-attention, which attends on a short segment of theinput sequence and a bank of memories. The memory bankstores the embedding information for all the processed seg-ments. On the librispeech benchmark, our proposed methodoutperforms all the existing streamable transformer methods bya large margin and achieved over 15% relative error reduction,compared with the widely used LC-BLSTM baseline. Our find-ings are also confirmed on some large internal datasets.

eess.AS↗

arXiv2020

Global Synchronization of Pulse-Coupled Oscillator Networks Under Byzantine Attacks

Synchronization of pulse-coupled oscillators (PCOs) has gained significant attention recently due to their increased applications in sensor networks and wireless communications. Given the distributed and unattended nature of wireless sensor networks, it is imperative to enhance the resilience of PCO synchronization against malicious attacks. However, most existing results on attack-resilient pulse-based synchronization are obtained under assumptions of all-to-all coupling topologies or restricted initial phase distributions. In this paper, we propose a new pulse-based synchronization mechanism to improve the attack resilience of PCO synchronization that is applicable to non-all-to-all networks. Under the proposed synchronization mechanism, we prove that perfect synchronization of legitimate oscillators can be guaranteed in the presence of multiple Byzantine attackers who can emit attack pulses arbitrarily without any constraint except that practical bit rate constraint renders the number of pulses from an attacker to be finite. The new mechanism can guarantee synchronization even when the initial phases of all legitimate oscillators are arbitrarily distributed in the entire oscillation period, which is in distinct difference from most existing attack-resilient synchronization approaches (including the seminal paper from Lamport and Melliar-Smith [1]) that require a priori (almost) synchronization among legitimate oscillators. Numerical simulation results are given to confirm the theoretical results.

eess.SP↗

arXiv2020

Transformer-based Acoustic Modeling for Hybrid Speech Recognition

We propose and evaluate transformer-based acoustic models (AMs) for hybrid speech recognition. Several modeling choices are discussed in this work, including various positional embedding methods and an iterated loss to enable training deep transformers. We also present a preliminary study of using limited right context in transformer models, which makes it possible for streaming applications. We demonstrate that on the widely used Librispeech benchmark, our transformer-based AM outperforms the best published hybrid result by 19% to 26% relative when the standard n-gram language model (LM) is used. Combined with neural network LM for rescoring, our proposed approach achieves state-of-the-art results on Librispeech. Our findings are also confirmed on a much larger internal dataset.

cs.CL↗

arXiv2020

Training ASR models by Generation of Contextual Information

Supervised ASR models have reached unprecedented levels of accuracy, thanks in part to ever-increasing amounts of labelled training data. However, in many applications and locales, only moderate amounts of data are available, which has led to a surge in semi- and weakly-supervised learning research. In this paper, we conduct a large-scale study evaluating the effectiveness of weakly-supervised learning for speech recognition by using loosely related contextual information as a surrogate for ground-truth labels. For weakly supervised training, we use 50k hours of public English social media videos along with their respective titles and post text to train an encoder-decoder transformer model. Our best encoder-decoder models achieve an average of 20.8% WER reduction over a 1000 hours supervised baseline, and an average of 13.4% WER reduction when using only the weakly supervised encoder for CTC fine-tuning. Our results show that our setup for weak supervision improved both the encoder acoustic representations as well as the decoder language generation abilities.

cs.CL↗

arXiv2020

Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks

Deep acoustic models typically receive features in the first layer of the network, and process increasingly abstract representations in the subsequent layers. Here, we propose to feed the input features at multiple depths in the acoustic model. As our motivation is to allow acoustic models to re-examine their input features in light of partial hypotheses we introduce intermediate model heads and loss function. We study this architecture in the context of deep Transformer networks, and we use an attention mechanism over both the previous layer activations and the input features. To train this model's intermediate output hypothesis, we apply the objective function at each layer right before feature re-use. We find that the use of such iterated loss significantly improves performance by itself, as well as enabling input feature re-use. We present results on both Librispeech, and a large scale video dataset, with relative improvements of 10 - 20% for Librispeech and 3.2 - 13% for videos.

cs.CL↗

arXiv2020

A new family of disorder-free Rare-Earth-based kagomé lattice magnets: structure and magnetic characterizations of RE3BWO9 (RE=Pr, Nd, Gd-Ho) Boratotungstates

Exploration of rare-earth (RE)-based Kagome lattice magnets with spin-orbital entangled jeff=1/2 moments will provide new platform for investigating the exotic magnetic phases. Here, we report a new family of RE3BWO9 (RE=Pr,Nd,Gd-Ho) boratotungstates with magnetic RE3+ ions arranged on Kagome lattice, and perform its structure and magnetic characterizations. This serial compounds crystallize in hexagonal coordinated structure with space group P63 (No.173), where magnetic RE3+ ions have distorted Kagomé lattice connections within the ab plane and stacked in a AB-type fashion along c axis. The interlayer RE-RE separation is comparable with that of intralayer distance, forming 3-dimensional (3D) exchange coupled magnetic framework of RE3+ ions. The magnetic susceptibility data of RE3BWO9 (RE=Pr, Nd, Gd-Ho) reveal dominant antiferromagnetic interactions between magnetic RE3+ ions, but without visible magnetic ordering down to 2 K. The magnetization analyses for different RE3+ ions show diverse anisotropic behaviors, make RE3BWO9 as an appealing Kagome-lattice antiferromagnet to explore exotic magnetic phases.

cond-mat.mtrl-sci↗

arXiv2020

An Attack-Resilient Pulse-Based Synchronization Strategy for General Connected Topologies

Synchronization of pulse-coupled oscillators (PCOs) has gained significant attention recently due to increased applications in sensor networks and wireless communications. Given the distributed and unattended nature of wireless sensor networks, it is imperative to enhance the resilience of pulse-based synchronization against malicious attacks. However, most existing results on resilient PCO synchronization are obtained for all-to-all networks. We propose a new pulse-based synchronization mechanism to improve the resilience of PCO synchronization that is applicable under general connected topologies. Under the proposed synchronization mechanism, we rigorously characterize the condition for stealthy Byzantine attacks and prove that perfect synchronization of legitimate oscillators can be guaranteed in the presence of multiple stealthy Byzantine attackers, irrespective of whether the attackers collude with each other or not. The new mechanism can guarantee resilient synchronization even when the initial phases of legitimate oscillators are widely distributed in a half circle, which is in distinct difference from most existing attack-resilient synchronization algorithms (including the seminal paper from Lamport and Melliar-Smith [1]) that require a priori (almost) synchronization among legitimate oscillators. Numerical simulation results are given to confirm the theoretical results.

eess.SP↗

arXiv2019

Effect of Proton Irradiation Temperature on Zinc Oxide Metal-Semiconductor-Metal Ultraviolet Photodetectors

The electrical and structural characteristics of 50 nm zinc oxide (ZnO) metal-semiconductor-metal (MSM) ultraviolet (UV) photodetectors subjected to proton irradiation at different temperatures are reported and compared. We irradiated the devices with 200 keV protons to a fluence of 1016 cm-2. Examination of the X-ray diffraction (XRD) rocking curves indicates a strongly preferred (100) orientation for the grains of the as-deposited film, with decreases in crystal quality for all irradiated samples. In addition, peak shifts in XRD and Raman spectra of the control sample relative to well-known theoretical positions are indicative of tensile strain in the as-deposited ZnO films. We observed shifts of these peaks towards theoretical unstrained positions in the irradiated films relative to the as-deposited film indicate partial relaxation of this strain. Raman spectra also indicate increases of oxygen vacancies (V_O ) and zinc interstitials (Zn_i ) relative to the control sample. Additionally, photocurrent versus time measurements showed up to 2x increases in time constants for samples irradiated at lower temperatures months after irradiation, indicating that the defects introduced by suppression of thermally-activated dynamic annealing process has a long-term deleterious effect on device performance.

physics.app-ph↗

arXiv2019

Improving N-gram Language Models with Pre-trained Deep Transformer

Although n-gram language models (LMs) have been outperformed by the state-of-the-art neural LMs, they are still widely used in speech recognition due to its high efficiency in inference. In this paper, we demonstrate that n-gram LM can be improved by neural LMs through a text generation based data augmentation method. In contrast to previous approaches, we employ a large-scale general domain pre-training followed by in-domain fine-tuning strategy to construct deep Transformer based neural LMs. Large amount of in-domain text data is generated with the well trained deep Transformer to construct new n-gram LMs, which are then interpolated with baseline n-gram systems. Empirical studies on different speech recognition tasks show that the proposed approach can effectively improve recognition accuracy. In particular, our proposed approach brings significant relative word error rate reduction up to 6.0% for domains with limited in-domain data.

cs.CL↗

arXiv2019

Chemical manipulation of hydrogen induced high p-type and n-type conductivity in Ga2O3

Advancement of optoelectronic and high-power devices is tied to the development of wide band gap materials with excellent transport properties. However, bipolar doping (n-type and p-type doping) and realizing high carrier density while maintaining good mobility have been big challenges in wide band gap materials. Here P-type and n-type conductivity was introduced in beta-Ga2O3, an ultra-wide band gap oxide, by controlling hydrogen incorporation in the lattice without further doping. Hydrogen induced a 9-order of magnitude increase of n-type conductivity with donor ionization energy of 20 meV and resistivity of 10-4 Ohm.cm. The conductivity was switched to p-type with acceptor ionization energy of 42 meV by altering hydrogen incorporation in the lattice. Density functional theory calculations were used to examine hydrogen location in the Ga2O3 lattice and identified a new donor type as the source of this remarkable n-type conductivity. Positron annihilation spectroscopy confirmed this finding and the interpretation of the results. This work illustrates a new approach that allows a tunable and reversible way of modifying the conductivity of semiconductors and it is expected to have profound implications on semiconductor field. At the same time it demonstrates for the first time p-type and remarkable n-type conductivity in Ga2O3 which should usher in the development of Ga2O3 devices and advance optoelectronics and high-power devices

cond-mat.mtrl-sci↗

arXiv2019

Transformer-Transducer: End-to-End Speech Recognition with Self-Attention

We explore options to use Transformer networks in neural transducer for end-to-end speech recognition. Transformer networks use self-attention for sequence modeling and comes with advantages in parallel computation and capturing contexts. We propose 1) using VGGNet with causal convolution to incorporate positional information and reduce frame rate for efficient inference 2) using truncated self-attention to enable streaming for Transformer and reduce computational complexity. All experiments are conducted on the public LibriSpeech corpus. The proposed Transformer-Transducer outperforms neural transducer with LSTM/BLSTM networks and achieved word error rates of 6.37 % on the test-clean set and 15.30 % on the test-other set, while remaining streamable, compact with 45.7M parameters for the entire system, and computationally efficient with complexity of O(T), where T is input sequence length.

eess.AS↗

arXiv2019

Decentralized Heading Control with Rate Constraints using Pulse-Coupled Oscillators

Decentralized heading control is crucial for robotic network operations such as surveillance, exploration, and cooperative construction. However, few results consider decentralized heading control when the speed of heading adjustment is restricted. In this paper, we propose a simple hybrid-dynamical model based on pulse-coupled oscillators for decentralized heading control in mobile robots while accounting for the constraint on the rate of heading change. The pulse-coupled oscillator model is effective in coordinating the phase of oscillator networks and hence is promising for robotic heading coordination given that both phase and heading evolve on the same one-dimensional torus. However, existing pulse-coupled oscillator results require the phase adjustment to be instantaneous, which cannot hold for robot heading adjustment due to physical limitations. We propose a generalization to the standard pulse-coupled oscillator model to allow for the phase to adjust at a finite rate, yet still have the oscillator network converge to the desired state, making our approach applicable to robotic heading coordination under rate constraints. We provide rigorous mathematical proof for the achievement of both synchronized and desynchronized heading relationships, and experimentally verify the results using extensive tests on a multi-robot platform.

eess.SY↗

arXiv2019

On the Global Synchronization of Pulse-coupled Oscillators Interacting on Chain and Directed Tree Graphs

Driven by increased applications in biological networks and wireless sensor networks, synchronization of pulse-coupled oscillators (PCOs) has gained increased popularity. However, most existing results address the local synchronization of PCOs with initial phases constrained in a half cycle, and results on global synchronization from any initial condition are very sparse. In this paper, we address global PCO synchronization from an arbitrary phase distribution under chain or directed tree graphs. Our results differ from existing global synchronization studies on decentralized PCO networks in two key aspects: first, our work allows heterogeneous coupling functions, and we analyze the behavior of oscillators with perturbations on their natural frequencies; secondly, rather than requiring a large enough coupling strength, our results hold under any coupling strength between zero and one, which is crucial because a large coupling strength has been shown to be detrimental to the robustness of PCO synchronization to disturbances.

math.DS↗