arXiv subjects

Yu Zhang

Publications and source records attributed to Yu Zhang.

At least 1,081 records · Page 60Linked to original sources

arXiv2022

Sim2Real for Soft Robotic Fish via Differentiable Simulation

Accurate simulation of soft mechanisms under dynamic actuation is critical for the design of soft robots. We address this gap with our differentiable simulation tool by learning the material parameters of our soft robotic fish. On the example of a soft robotic fish, we demonstrate an experimentally-verified, fast optimization pipeline for learning the material parameters from quasi-static data via differentiable simulation and apply it to the prediction of dynamic performance. Our method identifies physically plausible Young's moduli for various soft silicone elastomers and stiff acetal copolymers used in creation of our three different robotic fish tail designs. We show that our method is compatible with varying internal geometry of the actuators, such as the number of hollow cavities. Our framework allows high fidelity prediction of dynamic behavior for composite bi-morph bending structures in real hardware to millimeter-accuracy and within 3 percent error normalized to actuator length. We provide a differentiable and robust estimate of the thrust force using a neural network thrust predictor; this estimate allows for accurate modeling of our experimental setup measuring bollard pull. This work presents a prototypical hardware and simulation problem solved using our differentiable framework; the framework can be applied to higher dimensional parameter inference, learning control policies, and computational design due to its differentiable character.

cs.RO↗

arXiv2022

Max Markov Chain

In this paper, we introduce Max Markov Chain (MMC), a novel representation for a useful subset of High-order Markov Chains (HMCs) with sparse correlations among the states. MMC is parsimony while retaining the expressiveness of HMCs. Even though parameter optimization is generally intractable as with HMC approximate models, it has an analytical solution, better sample efficiency, and the desired spatial and computational advantages over HMCs and approximate HMCs. Simultaneously, efficient approximate solutions exist for this type of chains as we show empirically, which allow MMCs to scale to large domains where HMCs and approximate HMCs would struggle to perform. We compare MMC with HMC, first-order Markov chain, and an approximate HMC model in synthetic domains with various data types to demonstrate that MMC is a valuable alternative for modeling stochastic processes and has many potential applications.

cs.AI↗

arXiv2022

A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition

We propose a quantum kernel learning (QKL) framework to address the inherent data sparsity issues often encountered in training large-scare acoustic models in low-resource scenarios. We project acoustic features based on classical-to-quantum feature encoding. Different from existing quantum convolution techniques, we utilize QKL with features in the quantum space to design kernel-based classifiers. Experimental results on challenging spoken command recognition tasks for a few low-resource languages, such as Arabic, Georgian, Chuvash, and Lithuanian, show that the proposed QKL-based hybrid approach attains good improvements over existing classical and quantum solutions.

cs.SD↗

arXiv2022

Photometric properties and stellar parameters of the rapidly rotating magnetic early-B star HD 345439

We first present the multicolor photometry results of the rapidly rotating magnetic star HD 345439 using the Nanshan One-meter Wide-field Telescope. From the photometric observations, we derive a rotational period of 0.7699\pm0.0014 day. The light curves of HD 345439 are dominated by the double asymmetric S-wave feature that arises from the magnetic clouds. Pulsating behaviors are not observed in Sector 41 of the Transiting Exoplanet Survey Satellite. No evidence is found of the occurrence of centrifugal breakout events neither in the residual flux nor in the systematic variations at the extremum of the light curve. Based on the hypothesis of the Rigidly Rotating Magnetosphere model, we restrict the magnetic obliquity angle {$β$} and the rotational inclination angle $i$ so that they satisfy the approximate relation {$β+ i \approx 105^{\circ}$}. The colour excess, extinction, and luminosity are determined to be $E_{(B-V)}=0.745\pm0.016\,$mag, $A_{V}=2.31\pm0.05\,$mag, and $\rm log\,(L/L_{\odot})=3.82\pm0.1 $dex, respectively. Furthermore, we derive the effective temperature as $T$$\rm _{eff}=22\pm1 $kK and the surface gravity as log$g=4.00\pm0.22$. The mass$ M=7.24_{-1.24}^{+1.75}\rm M_{\odot}$, radius$ R=4.44_{-1.93}^{+2.68}\rm R_{\odot}$, and age$\rm τ_{age}=23.62\,_{-21.97}^{+4.24} $Myr are estimated from the Hertzsprung--Russell Diagram

astro-ph.SR↗

arXiv2022

Accelerating RNN-T Training and Inference Using CTC guidance

We propose a novel method to accelerate training and inference process of recurrent neural network transducer (RNN-T) based on the guidance from a co-trained connectionist temporal classification (CTC) model. We made a key assumption that if an encoder embedding frame is classified as a blank frame by the CTC model, it is likely that this frame will be aligned to blank for all the partial alignments or hypotheses in RNN-T and it can be discarded from the decoder input. We also show that this frame reduction operation can be applied in the middle of the encoder, which result in significant speed up for the training and inference in RNN-T. We further show that the CTC alignment, a by-product of the CTC decoder, can also be used to perform lattice reduction for RNN-T during training. Our method is evaluated on the Librispeech and SpeechStew tasks. We demonstrate that the proposed method is able to accelerate the RNN-T inference by 2.2 times with similar or slightly better word error rates (WER).

eess.AS↗

arXiv2022

Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR

Knowledge distillation is an effective machine learning technique to transfer knowledge from a teacher model to a smaller student model, especially with unlabeled data. In this paper, we focus on knowledge distillation for the RNN-T model, which is widely used in state-of-the-art (SoTA) automatic speech recognition (ASR). Specifically, we compared using soft and hard target distillation to train large-scaleRNN-T models on the LibriSpeech/LibriLight public dataset (60k hours) and our in-house data (600k hours). We found that hard tar-gets are more effective when the teacher and student have different architecture, such as large teacher and small streaming student. On the other hand, soft target distillation works better in self-training scenario like iterative large teacher training. For a large model with0.6B weights, we achieve a new SoTA word error rate (WER) on LibriSpeech (8% relative improvement on dev-other) using Noisy Student Training with soft target distillation. It also allows our production teacher to adapt new data domain continuously.

cs.LG↗

arXiv2022

Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptation

Adapting a neural text-to-speech (TTS) model to a target speaker typically involves fine-tuning most if not all of the parameters of a pretrained multi-speaker backbone model. However, serving hundreds of fine-tuned neural TTS models is expensive as each of them requires significant footprint and separate computational resources (e.g., accelerators, memory). To scale speaker adapted neural TTS voices to hundreds of speakers while preserving the naturalness and speaker similarity, this paper proposes a parameter-efficient few-shot speaker adaptation, where the backbone model is augmented with trainable lightweight modules called residual adapters. This architecture allows the backbone model to be shared across different target speakers. Experimental results show that the proposed approach can achieve competitive naturalness and speaker similarity compared to the full fine-tuning approaches, while requiring only $\sim$0.1% of the backbone model parameters for each speaker.

cs.SD↗

arXiv2022

LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training

Language-based environment manipulation requires agents to manipulate the environment following natural language instructions, which is challenging due to the huge space of the environments. To address this challenge, various approaches have been proposed in recent work. Although these approaches work well for their intended environments, they are difficult to generalize across environments. In this work, we propose LEMON, a general framework for language-based environment manipulation tasks. Specifically, we first specify a task-agnostic approach for language-based environment manipulation tasks, which can deal with various environments using the same generative language model. Then we propose an execution-guided pre-training strategy to inject prior knowledge of environments to the language model with a pure synthetic pre-training corpus. Experimental results on tasks including Alchemy, Scene, Tangrams, ProPara and Recipes demonstrate the effectiveness of LEMON: it achieves new state-of-the-art results on four of the tasks, and the execution-guided pre-training strategy brings remarkable improvements on all experimental tasks.

cs.CL↗

arXiv2022

Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASR

Training state-of-the-art Automated Speech Recognition (ASR) models typically requires a substantial amount of transcribed speech. In this work, we demonstrate that a modality-matched joint speech and text model can be leveraged to train a massively multilingual ASR model without any supervised (manually transcribed) speech for some languages. This paper explores the use of jointly learnt speech and text representations in a massively multilingual, zero supervised speech, real-world setting to expand the set of languages covered by ASR with only unlabeled speech and text in the target languages. Using the FLEURS dataset, we define the task to cover $102$ languages, where transcribed speech is available in $52$ of these languages and can be used to improve end-to-end ASR quality on the remaining $50$. First, we show that by combining speech representations with byte-level text representations and use of language embeddings, we can dramatically reduce the Character Error Rate (CER) on languages with no supervised speech from 64.8\% to 30.8\%, a relative reduction of 53\%. Second, using a subset of South Asian languages we show that Maestro-U can promote knowledge transfer from languages with supervised speech even when there is limited to no graphemic overlap. Overall, Maestro-U closes the gap to oracle performance by 68.5\% relative and reduces the CER of 19 languages below 15\%.

cs.CL↗

arXiv2022

Improving generalizability of distilled self-supervised speech processing models under distorted settings

Self-supervised learned (SSL) speech pre-trained models perform well across various speech processing tasks. Distilled versions of SSL models have been developed to match the needs of on-device speech applications. Though having similar performance as original SSL models, distilled counterparts suffer from performance degradation even more than their original versions in distorted environments. This paper proposes to apply Cross-Distortion Mapping and Domain Adversarial Training to SSL models during knowledge distillation to alleviate the performance gap caused by the domain mismatch problem. Results show consistent performance improvements under both in- and out-of-domain distorted setups for different downstream tasks while keeping efficient model size.

cs.SD↗

arXiv2022

JOIST: A Joint Speech and Text Streaming Model For ASR

We present JOIST, an algorithm to train a streaming, cascaded, encoder end-to-end (E2E) model with both speech-text paired inputs, and text-only unpaired inputs. Unlike previous works, we explore joint training with both modalities, rather than pre-training and fine-tuning. In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works. Through a series of ablation studies, we explore different types of text modeling, including how to model the length of the text sequence and the appropriate text sub-word unit representation. We find that best text representation for JOIST improves WER across a variety of search and rare-word test sets by 4-14% relative, compared to a model not trained with text. In addition, we quantitatively show that JOIST maintains streaming capabilities, which is important for good user-level experience.

cs.CL↗

arXiv2022

Generating Training Data with Language Models: Towards Zero-Shot Language Understanding

Pretrained language models (PLMs) have demonstrated remarkable performance in various natural language processing tasks: Unidirectional PLMs (e.g., GPT) are well known for their superior text generation capabilities; bidirectional PLMs (e.g., BERT) have been the prominent choice for natural language understanding (NLU) tasks. While both types of models have achieved promising few-shot learning performance, their potential for zero-shot learning has been underexplored. In this paper, we present a simple approach that uses both types of PLMs for fully zero-shot learning of NLU tasks without requiring any task-specific data: A unidirectional PLM generates class-conditioned texts guided by prompts, which are used as the training data for fine-tuning a bidirectional PLM. With quality training data selected based on the generation probability and regularization techniques (label smoothing and temporal ensembling) applied to the fine-tuning stage for better generalization and stability, our approach demonstrates strong performance across seven classification tasks of the GLUE benchmark (e.g., 72.3/73.8 on MNLI-m/mm and 92.8 on SST-2), significantly outperforming zero-shot prompting methods and achieving even comparable results to strong few-shot approaches using 32 training samples per class.

cs.CL↗

arXiv2022

Experimental Implementation of Noncyclic and Nonadiabatic Geometric Quantum Gates in a Superconducting Circuit

Quantum gates based on geometric phases possess intrinsic noise-resilience features and therefore attract much attention. However, the implementations of previous geometric quantum computation typically require a long pulse time of gates. As a result, their experimental control inevitably suffers from the cumulative disturbances of systematic errors due to excessive time consumption. Here, we experimentally implement a set of noncyclic and nonadiabatic geometric quantum gates in a superconducting circuit, which greatly shortens the gate time. And also, we experimentally verify that our universal single-qubit geometric gates are more robust to both the Rabi frequency error and qubit frequency shift-induced error, compared to the conventional dynamical gates, by using the randomized benchmarking method. Moreover, this scheme can be utilized to construct two-qubit geometric operations, while the generation of the maximally entangled Bell states is demonstrated. Therefore, our results provide a promising routine to achieve fast, high-fidelity, and error-resilient quantum gates in superconducting quantum circuits.

quant-ph↗

arXiv2022

Towards Multi-class Pre-movement Classification

In non-invasive brain-computer interface systems, pre-movement decoding plays an important role in the detection of movement before limbs actually move. Movement-related cortical potential is a kind of brain activity associated with pre-movement decoding. In current studies, patterns decoded from movement are mainly applied to the binary classification between movement state and resting state, such as elbow flexion and rest. The classifications between two movement states and among multiple movement states are still challenging. This study proposes a new method, the star-arrangement spectral filtering (SASF), to solve the multi-class pre-movement classification problem. We first design a referenced task-related component analysis (RTRCA) framework that consists of two modules. This first module is the classification between movement state and resting state; the second module is the classification of multiple movement states. SASF is developed by optimizing the features in RTRCA. In SASF, feature selection on filter banks is used on the first module of RTRCA, and feature selection on time windows is used on the second module of RTRCA. A linear discriminant analysis classifier is used to classify the optimized features. In the binary classification between two motions, the classification accuracy of SASF achieves 0.9670$\pm$0.0522, which is significantly higher than the result provided by the deep convolutional neural network (0.6247$\pm$0.0680) and the discriminative spatial pattern method (0.4400$\pm$0.0700). In the multi-class classification of 7 states, the classification accuracy of SASF is 0.9491$\pm$0.0372. The proposed SASF greatly improves the classification between two motions and enables the classification among multiple motions. The result shows that the movement can be decoded from EEG signals before the actual limb movement.

cs.HC↗

arXiv2022

Deep Safe Multi-Task Learning

In recent years, Multi-Task Learning (MTL) has attracted much attention due to its good performance in many applications. However, many existing MTL models cannot guarantee that their performance is no worse than their single-task counterparts on each task. Though some works have empirically observed this phenomenon, little work aims to handle the resulting problem. In this paper, we formally define this phenomenon as negative sharing and define safe multi-task learning where no negative sharing occurs. To achieve safe multi-task learning, we propose a Deep Safe Multi-Task Learning (DSMTL) model with two learning strategies: individual learning and joint learning. We theoretically study the safeness of both learning strategies in the DSMTL model to show that the proposed methods can achieve some versions of safe multi-task learning. Moreover, to improve the scalability of the DSMTL model, we propose an extension, which automatically learns a compact architecture and empirically achieves safe multi-task learning. Extensive experiments on benchmark datasets verify the safeness of the proposed methods.

cs.LG↗

arXiv2022

Domain Adaptation via Bidirectional Cross-Attention Transformer

Domain Adaptation (DA) aims to leverage the knowledge learned from a source domain with ample labeled data to a target domain with unlabeled data only. Most existing studies on DA contribute to learning domain-invariant feature representations for both domains by minimizing the domain gap based on convolution-based neural networks. Recently, vision transformers significantly improved performance in multiple vision tasks. Built on vision transformers, in this paper we propose a Bidirectional Cross-Attention Transformer (BCAT) for DA with the aim to improve the performance. In the proposed BCAT, the attention mechanism can extract implicit source and target mixup feature representations to narrow the domain discrepancy. Specifically, in BCAT, we design a weight-sharing quadruple-branch transformer with a bidirectional cross-attention mechanism to learn domain-invariant feature representations. Extensive experiments demonstrate that the proposed BCAT model achieves superior performance on four benchmark datasets over existing state-of-the-art DA methods that are based on convolutions or transformers.

cs.CV↗

arXiv2022

On Clustering Trend in Language Evolution Based on Dynamical Behaviors of Multi-Agent Model

Computer model has been extensively adopted to overcome the time limitation of language evolution by transforming language theory into physical modeling mechanism, which helps to explore the general laws of the evolution. In this paper, a multi-agent model is designed to simulate the evolution process of language in human settlements, with the network topology being lattice. The language of each node in the lattice will evolve gradually under the influence of its own fixed evolutionary direction and neighbors. According to the computational experiment results, it is discovered that the state points of languages always converge into several clusters during evolution process, which gives us an insight into language evolution.

eess.SY↗

arXiv2022

Measuring quantum geometric tensor of non-Abelian system in superconducting circuits

Topology played an important role in physics research during the last few decades. In particular, the quantum geometric tensor that provides local information about topological properties has attracted much attention. It will reveal interesting topological properties in non-Abelian systems, which have not been realized in practice. Here, we use a four-qubit quantum system in superconducting circuits to construct a degenerate Hamiltonian with parametric modulation. By manipulating the Hamiltonian with periodic drivings, we simulate the Bernevig-Hughes-Zhang model and obtain the quantum geometric tensor from interference oscillation. In addition, we reveal its topological feature by extracting the topological invariant, demonstrating an effective protocol for quantum simulation of a non-Abelian system.

quant-ph↗