arXiv subjects

Xin Wang

Publications and source records attributed to Xin Wang.

At least 1,621 records · Page 90Linked to original sources

arXiv2019

Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet

We investigated the training of a shared model for both text-to-speech (TTS) and voice conversion (VC) tasks. We propose using an extended model architecture of Tacotron, that is a multi-source sequence-to-sequence model with a dual attention mechanism as the shared model for both the TTS and VC tasks. This model can accomplish these two different tasks respectively according to the type of input. An end-to-end speech synthesis task is conducted when the model is given text as the input while a sequence-to-sequence voice conversion task is conducted when it is given the speech of a source speaker as the input. Waveform signals are generated by using WaveNet, which is conditioned by using a predicted mel-spectrogram. We propose jointly training a shared model as a decoder for a target speaker that supports multiple sources. Listening experiments show that our proposed multi-source encoder-decoder model can efficiently achieve both the TTS and VC tasks.

eess.AS↗

arXiv2019

Training Multi-Speaker Neural Text-to-Speech Systems using Speaker-Imbalanced Speech Corpora

When the available data of a target speaker is insufficient to train a high quality speaker-dependent neural text-to-speech (TTS) system, we can combine data from multiple speakers and train a multi-speaker TTS model instead. Many studies have shown that neural multi-speaker TTS model trained with a small amount data from multiple speakers combined can generate synthetic speech with better quality and stability than a speaker-dependent one. However when the amount of data from each speaker is highly unbalanced, the best approach to make use of the excessive data remains unknown. Our experiments showed that simply combining all available data from every speaker to train a multi-speaker model produces better than or at least similar performance to its speaker-dependent counterpart. Moreover by using an ensemble multi-speaker model, in which each subsystem is trained on a subset of available data, we can further improve the quality of the synthetic speech especially for underrepresented speakers whose training data is limited.

eess.AS↗

arXiv2019

Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation

Vision-language navigation (VLN) is the task of navigating an embodied agent to carry out natural language instructions inside real 3D environments. In this paper, we study how to address three critical challenges for this task: the cross-modal grounding, the ill-posed feedback, and the generalization problems. First, we propose a novel Reinforced Cross-Modal Matching (RCM) approach that enforces cross-modal grounding both locally and globally via reinforcement learning (RL). Particularly, a matching critic is used to provide an intrinsic reward to encourage global matching between instructions and trajectories, and a reasoning navigator is employed to perform cross-modal grounding in the local visual scene. Evaluation on a VLN benchmark dataset shows that our RCM model significantly outperforms previous methods by 10% on SPL and achieves the new state-of-the-art performance. To improve the generalizability of the learned policy, we further introduce a Self-Supervised Imitation Learning (SIL) method to explore unseen environments by imitating its own past, good decisions. We demonstrate that SIL can approximate a better and more efficient policy, which tremendously minimizes the success rate performance gap between seen and unseen environments (from 30.7% to 11.7%).

cs.CV↗

arXiv2019

Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry

Most previous learning-based visual odometry (VO) methods take VO as a pure tracking problem. In contrast, we present a VO framework by incorporating two additional components called Memory and Refining. The Memory component preserves global information by employing an adaptive and efficient selection strategy. The Refining component ameliorates previous results with the contexts stored in the Memory by adopting a spatial-temporal attention mechanism for feature distilling. Experiments on the KITTI and TUM-RGBD benchmark datasets demonstrate that our method outperforms state-of-the-art learning-based methods by a large margin and produces competitive results against classic monocular VO approaches. Especially, our model achieves outstanding performance in challenging scenarios such as texture-less regions and abrupt motions, where classic VO algorithms tend to fail.

cs.CV↗

arXiv2019

Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation

The overreliance on large parallel corpora significantly limits the applicability of machine translation systems to the majority of language pairs. Back-translation has been dominantly used in previous approaches for unsupervised neural machine translation, where pseudo sentence pairs are generated to train the models with a reconstruction loss. However, the pseudo sentences are usually of low quality as translation errors accumulate during training. To avoid this fundamental issue, we propose an alternative but more effective approach, extract-edit, to extract and then edit real sentences from the target monolingual corpora. Furthermore, we introduce a comparative translation loss to evaluate the translated target sentences and thus train the unsupervised translation systems. Experiments show that the proposed approach consistently outperforms the previous state-of-the-art unsupervised machine translation systems across two benchmarks (English-French and English-German) and two low-resource language pairs (English-Romanian and English-Russian) by more than 2 (up to 3.63) BLEU points.

cs.CL↗

arXiv2019

Spin-qubit noise spectroscopy from randomized benchmarking by supervised learning

We demonstrate a method to obtain the spectra of $1/f$ noises in spin-qubit devices from randomized benchmarking, assisted by supervised learning. The noise exponent, which indicates the correlation within the noise, is determined by training a double-layer neural network with the ratio between the randomized benchmarking results of pulse sequences that correct noise and not. After the training is completed, the neural network is able to predict the exponent within an absolute error of about 0.05, comparable with existing methods. The noise amplitude is then evaluated by training another neural network with the decaying fidelity of randomized benchmarking results from the uncorrected sequences. The relative error for the prediction of the noise amplitude is as low as 5\% provided that the noise exponent is known. Our results suggest that the neural network is capable of predicting noise spectra from randomized benchmarking, which can be an alternative method to measure noise spectra in spin-qubit devices.

quant-ph↗

arXiv2019

DeepCenterline: a Multi-task Fully Convolutional Network for Centerline Extraction

A novel centerline extraction framework is reported which combines an end-to-end trainable multi-task fully convolutional network (FCN) with a minimal path extractor. The FCN simultaneously computes centerline distance maps and detects branch endpoints. The method generates single-pixel-wide centerlines with no spurious branches. It handles arbitrary tree-structured object with no prior assumption regarding depth of the tree or its bifurcation pattern. It is also robust to substantial scale changes across different parts of the target object and minor imperfections of the object's segmentation mask. To the best of our knowledge, this is the first deep-learning based centerline extraction method that guarantees single-pixel-wide centerline for a complex tree-structured object. The proposed method is validated in coronary artery centerline extraction on a dataset of 620 patients (400 of which used as test set). This application is challenging due to the large number of coronary branches, branch tortuosity, and large variations in length, thickness, shape, etc. The proposed method generates well-positioned centerlines, exhibiting lower number of missing branches and is more robust in the presence of minor imperfections of the object segmentation mask. Compared to a state-of-the-art traditional minimal path approach, our method improves patient-level success rate of centerline extraction from 54.3% to 88.8% according to independent human expert review.

cs.CV↗

arXiv2019

Visual Query Answering by Entity-Attribute Graph Matching and Reasoning

Visual Query Answering (VQA) is of great significance in offering people convenience: one can raise a question for details of objects, or high-level understanding about the scene, over an image. This paper proposes a novel method to address the VQA problem. In contrast to prior works, our method that targets single scene VQA, replies on graph-based techniques and involves reasoning. In a nutshell, our approach is centered on three graphs. The first graph, referred to as inference graph GI , is constructed via learning over labeled data. The other two graphs, referred to as query graph Q and entity-attribute graph GEA, are generated from natural language query Qnl and image Img, that are issued from users, respectively. As GEA often does not take sufficient information to answer Q, we develop techniques to infer missing information of GEA with GI . Based on GEA and Q, we provide techniques to find matches of Q in GEA, as the answer of Qnl in Img. Unlike commonly used VQA methods that are based on end-to-end neural networks, our graph-based method shows well-designed reasoning capability, and thus is highly interpretable. We also create a dataset on soccer match (Soccer-VQA) with rich annotations. The experimental results show that our approach outperforms the state-of-the-art method and has high potential for future investigation.

cs.CV↗

arXiv2019

The redshift dependence of Alcock-Paczynski effect: cosmological constraints from the current and next generation observations

The tomographic Alcock-Paczynski (AP) test is a robust large-scale structure (LSS) measurement that receives little contamination from the redshift space distortion (RSD). It has placed tight cosmological constraints by using small and intermediate clustering scales of the LSS data. However, previous works have neglected the cross-correlation among different redshift bins, which could cause the statistical uncertainty being underestimated by $\sim$20\%. In this work, we further improve this method by including this multi-redshifts full correlation. We apply it to the SDSS DR12 galaxies sample and find out that, for $Λ$CDM, the combination of AP with the Planck+BAO dataset slightly reduces (within 1-$σ$) $Ω_m$ to $0.304\pm0.007$ (68.3\% CL). This then leads to a larger $H_0$ and also mildly affects $Ω_b h^2$, $n_s$ and the derived parameters $z_*$, $r_*$, $z_{re}$ but not $τ$, $A_s$ and $σ_8$. For the flat $w$CDM model, our measurement gives $Ω_m=0.301\pm 0.010$ and $w=-1.090\pm 0.047$, where the additional AP measurement reduces the error budget by $\sim 25\%$. When including more parameters into the analysis, the AP method also improves the constraints on $Ω_k$, $\sum m_μ$, $N_{\rm eff}$ by $20-30\%$. Early universe parameters such as $dn_s/d{\rm ln}k$ and $r$, however, are unaffected. Assuming the dark energy equation of state $w=w_0+w_a \frac{z}{1+z}$, the Planck+BAO+SNIa+$H_0$+AP datasets prefer a dynamical dark energy at $\approx1.5 σ$ CL. Finally, we forecast the cosmological constraints expected from the DESI galaxy survey and find that combining AP with CMB+BAO method would improve the $w_0$-$w_a$ constraint by a factor of $\sim 10$.

astro-ph.CO↗

arXiv2019

Growth strategy of microbes on mixed carbon sources

A classic problem in microbiology is that bacteria display two types of growth behavior when cultured on a mixture of two carbon sources: the two sources are sequentially consumed one after another (diauxie) or they are simultaneously consumed (co-utilization). The search for the molecular mechanism of diauxie led to the discovery of the lac operon. However, questions remain as why microbes would bother to have different strategies of taking up nutrients. Here we show that diauxie versus co-utilization can be understood from the topological features of the metabolic network. A model of optimal allocation of protein resources quantitatively explains why and how the cell makes the choice. In case of co-utilization, the model predicts the percentage of each carbon source in supplying the amino acid pools, which is quantitatively verified by experiments. Our work solves a long-standing puzzle and provides a quantitative framework for the carbon source utilization of microbes.

physics.bio-ph↗

arXiv2019

Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language

End-to-end speech synthesis is a promising approach that directly converts raw text to speech. Although it was shown that Tacotron2 outperforms classical pipeline systems with regards to naturalness in English, its applicability to other languages is still unknown. Japanese could be one of the most difficult languages for which to achieve end-to-end speech synthesis, largely due to its character diversity and pitch accents. Therefore, state-of-the-art systems are still based on a traditional pipeline framework that requires a separate text analyzer and duration model. Towards end-to-end Japanese speech synthesis, we extend Tacotron to systems with self-attention to capture long-term dependencies related to pitch accents and compare their audio quality with classical pipeline systems under various conditions to show their pros and cons. In a large-scale listening test, we investigated the impacts of the presence of accentual-type labels, the use of force or predicted alignments, and acoustic features used as local condition parameters of the Wavenet vocoder. Our results reveal that although the proposed systems still do not match the quality of a top-line pipeline system for Japanese, we show important stepping stones towards end-to-end Japanese speech synthesis.

eess.AS↗

arXiv2019

Joint Base Station Activation and Coordinated Downlink Beamforming for HetNets: Efficient Optimal and Suboptimal Algorithms

In cellular heterogeneous networks (HetNets), a number of distributed base stations cooperatively provide services to multiple mobile users. This paper addresses joint base-station activation and coordinated beamforming design for downlink transmission in HetNets. To this end, a mixed integer program is formulated to optimize the total power consumption of a HetNet. A novel approach based on Benders' decomposition is then put forth to obtain the optimal solution for the problem of interest with guaranteed convergence. Building on our new formulation, a dual-subgradient algorithm is also proposed to find an approximate solution in polynomial time. The proposed approaches can be generalized to more general setups, including robust beamforming designs under channel uncertainty, and coordinated beamforming for multi-cell scenarios.

eess.SP↗

arXiv2019

Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding

Modeling the sequential information of image sequences has been a vital step of various vision tasks and convolutional long short-term memory (ConvLSTM) has demonstrated its superb performance in such spatiotemporal problems. Nevertheless, the hierarchical data structures in a significant amount of tasks (e.g., human body parts and vessel/airway tree in biomedical images) cannot be properly modeled by sequential models. Thus, ConvLSTM is not suitable for tree-structured image data analysis. In order to address these limitations, we present tree-structured ConvLSTM models for tree-structured image analysis tasks which can be trained end-to-end. To demonstrate the effectiveness of the proposed tree-structured ConvLSTM model, we present a tree-structured segmentation framework which consists of a tree-structured ConvLSTM and an attention fully convolutional network (FCN) model. The proposed framework is extensively validated on four large-scale coronary artery datasets. The results demonstrate the effectiveness and efficiency of the proposed method.

cs.CV↗

arXiv2019

Suppression of leakage for a charge quadrupole qubit in triangular geometry

We present a simple yet effective strategy to suppress the leakage in the charge quadrupole qubit system by having the dots in a triangle as opposed to the linear geometry originally conceived. We find that the tunnel coupling between the two outmost dots are amplified in triangular triple dots, which consequently reduces leakage by separating the leaked state and qubit states. We have found that the leakage can be suppressed by as much as five orders of magnitude when the dots form an equilateral triangle, with further improvement of control fidelities possible if composite pulses are applied.

cond-mat.mes-hall↗

arXiv2019

Precoding Design for Energy Efficiency of Multibeam Satellite Communications

Instead of merely improving the spectral efficiency (SE), improving the energy efficiency (EE) is another important concern for multibeam satellite systems, due to the power constraint of satellites. However, so far there has been no detailed work on the precoding design concerning the EE for multibeam satellite. In this work, the EE maximization problem is investigated for multibeam satellite systems under the total power constraint as well as the quality of service (QoS) constraints. Precoding design algorithms based on zero forcing (ZF) and sequential convex approximation (SCA) are presented respectively. In particular, these algorithms are verified by the real measured channel data of multibeam satellite systems. Numerical results show that the precoding algorithm based on SCA outperforms that based on ZF. It is also implied that the EE cannot be always improved by solely increasing the power of the satellite, while reducing the satellite operation power is an effective way for the EE improvement.

eess.SP↗

arXiv2018

Historical shear deformation of rock fractures derived from digital outcrop models and its implications on the development of fracture systems

The initiation and development of fractures in rocks is the key part of many problems from academic to industrial, such as faulting, folding, rock mass engineering, reservoir characterization, etc. Conventional ways of evaluating the fracture historical deformations depend on the geologists' visual interpretation of indicating structures such as fault striations, fault steps, plumose structures, etc. on the fracture surface produced by previous deformations, and hence suffer from problems like subjectivity and the absence of obvious indicating structures. In this study, we propose a quantitative method to derive historical shear deformations of rock fractures from digital outcrop models (DOMs) based on the analysis of effects of fault striations and fault steps on the shear strength parameter of the fracture surface. A theoretical model that combines effects of fault striations, fault steps and isotropic base shear strength is fitted to the shear strength parameter. The amount of fault striations and fault steps and their occurrences are estimated, and the historical shear deformations can be inferred. The validity and the effectiveness of the proposed method was proved by testing it on a constructed fracture surface with idealized striations and a fracture surface with clear fault steps. The application of this method on an example outcrop shows an intuitive idea of how the rock mass was deformed and that the distribution, occurrence and mode of new fractures are strictly controlled by preexisting fractures, and hence emphasizes the importance of preexisting fractures in modeling the development of fracture systems.

physics.geo-ph↗

arXiv2018

Residual Attention based Network for Hand Bone Age Assessment

Computerized automatic methods have been employed to boost the productivity as well as objectiveness of hand bone age assessment. These approaches make predictions according to the whole X-ray images, which include other objects that may introduce distractions. Instead, our framework is inspired by the clinical workflow (Tanner-Whitehouse) of hand bone age assessment, which focuses on the key components of the hand. The proposed framework is composed of two components: a Mask R-CNN subnet of pixelwise hand segmentation and a residual attention network for hand bone age assessment. The Mask R-CNN subnet segments the hands from X-ray images to avoid the distractions of other objects (e.g., X-ray tags). The hierarchical attention components of the residual attention subnet force our network to focus on the key components of the X-ray images and generate the final predictions as well as the associated visual supports, which is similar to the assessment procedure of clinicians. We evaluate the performance of the proposed pipeline on the RSNA pediatric bone age dataset and the results demonstrate its superiority over the previous methods.

cs.CV↗

arXiv2018

Explanatory Graphs for CNNs

This paper introduces a graphical model, namely an explanatory graph, which reveals the knowledge hierarchy hidden inside conv-layers of a pre-trained CNN. Each filter in a conv-layer of a CNN for object classification usually represents a mixture of object parts. We develop a simple yet effective method to disentangle object-part pattern components from each filter. We construct an explanatory graph to organize the mined part patterns, where a node represents a part pattern, and each edge encodes co-activation relationships and spatial relationships between patterns. More crucially, given a pre-trained CNN, the explanatory graph is learned without a need of annotating object parts. Experiments show that each graph node consistently represented the same object part through different images, which boosted the transferability of CNN features. We transferred part patterns in the explanatory graph to the task of part localization, and our method significantly outperformed other approaches.

cs.CV↗