arXiv subjects

Hai Li

Publications and source records attributed to Hai Li.

At least 109 records · Page 6Linked to original sources

arXiv2022

Preserving background sound in noise-robust voice conversion via multi-task learning

Background sound is an informative form of art that is helpful in providing a more immersive experience in real-application voice conversion (VC) scenarios. However, prior research about VC, mainly focusing on clean voices, pay rare attention to VC with background sound. The critical problem for preserving background sound in VC is inevitable speech distortion by the neural separation model and the cascade mismatch between the source separation model and the VC model. In this paper, we propose an end-to-end framework via multi-task learning which sequentially cascades a source separation (SS) module, a bottleneck feature extraction module and a VC module. Specifically, the source separation task explicitly considers critical phase information and confines the distortion caused by the imperfect separation process. The source separation task, the typical VC task and the unified task shares a uniform reconstruction loss constrained by joint training to reduce the mismatch between the SS and VC modules. Experimental results demonstrate that our proposed framework significantly outperforms the baseline systems while achieving comparable quality and speaker similarity to the VC models trained with clean data.

eess.AS↗

arXiv2022

Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environments

Many real-world scenarios in which DNN-based recognition systems are deployed have inherently fine-grained attributes (e.g., bird-species recognition, medical image classification). In addition to achieving reliable accuracy, a critical subtask for these models is to detect Out-of-distribution (OOD) inputs. Given the nature of the deployment environment, one may expect such OOD inputs to also be fine-grained w.r.t. the known classes (e.g., a novel bird species), which are thus extremely difficult to identify. Unfortunately, OOD detection in fine-grained scenarios remains largely underexplored. In this work, we aim to fill this gap by first carefully constructing four large-scale fine-grained test environments, in which existing methods are shown to have difficulties. Particularly, we find that even explicitly incorporating a diverse set of auxiliary outlier data during training does not provide sufficient coverage over the broad region where fine-grained OOD samples locate. We then propose Mixture Outlier Exposure (MixOE), which mixes ID data and training outliers to expand the coverage of different OOD granularities, and trains the model such that the prediction confidence linearly decays as the input transitions from ID to OOD. Extensive experiments and analyses demonstrate the effectiveness of MixOE for building up OOD detector in fine-grained environments. The code is available at https://github.com/zjysteven/MixOE.

cs.LG↗

arXiv2022

All-electrical spin-to-charge conversion in sputtered Bi$_x$Se$_{1-x}$

One of the major obstacles to realizing spintronic devices such as MESO logic devices is the small signal magnitude used for magnetization readout, making it important to find materials with high spin-to-charge conversion efficiency. Although intermixing at the junction of two materials is a widely occurring phenomenon, its influence on material characterization and the estimation of spin-to-charge conversion efficiencies is easily neglected or underestimated. Here, we demonstrate all electrical spin-to-charge conversion in Bi$_x$Se$_{1-x}$ nanodevices and show how the conversion efficiency can be overestimated by tens of times depending on the adjacent metal used as a contact. We attribute this to the intermixing-induced compositional change and the properties of a polycrystal that lead to drastic changes in resistivity and spin Hall angle. Strategies to improve the spin-to-charge conversion signal in similar structures for functional devices are discussed.

cond-mat.mes-hall↗

arXiv2022

Approximate Computing and the Efficient Machine Learning Expedition

Approximate computing (AxC) has been long accepted as a design alternative for efficient system implementation at the cost of relaxed accuracy requirements. Despite the AxC research activities in various application domains, AxC thrived the past decade when it was applied in Machine Learning (ML). The by definition approximate notion of ML models but also the increased computational overheads associated with ML applications-that were effectively mitigated by corresponding approximations-led to a perfect matching and a fruitful synergy. AxC for AI/ML has transcended beyond academic prototypes. In this work, we enlighten the synergistic nature of AxC and ML and elucidate the impact of AxC in designing efficient ML systems. To that end, we present an overview and taxonomy of AxC for ML and use two descriptive application scenarios to demonstrate how AxC boosts the efficiency of ML systems.

cs.AR↗

arXiv2022

NashAE: Disentangling Representations through Adversarial Covariance Minimization

We present a self-supervised method to disentangle factors of variation in high-dimensional data that does not rely on prior knowledge of the underlying variation profile (e.g., no assumptions on the number or distribution of the individual latent variables to be extracted). In this method which we call NashAE, high-dimensional feature disentanglement is accomplished in the low-dimensional latent space of a standard autoencoder (AE) by promoting the discrepancy between each encoding element and information of the element recovered from all other encoding elements. Disentanglement is promoted efficiently by framing this as a minmax game between the AE and an ensemble of regression networks which each provide an estimate of an element conditioned on an observation of all other elements. We quantitatively compare our approach with leading disentanglement methods using existing disentanglement metrics. Furthermore, we show that NashAE has increased reliability and increased capacity to capture salient data characteristics in the learned latent representation.

cs.LG↗

arXiv2022

Fine-grain Inference on Out-of-Distribution Data with Hierarchical Classification

Machine learning methods must be trusted to make appropriate decisions in real-world environments, even when faced with out-of-distribution (OOD) samples. Many current approaches simply aim to detect OOD examples and alert the user when an unrecognized input is given. However, when the OOD sample significantly overlaps with the training data, a binary anomaly detection is not interpretable or explainable, and provides little information to the user. We propose a new model for OOD detection that makes predictions at varying levels of granularity as the inputs become more ambiguous, the model predictions become coarser and more conservative. Consider an animal classifier that encounters an unknown bird species and a car. Both cases are OOD, but the user gains more information if the classifier recognizes that its uncertainty over the particular species is too large and predicts bird instead of detecting it as OOD. Furthermore, we diagnose the classifiers performance at each level of the hierarchy improving the explainability and interpretability of the models predictions. We demonstrate the effectiveness of hierarchical classifiers for both fine- and coarse-grained OOD tasks.

cs.LG↗

arXiv2022

Joint Optimization of STAR-RIS Assisted UAV Communication Systems

In this letter, we study the simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted unmanned aerial vehicle (UAV) communications. Our goal is to maximize the sum rate of all users by jointly optimizing the STAR-RIS's beamforming vectors, the UAV's trajectory and power allocation. We decompose the formulated non-convex problem into three subproblems and solve them alternately to obtain the solution. Simulations show that: 1) the STAR-RIS achieves a higher sum rate than traditional RIS; 2) to exploit the benefits of STAR-RIS, the UAV's trajectory is closer to STAR-RIS than that of RIS; 3) the energy splitting for reflection and transmission highly depends on the real-time trajectory of UAV.

cs.IT↗

arXiv2022

Learning and Compositionality: a Unification Attempt via Connectionist Probabilistic Programming

We consider learning and compositionality as the key mechanisms towards simulating human-like intelligence. While each mechanism is successfully achieved by neural networks and symbolic AIs, respectively, it is the combination of the two mechanisms that makes human-like intelligence possible. Despite the numerous attempts on building hybrid neuralsymbolic systems, we argue that our true goal should be unifying learning and compositionality, the core mechanisms, instead of neural and symbolic methods, the surface approaches to achieve them. In this work, we review and analyze the strengths and weaknesses of neural and symbolic methods by separating their forms and meanings (structures and semantics), and propose Connectionist Probabilistic Program (CPPs), a framework that connects connectionist structures (for learning) and probabilistic program semantics (for compositionality). Under the framework, we design a CPP extension for small scale sequence modeling and provide a learning algorithm based on Bayesian inference. Although challenges exist in learning complex patterns without supervision, our early results demonstrate CPP's successful extraction of concepts and relations from raw sequential data, an initial step towards compositional learning.

cs.AI↗

arXiv2022

Nonvolatile Electric Field Control of Thermal Magnons in the Absence of an Applied Magnetic Field

Spin transport through magnetic insulators has been demonstrated in a variety of materials and is an emerging pathway for next-generation spin-based computing. To modulate spin transport in these systems, one typically applies a sufficiently strong magnetic field to allow for deterministic control of magnetic order. Here, we make use of the well-known multiferroic magnetoelectric, BiFeO3, to demonstrate non-volatile, hysteretic, electric-field control of thermally excited magnon current in the absence of an applied magnetic field. These findings are an important step toward magnon-based devices, where electric-field-only control is highly desirable.

cond-mat.mtrl-sci↗

arXiv2022

Spin Torque Generated by Valley Hall Effect in WSe2

Monolayer transition metal dichalcogenides are promising materials for spintronics due to their robust spin-valley locked valence states, enabling efficient charge-to-spin conversion via valley Hall effect with non-equilibrium spins possessing long spin diffusion lengths of hundreds of nanometers. In this work, we show that the injection of a pure valley current, induced by valley Hall effect in a WSe2 monolayer, imparts a spin torque on the magnetization of an overlaid Fe or CoFe in a tunneling structure. The torque efficiency is found to be comparable to that in conventional perpendicular magnetic tunnel junctions and can be further optimized with valley Hall angle in WSe2. The valley nature of the spin torque gives rise to out-of-plane damping-like torques in a current-in-plane configuration, vanishing charge transport perpendicular-to-the-plane as well as torque efficiency tunable through gating.

cond-mat.mes-hall↗

arXiv2022

Relevance between Information scrambling and quantum Darwinism

Quantum system interacting with environment can induce redundant encoding of the information of system into a multipartite environment, which is the essence of quantum Darwinism. At the same time, environment may scramble the initially localized information about the system. We mainly investigate the relevance between information scrambling in environment and the emergence of quantum Darwinism. First, we generally identify that when the system shows a Darwinistic behavior system information that is initially localized in the environment is not scrambled, while when Darwinism disappears scrambling occurs.We then verify our result through a collision model where the system, consisting of one or two qubits, interacts with an ensemble of environmental ancillas.Moreover, dependent on the nature of system-environment interactions, our results also shows that the single qubit and two-qubit systems behave differently for the emergence of QD and the scrambling, but the above relevance between them remains valid.

quant-ph↗

arXiv2022

Quantum information scrambling in non-Markovian open quantum systems

In this paper we investigate the dynamics of a spin chain whose two end spins interact with two independent non-Markovian baths by using the non-Markovian quantum state diffusion (QSD) equation approach. Specifically two issues about quantum information scrambling in open quantum system are addressed. The first issue is that tripartite mutual information (TMI) can quantify information scrambling properly via its negative value in closed system, whether it is still suitable to indicate quantum scrambling in open quantum system. However we find that negative TMI is not an suitable quantifier of information scrambling in open quantum system in some cases while negative tripartite logarithmic negativity (TLN) is more appropriate. The second one is that up to now almost all the open quantum system effects on information scrambling reported were focus on the Markovian environment, while the effect of non-Markovian environment on information scrambling is still elusive. Significantly our results show that the memory effect of environment will be beneficial to the emergence of quantum information scrambling. Moreover, it is found that environment is generally detrimental for information scrambling in a long time, while in some cases it will be helpful for information scrambling in a short time.

quant-ph↗

arXiv2022

First-principles Landau-like potential for BiFeO$_3$ and related materials

In this work we introduce the simplest, lowest-order Landau-like potential for BiFeO$_3$ and La-doped BiFeO$_3$ as an expansion around the paraelectric cubic phase in powers of polarization, FeO$_6$ octahedral rotations and strains. We present an analytical approach for computing the model parameters from density functional theory. We illustrate our approach by computing the potentials for BiFeO$_3$ and La$_{0.25}$Bi$_{0.75}$FeO$_3$ and show that, overall, we are able to capture the first-principles results accurately. The computed models allow us to identify and explain the main interactions controlling the relative stability of the competing low-energy phases of these compounds.

cond-mat.mtrl-sci↗

arXiv2022

FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learning

Client-wise data heterogeneity is one of the major issues that hinder effective training in federated learning (FL). Since the data distribution on each client may vary dramatically, the client selection strategy can significantly influence the convergence rate of the FL process. Active client selection strategies are popularly proposed in recent studies. However, they neglect the loss correlations between the clients and achieve only marginal improvement compared to the uniform selection strategy. In this work, we propose FedCor -- an FL framework built on a correlation-based client selection strategy, to boost the convergence rate of FL. Specifically, we first model the loss correlations between the clients with a Gaussian Process (GP). Based on the GP model, we derive a client selection strategy with a significant reduction of expected global loss in each round. Besides, we develop an efficient GP training method with a low communication overhead in the FL scenario by utilizing the covariance stationarity. Our experimental results show that compared to the state-of-the-art method, FedCorr can improve the convergence rates by $34\%\sim 99\%$ and $26\%\sim 51\%$ on FMNIST and CIFAR-10, respectively.

cs.LG↗

arXiv2022

ReaLPrune: ReRAM Crossbar-aware Lottery Ticket Pruned CNNs

Training machine learning (ML) models at the edge (on-chip training on end user devices) can address many pressing challenges including data privacy/security, increase the accessibility of ML applications to different parts of the world by reducing the dependence on the communication fabric and the cloud infrastructure, and meet the real-time requirements of AR/VR applications. However, existing edge platforms do not have sufficient computing capabilities to support complex ML tasks such as training large CNNs. ReRAM-based architectures offer high-performance yet energy efficient computing platforms for on-chip CNN training/inferencing. However, ReRAM-based architectures are not scalable with the size of the CNN. Larger CNNs have more weights, which requires more ReRAM cells that cannot be integrated in a single chip. Moreover, training larger CNNs on-chip will require higher power, which cannot be afforded by these smaller devices. Pruning is an effective way to solve this problem. However, existing pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. This leads to sub-optimal hardware savings and performance benefits for CNN training on ReRAM-based architectures. In this paper, we address this problem by proposing a novel crossbar-aware pruning strategy, referred as ReaLPrune, which can prune more than 90% of CNN weights. The pruned model can be trained from scratch without any accuracy loss. Experimental results indicate that ReaLPrune reduces hardware requirements by 77.2% and accelerates CNN training by ~20X compared to unpruned CNNs. ReaLPrune also outperforms other crossbar-aware pruning techniques in terms of both performance and hardware savings. In addition, ReaLPrune is equally effective for diverse datasets and more complex CNNs

cs.AR↗

arXiv2022

Field free switching through bulk spin-orbit torque in L10-FePt films deposited on vicinal substrates

L10-FePt distinguishes itself for its ultrahigh perpendicular magnetic anisotropy (PMA), which enables memory cells with sufficient thermal stability to scale down to 3 nm. The recently discovered "bulk" spin-orbit torques in L10-FePt provide an efficient and scalable way to manipulate the L10-FePt magnetization. However, the existence of external field during the switching limits its practical application, and therefore field-free switching of the L10-FePt is in highly demand. In this manuscript, we demonstrate the field-free switching of the L10-FePt by growing it on vicinal MgO (001) substrates. This method is different from previously established strategies, as it does not need to add other functional layers or create asymmetry in the film structure. We demonstrate the field-free switching is robust and can withstand strong field disturbance up to ~1 kOe. The dependence on vicinal angle, film thickness, and growth temperature demonstrated a wide operation window for the field-free switching of the L10-FePt. We confirmed that the physical origin of the field-free switching is the vicinal surface-induced the tilted anisotropy of L10-FePt. We quantitatively characterize the spin-orbit torques in the L10-FePt films, and found the spin-orbit torques are not significantly influenced by the lattice strain from vicinal substrates. Our results extend beyond the established strategies to realize field-free switching, and potentially could be applied to other magnetic and antiferromagnetic systems.

cond-mat.mtrl-sci↗

arXiv2022

Privacy Leakage of Adversarial Training Models in Federated Learning Systems

Adversarial Training (AT) is crucial for obtaining deep neural networks that are robust to adversarial attacks, yet recent works found that it could also make models more vulnerable to privacy attacks. In this work, we further reveal this unsettling property of AT by designing a novel privacy attack that is practically applicable to the privacy-sensitive Federated Learning (FL) systems. Using our method, the attacker can exploit AT models in the FL system to accurately reconstruct users' private training images even when the training batch size is large. Code is available at https://github.com/zjysteven/PrivayAttack_AT_FL.

cs.LG↗

arXiv2022

IQDUBBING: Prosody modeling based on discrete self-supervised speech representation for expressive voice conversion

Prosody modeling is important, but still challenging in expressive voice conversion. As prosody is difficult to model, and other factors, e.g., speaker, environment and content, which are entangled with prosody in speech, should be removed in prosody modeling. In this paper, we present IQDubbing to solve this problem for expressive voice conversion. To model prosody, we leverage the recent advances in discrete self-supervised speech representation (DSSR). Specifically, prosody vector is first extracted from pre-trained VQ-Wav2Vec model, where rich prosody information is embedded while most speaker and environment information are removed effectively by quantization. To further filter out the redundant information except prosody, such as content and partial speaker information, we propose two kinds of prosody filters to sample prosody from the prosody vector. Experiments show that IQDubbing is superior to baseline and comparison systems in terms of speech quality while maintaining prosody consistency and speaker similarity.

eess.AS↗