Search arXiv⌕ Search

arXiv subjects

Xiang Li

Publications and source records attributed to Xiang Li.

At least 937 records · Page 52Linked to original sources
arXiv2023

Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Model

Expressive human speech generally abounds with rich and flexible speech prosody variations. The speech prosody predictors in existing expressive speech synthesis methods mostly produce deterministic predictions, which are learned by directly minimizing the norm of prosody prediction error. Its unimodal nature leads to a mismatch with ground truth distribution and harms the model's ability in making diverse predictions. Thus, we propose a novel prosody predictor based on the denoising diffusion probabilistic model to take advantage of its high-quality generative modeling and training stability. Experiment results confirm that the proposed prosody predictor outperforms the deterministic baseline on both the expressiveness and diversity of prediction results with even fewer network parameters.

cs.SD↗
arXiv2023

DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning

Federated Learning (FL) is a privacy-constrained decentralized machine learning paradigm in which clients enable collaborative training without compromising private data. However, how to learn a robust global model in the data-heterogeneous and model-heterogeneous FL scenarios is challenging. To address it, we resort to data-free knowledge distillation to propose a new FL method (namely DFRD). DFRD equips a conditional generator on the server to approximate the training space of the local models uploaded by clients, and systematically investigates its training in terms of fidelity, transferability} and diversity. To overcome the catastrophic forgetting of the global model caused by the distribution shifts of the generator across communication rounds, we maintain an exponential moving average copy of the generator on the server. Additionally, we propose dynamic weighting and label sampling to accurately extract knowledge from local models. Finally, our extensive experiments on various image classification tasks illustrate that DFRD achieves significant performance gains compared to SOTA baselines.

cs.CV↗
arXiv2023

ResolverFuzz: Automated Discovery of DNS Resolver Vulnerabilities with Query-Response Fuzzing

Domain Name System (DNS) is a critical component of the Internet. DNS resolvers, which act as the cache between DNS clients and DNS nameservers, are the central piece of the DNS infrastructure, essential to the scalability of DNS. However, finding the resolver vulnerabilities is non-trivial, and this problem is not well addressed by the existing tools. To list a few reasons, first, most of the known resolver vulnerabilities are non-crash bugs that cannot be directly detected by the existing oracles (or sanitizers). Second, there lacks rigorous specifications to be used as references to classify a test case as a resolver bug. Third, DNS resolvers are stateful, and stateful fuzzing is still challenging due to the large input space. In this paper, we present a new fuzzing system termed ResolverFuzz to address the aforementioned challenges related to DNS resolvers, with a suite of new techniques being developed. First, ResolverFuzz performs constrained stateful fuzzing by focusing on the short query-response sequence, which has been demonstrated as the most effective way to find resolver bugs, based on our study of the published DNS CVEs. Second, to generate test cases that are more likely to trigger resolver bugs, we combine probabilistic context-free grammar (PCFG) based input generation with byte-level mutation for both queries and responses. Third, we leverage differential testing and clustering to identify non-crash bugs like cache poisoning bugs. We evaluated ResolverFuzz against 6 mainstream DNS software under 4 resolver modes. Overall, we identify 23 vulnerabilities that can result in cache poisoning, resource consumption, and crash attacks. After responsible disclosure, 19 of them have been confirmed or fixed, and 15 CVE numbers have been assigned.

cs.CR↗
arXiv2023

Quantum Monte Carlo simulations of thermodynamic properties of attractive SU($3$) Dirac fermions

We employ the determinant quantum Monte Carlo method to study the finite-temperature properties of the half-filled attractive SU($3$) Hubbard model on a honeycomb lattice. We calculate the phase diagram in which the phase boundary separates the disordered phase and the charge-density-wave (CDW) phase and the transition temperature $T_{\text{tr}}(|U|)$ varies non-monotonically with attractive Hubbard interaction $|U|$. As the Hubbard $|U|$ increases at constant temperature $T<\text{max}(T_{\text{tr}}(|U|))$, the system first undergoes a transition from thermal Dirac semimetal phase to CDW phase, and eventually the CDW state is thermally melted at a strong Hubbard $|U|$ where the system enters a trion liquid phase. In between the two transition points the non-monotonic $|U|$ dependence of CDW order strength is strikingly different from the zero-temperature monotonic behavior. In the trion CDW state where off-site trions arise from quantum fluctuations (a fermion inside an on-site trion hops to a nearest-neighbor site), the simulated triple occupancy at constant Hubbard $|U|$ surprisingly increases with temperature, implying that the formation of off-site trions is suppressed by the thermal delocalization of on-site trions. We have also calculated the entropy-temperature relations for various attractive Hubbrad interactions, which exhibit the prominent characteristic of the Pomeranchuk effect. Our work has revealed that the formation of on-site and off-site trions has significant consequences for thermodynamic properties of SU(3) Dirac fermions.

cond-mat.quant-gas↗
arXiv2023

Uncertainty Quantification in Inverse Models in Hydrology

In hydrology, modeling streamflow remains a challenging task due to the limited availability of basin characteristics information such as soil geology and geomorphology. These characteristics may be noisy due to measurement errors or may be missing altogether. To overcome this challenge, we propose a knowledge-guided, probabilistic inverse modeling method for recovering physical characteristics from streamflow and weather data, which are more readily available. We compare our framework with state-of-the-art inverse models for estimating river basin characteristics. We also show that these estimates offer improvement in streamflow modeling as opposed to using the original basin characteristic values. Our inverse model offers 3\% improvement in R$^2$ for the inverse model (basin characteristic estimation) and 6\% for the forward model (streamflow prediction). Our framework also offers improved explainability since it can quantify uncertainty in both the inverse and the forward model. Uncertainty quantification plays a pivotal role in improving the explainability of machine learning models by providing additional insights into the reliability and limitations of model predictions. In our analysis, we assess the quality of the uncertainty estimates. Compared to baseline uncertainty quantification methods, our framework offers 10\% improvement in the dispersion of epistemic uncertainty and 13\% improvement in coverage rate. This information can help stakeholders understand the level of uncertainty associated with the predictions and provide a more comprehensive view of the potential outcomes.

cs.LG↗
arXiv2023

MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

The recent GPT-4 has demonstrated extraordinary multi-modal abilities, such as directly generating websites from handwritten text and identifying humorous elements within images. These features are rarely observed in previous vision-language models. However, the technical details behind GPT-4 continue to remain undisclosed. We believe that the enhanced multi-modal generation capabilities of GPT-4 stem from the utilization of sophisticated large language models (LLM). To examine this phenomenon, we present MiniGPT-4, which aligns a frozen visual encoder with a frozen advanced LLM, Vicuna, using one projection layer. Our work, for the first time, uncovers that properly aligning the visual features with an advanced large language model can possess numerous advanced multi-modal abilities demonstrated by GPT-4, such as detailed image description generation and website creation from hand-drawn drafts. Furthermore, we also observe other emerging capabilities in MiniGPT-4, including writing stories and poems inspired by given images, teaching users how to cook based on food photos, and so on. In our experiment, we found that the model trained on short image caption pairs could produce unnatural language outputs (e.g., repetition and fragmentation). To address this problem, we curate a detailed image description dataset in the second stage to finetune the model, which consequently improves the model's generation reliability and overall usability. Our code, pre-trained model, and collected dataset are available at https://minigpt-4.github.io/.

cs.CV↗
arXiv2023

Unadjusted Langevin Algorithms for SDEs with Hoelder Drift

Consider the following stochastic differential equation for $(X_t)_{t\ge 0}$ on $\mathbb R^d$ and its Euler-Maruyama (EM) approximation $(Y_{t_n})_{n\in \mathbb Z^+}$: \begin{align*} &d X_t=b( X_t) d t+σ(X_t) d B_t, \\ & Y_{t_{n+1}}=Y_{t_{n}}+η_{n+1} b(Y_{t_{n}})+σ(Y_{t_{n}})\left(B_{t_{n+1}}-B_{t_{n}}\right), \end{align*} where $b:\mathbb{R}^d \rightarrow \mathbb{R}^d,\ \ σ: \mathbb R^d \rightarrow \mathbb{R}^{d \times d}$ are measurable, $B_t$ is the $d$-dimensional Brownian motion, $t_0:=0,t_{n}:=\sum_{k=1}^{n} η_{k}$ for constants $η_k>0$ satisfying $\lim_{k \rightarrow \infty} η_k=0$ and $\sum_{k=1}^\inftyη_k =\infty$. Under (partial) dissipation conditions ensuring the ergodicity, we obtain explicit convergence rates of $\mathbb W_p(\mathscr{L}(Y_{t_n}), \mathscr{L}(X_{t_n}))+\mathbb W_p(\mathscr{L}(Y_{t_n}), μ)\rightarrow 0$ as $n\rightarrow \infty$, where $\mathbb W_p$ is the $L^p$-Wasserstein distance for certain $p\in [0,\infty)$, $\mathscr{L}(ξ)$ is the distribution of random variable $ξ$, and $μ$ is the unique invariant probability measure of $(X_t)_{t \ge 0}$. Comparing with the existing results where $b$ is at least $C^2$-smooth, our estimates apply to Hoelder continuous drift and can be sharp in several specific situations.

math.PR↗
arXiv2023

In-Hand Re-grasp Manipulation with Passive Dynamic Actions via Imitation Learning

Re-grasp manipulation leverages on ergonomic tools to assist humans in accomplishing diverse tasks. In certain scenarios, humans often employ external forces to effortlessly and precisely re-grasp tools like a hammer. Previous development on controllers for in-grasp sliding motion using passive dynamic actions (e.g.,gravity) relies on apprehension of finger-object contact information, and requires customized design for individual objects with varied geometry and weight distribution. It limits their adaptability to diverse objects. In this paper, we propose an end-to-end sliding motion controller based on imitation learning (IL) that necessitates minimal prior knowledge of object mechanics, relying solely on object position information. To expedite training convergence, we utilize a data glove to collect expert data trajectories and train the policy through Generative Adversarial Imitation Learning (GAIL). Simulation results demonstrate the controller's versatility in performing in-hand sliding tasks with objects of varying friction coefficients, geometric shapes, and masses. By migrating to a physical system using visual position estimation, the controller demonstrated an average success rate of 86%, surpassing the baseline algorithm's success rate of 35% of Behavior Cloning(BC) and 20% of Proximal Policy Optimization (PPO).

cs.RO↗
arXiv2023

Learning to Assist Different Wearers in Multitasks: Efficient and Individualized Human-In-the-Loop Adaption Framework for Exoskeleton Robots

One of the typical purposes of using lower-limb exoskeleton robots is to provide assistance to the wearer by supporting their weight and augmenting their physical capabilities according to a given task and human motion intentions. The generalizability of robots across different wearers in multiple tasks is important to ensure that the robot can provide correct and effective assistance in actual implementation. However, most lower-limb exoskeleton robots exhibit only limited generalizability. Therefore, this paper proposes a human-in-the-loop learning and adaptation framework for exoskeleton robots to improve their performance in various tasks and for different wearers. To suit different wearers, an individualized walking trajectory is generated online using dynamic movement primitives and Bayes optimization. To accommodate various tasks, a task translator is constructed using a neural network to generalize a trajectory to more complex scenarios. These generalization techniques are integrated into a unified variable impedance model, which regulates the exoskeleton to provide assistance while ensuring safety. In addition, an anomaly detection network is developed to quantitatively evaluate the wearer's comfort, which is considered in the trajectory learning procedure and contributes to the relaxation of conflicts in impedance control. The proposed framework is easy to implement, because it requires proprioceptive sensors only to perform and deploy data-efficient learning schemes. This makes the exoskeleton practical for deployment in complex scenarios, accommodating different walking patterns, habits, tasks, and conflicts. Experiments and comparative studies on a lower-limb exoskeleton robot are performed to demonstrate the effectiveness of the proposed framework.

cs.RO↗
arXiv2023

AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data

Recently, the utilization of extensive open-sourced text data has significantly advanced the performance of text-based large language models (LLMs). However, the use of in-the-wild large-scale speech data in the speech technology community remains constrained. One reason for this limitation is that a considerable amount of the publicly available speech data is compromised by background noise, speech overlapping, lack of speech segmentation information, missing speaker labels, and incomplete transcriptions, which can largely hinder their usefulness. On the other hand, human annotation of speech data is both time-consuming and costly. To address this issue, we introduce an automatic in-the-wild speech data preprocessing framework (AutoPrep) in this paper, which is designed to enhance speech quality, generate speaker labels, and produce transcriptions automatically. The proposed AutoPrep framework comprises six components: speech enhancement, speech segmentation, speaker clustering, target speech extraction, quality filtering and automatic speech recognition. Experiments conducted on the open-sourced WenetSpeech and our self-collected AutoPrepWild corpora demonstrate that the proposed AutoPrep framework can generate preprocessed data with similar DNSMOS and PDNSMOS scores compared to several open-sourced TTS datasets. The corresponding TTS system can achieve up to 0.68 in-domain speaker similarity.

eess.AS↗
arXiv2023

Assessing the Dissipative Capacity of Particle Impact Dampers Based on their Nonlinear Bandwidth Characteristics

The dissipative capacity as quantified by the nonlinear bandwidth measure of impulsively loaded primary structures (PSs) coupled to particle impact dampers (PIDs) is assessed. The considered PIDs are designed by initially placing different numbers of spherical, linearly viscoelastic granules at different 2D initial topologies and clearances. The strongly nonlinear and highly discontinuous dynamics of the PIDs are simulated via the discrete element method taking Hertzian interactions, slipping friction and granular rotations into account. The general definition of nonlinear bandwidth is used to evaluate the energy dissipation capacity of the integrated PS-PID systems. Moreover, the effect of the dynamics of the PIDs on the time-bandwidth product of these systems is studied, as a measure of their capacity to store or dissipate vibration energy. It is found that the initial topologies of the granules in the PID drastically affect the time-bandwidth product, which, depending on shock intensity, may break the classical limit of unity which holds for linear time-invariant dissipative resonators. The optimal PS-PID systems composed of multiple granules produce large nonlinear bandwidths, indicating strong dissipative capacity of broadband input energy by the PIDs. Additionally, in the optimal configurations, the time-bandwidth product, i.e., the measure of the frequency bandwidth of the input shock that is stored in the PS-PID system, in tandem with the amount of time it takes for the system to dissipate (1/e) of the initial energy, can be tuned either above or below unity by varying the applied shock intensity. The implications of these findings on the dissipative capacity of the system considered are discussed, showing that it can be predictively assessed so that PIDs can act as highly effective nonlinear energy sinks capable of rapid and efficient suppression of vibration induced by shocks.

math.OC↗
arXiv2023

Asca: less audio data is more insightful

Audio recognition in specialized areas such as birdsong and submarine acoustics faces challenges in large-scale pre-training due to the limitations in available samples imposed by sampling environments and specificity requirements. While the Transformer model excels in audio recognition, its dependence on vast amounts of data becomes restrictive in resource-limited settings. Addressing this, we introduce the Audio Spectrogram Convolution Attention (ASCA) based on CoAtNet, integrating a Transformer-convolution hybrid architecture, novel network design, and attention techniques, further augmented with data enhancement and regularization strategies. On the BirdCLEF2023 and AudioSet(Balanced), ASCA achieved accuracies of 81.2% and 35.1%, respectively, significantly outperforming competing methods. The unique structure of our model enriches output, enabling generalization across various audio detection tasks. Our code can be found at https://github.com/LeeCiang/ASCA.

cs.SD↗
arXiv2023

Realization of Causal Representation Learning to Adjust Confounding Bias in Latent Space

Causal DAGs(Directed Acyclic Graphs) are usually considered in a 2D plane. Edges indicate causal effects' directions and imply their corresponding time-passings. Due to the natural restriction of statistical models, effect estimation is usually approximated by averaging the individuals' correlations, i.e., observational changes over a specific time. However, in the context of Machine Learning on large-scale questions with complex DAGs, such slight biases can snowball to distort global models - More importantly, it has practically impeded the development of AI, for instance, the weak generalizability of causal models. In this paper, we redefine causal DAG as \emph{do-DAG}, in which variables' values are no longer time-stamp-dependent, and timelines can be seen as axes. By geometric explanation of multi-dimensional do-DAG, we identify the \emph{Causal Representation Bias} and its necessary factors, differentiated from common confounding biases. Accordingly, a DL(Deep Learning)-based framework will be proposed as the general solution, along with a realization method and experiments to verify its feasibility.

cs.LG↗
arXiv2023

A Discourse-level Multi-scale Prosodic Model for Fine-grained Emotion Analysis

This paper explores predicting suitable prosodic features for fine-grained emotion analysis from the discourse-level text. To obtain fine-grained emotional prosodic features as predictive values for our model, we extract a phoneme-level Local Prosody Embedding sequence (LPEs) and a Global Style Embedding as prosodic speech features from the speech with the help of a style transfer model. We propose a Discourse-level Multi-scale text Prosodic Model (D-MPM) that exploits multi-scale text to predict these two prosodic features. The proposed model can be used to analyze better emotional prosodic features and thus guide the speech synthesis model to synthesize more expressive speech. To quantitatively evaluate the proposed model, we contribute a new and large-scale Discourse-level Chinese Audiobook (DCA) dataset with more than 13,000 utterances annotated sequences to evaluate the proposed model. Experimental results on the DCA dataset show that the multi-scale text information effectively helps to predict prosodic features, and the discourse-level text improves both the overall coherence and the user experience. More interestingly, although we aim at the synthesis effect of the style transfer model, the synthesized speech by the proposed text prosodic analysis model is even better than the style transfer from the original speech in some user evaluation indicators.

cs.SD↗
arXiv2023

Realization of a minimal Kitaev chain in coupled quantum dots

Majorana bound states constitute one of the simplest examples of emergent non-Abelian excitations in condensed matter physics. A toy model proposed by Kitaev shows that such states can arise at the ends of a spinless $p$-wave superconducting chain. Practical proposals for its realization require coupling neighboring quantum dots in a chain via both electron tunneling and crossed Andreev reflection. While both processes have been observed in semiconducting nanowires and carbon nanotubes, crossed-Andreev interaction was neither easily tunable nor strong enough to induce coherent hybridization of dot states. Here we demonstrate the simultaneous presence of all necessary ingredients for an artificial Kitaev chain: two spin-polarized quantum dots in an InSb nanowire strongly coupled by both elastic co-tunneling and crossed Andreev reflection. We fine-tune this system to a sweet spot where a pair of Poor Man's Majorana states is predicted to appear. At this sweet spot, the transport characteristics satisfy the theoretical predictions for such a system, including pairwise correlation, zero charge and stability against local perturbations. While the simple system presented here can be scaled to simulate a full Kitaev chain with an emergent topological order, it can also be used imminently to explore relevant physics related to non-Abelian anyons.

cond-mat.mes-hall↗
arXiv2023

Fused Spectatorship: Designing Bodily Experiences Where Spectators Become Players

Spectating digital games can be exciting. However, due to its vicarious nature, spectators often wish to engage in the gameplay beyond just watching and cheering. To blur the boundaries between spectators and players, we propose a novel approach called "Fused Spectatorship", where spectators watch their hands play games by loaning bodily control to a computational Electrical Muscle Stimulation (EMS) system. To showcase this concept, we designed three games where spectators loan control over both their hands to the EMS system and watch them play these competitive and collaborative games. A study with 12 participants suggested that participants could not distinguish if they were watching their hands play, or if they were playing the games themselves. We used our results to articulate four spectator experience themes and four fused spectator types, the behaviours they elicited and offer one design consideration to support each of these behaviours. We also discuss the ethical design considerations of our approach to help game designers create future fused spectatorship experiences.

cs.HC↗
arXiv2023

Auto-Paízo Games: Towards Understanding the Design of Games that Aim to Unify a Player's Physical Body and the Virtual World

Most digital bodily games focus on the body as they use movement as input. However, they also draw the player's focus away from the body as the output occurs on visual displays, creating a divide between the physical body and the virtual world. We propose a novel approach - the "Body as a Play Material" - where a player uses their body as both input and output to unify the physical body and the virtual world. To showcase this approach, we designed three games where a player uses one of their hands (input) to play against the other hand (output) by loaning control over its movements to an Electrical Muscle Stimulation (EMS) system. We conducted a thematic analysis on the data obtained from a field study with 12 participants to articulate four player experience themes. We discuss our results about how participants appreciated the engagement with the variety of bodily movements for play and the ambiguity of using their body as a play material. Ultimately, our work aims to unify the physical body and the virtual world.

cs.HC↗
arXiv2023

ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs

ChatGPT, as a recently launched large language model (LLM), has shown superior performance in various natural language processing (NLP) tasks. However, two major limitations hinder its potential applications: (1) the inflexibility of finetuning on downstream tasks and (2) the lack of interpretability in the decision-making process. To tackle these limitations, we propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification, while improving its interpretability. The proposed framework conducts a knowledge graph extraction task to extract refined and structural knowledge from the raw data using ChatGPT. The rich knowledge is then converted into a graph, which is further used to train an interpretable linear classifier to make predictions. To evaluate the effectiveness of our proposed method, we conduct experiments on four datasets. The result shows that our method can significantly improve the performance compared to directly utilizing ChatGPT for text classification tasks. And our method provides a more transparent decision-making process compared with previous text classification methods.

cs.CL↗