arXiv subjects

Chao Zhang

Publications and source records attributed to Chao Zhang.

At least 433 records · Page 24Linked to original sources

arXiv2024

SWIM: Short-Window CNN Integrated with Mamba for EEG-Based Auditory Spatial Attention Decoding

In complex auditory environments, the human auditory system possesses the remarkable ability to focus on a specific speaker while disregarding others. In this study, a new model named SWIM, a short-window convolution neural network (CNN) integrated with Mamba, is proposed for identifying the locus of auditory attention (left or right) from electroencephalography (EEG) signals without relying on speech envelopes. SWIM consists of two parts. The first is a short-window CNN (SW$_\text{CNN}$), which acts as a short-term EEG feature extractor and achieves a final accuracy of 84.9% in the leave-one-speaker-out setup on the widely used KUL dataset. This improvement is due to the use of an improved CNN structure, data augmentation, multitask training, and model combination. The second part, Mamba, is a sequence model first applied to auditory spatial attention decoding to leverage the long-term dependency from previous SW$_\text{CNN}$ time steps. By joint training SW$_\text{CNN}$ and Mamba, the proposed SWIM structure uses both short-term and long-term information and achieves an accuracy of 86.2%, which reduces the classification errors by a relative 31.0% compared to the previous state-of-the-art result. The source code is available at https://github.com/windowso/SWIM-ASAD.

eess.AS↗

arXiv2024

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation

Full-duplex multimodal large language models (LLMs) provide a unified framework for addressing diverse speech understanding and generation tasks, enabling more natural and seamless human-machine conversations. Unlike traditional modularised conversational AI systems, which separate speech recognition, understanding, and text-to-speech generation into distinct components, multimodal LLMs operate as single end-to-end models. This streamlined design eliminates error propagation across components and fully leverages the rich non-verbal information embedded in input speech signals. We introduce SALMONN-omni, a codec-free, full-duplex speech understanding and generation model capable of simultaneously listening to its own generated speech and background sounds while speaking. To support this capability, we propose a novel duplex spoken dialogue framework incorporating a ``thinking'' mechanism that facilitates asynchronous text and speech generation relying on embeddings instead of codecs (quantized speech and audio tokens). Experimental results demonstrate SALMONN-omni's versatility across a broad range of streaming speech tasks, including speech recognition, speech enhancement, and spoken question answering. Additionally, SALMONN-omni excels at managing turn-taking, barge-in, and echo cancellation scenarios, establishing its potential as a robust prototype for full-duplex conversational AI systems. To the best of our knowledge, SALMONN-omni is the first codec-free model of its kind. A full technical report along with model checkpoints will be released soon.

eess.AS↗

arXiv2024

Generative AI for RF Sensing in IoT systems

The development of wireless sensing technologies, using signals such as Wi-Fi, infrared, and RF to gather environmental data, has significantly advanced within Internet of Things (IoT) systems. Among these, Radio Frequency (RF) sensing stands out for its cost-effective and non-intrusive monitoring of human activities and environmental changes. However, traditional RF sensing methods face significant challenges, including noise, interference, incomplete data, and high deployment costs, which limit their effectiveness and scalability. This paper investigates the potential of Generative AI (GenAI) to overcome these limitations within the IoT ecosystem. We provide a comprehensive review of state-of-the-art GenAI techniques, focusing on their application to RF sensing problems. By generating high-quality synthetic data, enhancing signal quality, and integrating multi-modal data, GenAI offers robust solutions for RF environment reconstruction, localization, and imaging. Additionally, GenAI's ability to generalize enables IoT devices to adapt to new environments and unseen tasks, improving their efficiency and performance. The main contributions of this article include a detailed analysis of the challenges in RF sensing, the presentation of innovative GenAI-based solutions, and the proposal of a unified framework for diverse RF sensing tasks. Through case studies, we demonstrate the effectiveness of integrating GenAI models, leading to advanced, scalable, and intelligent IoT systems.

eess.SP↗

arXiv2024

ScalingNote: Scaling up Retrievers with Large Language Models for Real-World Dense Retrieval

Dense retrieval in most industries employs dual-tower architectures to retrieve query-relevant documents. Due to online deployment requirements, existing real-world dense retrieval systems mainly enhance performance by designing negative sampling strategies, overlooking the advantages of scaling up. Recently, Large Language Models (LLMs) have exhibited superior performance that can be leveraged for scaling up dense retrieval. However, scaling up retrieval models significantly increases online query latency. To address this challenge, we propose ScalingNote, a two-stage method to exploit the scaling potential of LLMs for retrieval while maintaining online query latency. The first stage is training dual towers, both initialized from the same LLM, to unlock the potential of LLMs for dense retrieval. Then, we distill only the query tower using mean squared error loss and cosine similarity to reduce online costs. Through theoretical analysis and comprehensive offline and online experiments, we show the effectiveness and efficiency of ScalingNote. Our two-stage scaling method outperforms end-to-end models and verifies the scaling law of dense retrieval with LLMs in industrial scenarios, enabling cost-effective scaling of dense retrieval systems. Our online method incorporating ScalingNote significantly enhances the relevance between retrieved documents and queries.

cs.IR↗

arXiv2024

Detecting the massive vector field with extreme mass-ratio inspirals

The future space-borne gravitational wave detector, Laser Interferometer Space Antenna (LISA), has the potential of detecting the fundamental fields, such as the charge and mass of ultra-light scalar field. In this paper we study the effect of lighter vector field on the gravitational waveforms from extreme mass-ratio inspirals (EMRI) system, consisting of a stellar-mass object and the massive black hole (MBH) in the Einstein-Proca theory of a massive vector field coupling to gravity. Using the perturbation theory, we compute the energy fluxes including the contributions of the Proca field and the gravitational field, then obtain the adiabatic inspiraling orbits and corresponding waveforms. Our results demonstrate that the vector charge and mass carried by the secondary body lead to detectable effects on EMRI waveform, and LISA has the potential to measure the mass of the Proca field with greater precision.

gr-qc↗

arXiv2024

Probing new fundamental fields with Extreme Mass Ratio Inspirals

We examine extreme mass ratio inspirals (EMRIs), where a charged compact object spirals into a supermassive black hole, in modified gravity theories with additional scalar or vector fields. Using the Teukolsky and generalized Sasaki-Nakamura formalisms, we provide the post-Newtonian expansion of the energy flux of the vector waves up to $O(v^5)$ beyond the quadrupole formula in the weak field and numerically calculate the energy flux in the strong field for a charged particle moving in circular orbits. Our findings reveal a degeneracy in the scalar and vector charge parameters for weak-field, slow-motion orbits. However, for strong-field, fast-motion orbits close to the innermost stable circular orbit, we observe distinct behaviors between scalar and vector fields. We investigate the potential of using EMRIs detected by space-based gravitational-wave detectors, such as the Laser Interferometer Space Antenna to identify whether a black hole carries a scalar or vector charge. The influence of scalar and vector flux on the orbital evolution and tensor GW phase can not help us distinguish scalar and vector fields. However, extra polarizations emitted by the scalar or vector field can break the correlations between the scalar field and vector field and then help us distinguish the scalar and vector field.

gr-qc↗

arXiv2024

Large Language Models for Power Scheduling: A User-Centric Approach

While traditional optimization and scheduling schemes are designed to meet fixed, predefined system requirements, future systems are moving toward user-driven approaches and personalized services, aiming to achieve high quality-of-experience (QoE) and flexibility. This challenge is particularly pronounced in wireless and digitalized energy networks, where users' requirements have largely not been taken into consideration due to the lack of a common language between users and machines. The emergence of powerful large language models (LLMs) marks a radical departure from traditional system-centric methods into more advanced user-centric approaches by providing a natural communication interface between users and devices. In this paper, for the first time, we introduce a novel architecture for resource scheduling problems by constructing three LLM agents to convert an arbitrary user's voice request (VRQ) into a resource allocation vector. Specifically, we design an LLM intent recognition agent to translate the request into an optimization problem (OP), an LLM OP parameter identification agent, and an LLM OP solving agent. To evaluate system performance, we construct a database of typical VRQs in the context of electric vehicle (EV) charging. As a proof of concept, we primarily use Llama 3 8B. Through testing with different prompt engineering scenarios, the obtained results demonstrate the efficiency of the proposed architecture. The conducted performance analysis allows key insights to be extracted. For instance, having a larger set of candidate OPs to model the real-world problem might degrade the final performance because of a higher recognition/OP classification noise level. All results and codes are open source.

cs.CL↗

arXiv2024

Fast Disentangled Slim Tensor Learning for Multi-view Clustering

Tensor-based multi-view clustering has recently received significant attention due to its exceptional ability to explore cross-view high-order correlations. However, most existing methods still encounter some limitations. (1) Most of them explore the correlations among different affinity matrices, making them unscalable to large-scale data. (2) Although some methods address it by introducing bipartite graphs, they may result in sub-optimal solutions caused by an unstable anchor selection process. (3) They generally ignore the negative impact of latent semantic-unrelated information in each view. To tackle these issues, we propose a new approach termed fast Disentangled Slim Tensor Learning (DSTL) for multi-view clustering . Instead of focusing on the multi-view graph structures, DSTL directly explores the high-order correlations among multi-view latent semantic representations based on matrix factorization. To alleviate the negative influence of feature redundancy, inspired by robust PCA, DSTL disentangles the latent low-dimensional representation into a semantic-unrelated part and a semantic-related part for each view. Subsequently, two slim tensors are constructed with tensor-based regularization. To further enhance the quality of feature disentanglement, the semantic-related representations are aligned across views through a consensus alignment indicator. Our proposed model is computationally efficient and can be solved effectively. Extensive experiments demonstrate the superiority and efficiency of DSTL over state-of-the-art approaches. The code of DSTL is available at https://github.com/dengxu-nju/DSTL.

cs.CV↗

arXiv2024

Data Diversity Matters for Robust Instruction Tuning

Recent works have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. However, creating such datasets is difficult and most works rely on manual curation or proprietary language models. Automatic data curation is difficult as it is still not clear how we can define diversity for instruction tuning, how diversity and quality depend on one other, and how we can optimize dataset quality and diversity. To resolve these issue, we propose a new algorithm, Quality-Diversity Instruction Tuning (QDIT). QDIT provides a simple method to simultaneously control dataset diversity and quality, allowing us to conduct an in-depth study on the effect of diversity and quality on instruction tuning performance. From this study we draw two key insights (1) there is a natural tradeoff between data diversity and quality and (2) increasing data diversity significantly improves the worst case instruction following performance, therefore improving robustness. We validate the performance of QDIT on several large scale instruction tuning datasets, where we find it can substantially improve worst and average case performance compared to quality-driven data selection.

cs.CL↗

arXiv2024

Perturbational Decomposition Analysis for Quantum Ising Model with Weak Transverse Fields

This work presented a perturbational decomposition method for simulating quantum evolution under the one-dimensional Ising model with both longitudinal and transverse fields. By treating the transverse field terms as perturbations in the expansion, our approach is particularly effective in systems with moderate longitudinal fields and weak to moderate transverse fields relative to the coupling strength. Through systematic numerical exploration, we characterized parameter regimes and evolution time windows where the decomposition achieved measurable improvements over conventional Trotter decomposition methods. The developed perturbational approach and its characterized parameter space may provide practical guidance for choosing appropriate simulation strategies in different parameter regimes of the one-dimensional Ising model.

quant-ph↗

arXiv2024

An Improved Empirical Fisher Approximation for Natural Gradient Descent

Approximate Natural Gradient Descent (NGD) methods are an important family of optimisers for deep learning models, which use approximate Fisher information matrices to pre-condition gradients during training. The empirical Fisher (EF) method approximates the Fisher information matrix empirically by reusing the per-sample gradients collected during back-propagation. Despite its ease of implementation, the EF approximation has its theoretical and practical limitations. This paper investigates the inversely-scaled projection issue of EF, which is shown to be a major cause of its poor empirical approximation quality. An improved empirical Fisher (iEF) method is proposed to address this issue, which is motivated as a generalised NGD method from a loss reduction perspective, meanwhile retaining the practical convenience of EF. The exact iEF and EF methods are experimentally evaluated using practical deep learning setups. Optimisation experiments show that applying exact iEF directly as an optimiser provides strong convergence and generalisation. Additionally, under a novel empirical evaluation framework, the proposed iEF method shows consistently better approximation quality to exact Natural Gradient updates than both the EF and the more expensive sampled Fisher methods, meanwhile demonstrating the superior property of being robust to the choice of damping across tasks and training stages. Improving existing approximate NGD optimisers with iEF is expected to lead to better convergence and robustness. Furthermore, the iEF method also serves as a better approximation method to the Fisher information matrix itself, which enables the improvement of a variety of Fisher-based methods, not limited to the scope of optimisation.

cs.LG↗

arXiv2024

Effects of Lanthanides on the Structure and Oxygen Permeability of Ti-doped Dual-phase Membranes

The trade-off effect of the oxygen permeability and stability of oxygen transport membranes (OTMs) still exists in working atmospheres containing CO2. Herein, we reported a new series of 60 wt%Ce0.9Ln0.1O2-δ-40wt%Ln0.6Sr0.4Fe0.9Ti0.1O3-δ (CLnO-LnSFTO, Ln = La, Pr, Nd, Sm, Gd, Tb) dual-phase OTMs by selecting different Ln elements based on the reported highly stable Ti-doped CPrO-PrSFTO. The effects of different Ln elements on the structure and oxygen permeability of Ti-doped dual-phase OTMs were systematically studied. Basically, as the atomic number of Ln elements increases, the unit cell parameters of both the fluorite phase and the perovskite phase become smaller. The unit cell volume and spatial symmetry of the perovskite phase are reduced, resulting in a reduction in oxygen permeability. The optimal CLaO-LaSFTO showed JO2 of 0.60 and 0.54 mL min-1 cm-2 with He and CO2 sweeping at 1000 oC, respectively. In addition, all CLnO-LnSFTO OTMs could work for more than 100 hours with no significant performance degradation in a CO2 atmosphere, maintaining excellent stability. This work explores candidate OTM materials for CO2 capture and oxygen separation, as well as provides some ideas for addressing the trade-off effect.

cond-mat.mtrl-sci↗

arXiv2024

Waveguide-multiplexed photonic matrix-vector multiplication processor using multiport photodetectors

The slowing down of Moore's law has driven the development of application-specific processors for deep learning. Analog photonic processors offer a promising solution for accelerating matrix-vector multiplications (MVMs) in deep learning by leveraging parallel computations in the optical domain. Intensity-based photonic MVM processors, which do not utilize the phase information of light, are appealing due to their simplified operations. However, existing intensity-based schemes for such processors often employ wavelength multiplexing or mode multiplexing, both of which have limited scalability due to high insertion loss or wavelength crosstalk. In this work, we present a scalable intensity-based photonic MVM processor based on the concept of waveguide multiplexing. This scheme employs multiport photodetectors (PDs) to sum the intensities of multiple optical signals, eliminating the need for multiple wavelengths or modes. A 16-port Ge PD with a 3 dB bandwidth of 11.8 GHz at a bias voltage of -3 V is demonstrated, and it can be further scaled up to handle 250 ports while maintaining a 6.1 GHz operation bandwidth. A 4 $\times$ 4 circuit fabricated on a Si-on-insulator (SOI) platform is used to perform MVMs in a 3-layer neural network designed for classifying Iris flowers, achieving a classification accuracy of 93.3%. Furthermore, the performance of large-scale circuits in a convolutional neural network (CNN) for Fashion-MNIST is simulated, resulting in a classification accuracy of 90.53%. This work provides a simplified and scalable approach to photonic MVM, laying a foundation for large-scale and multi-dimensional photonic matrix-matrix multiplication in optical neural networks.

physics.optics↗

arXiv2024

Renormalized and entropy solutions to the general nonlinear parabolic equations in Musielak-Orlicz spaces

We study the well-posedness of solutions to the general nonlinear parabolic equations with merely integrable data in time-dependent Musielak-Orlicz spaces. With the help of a density argument, we establish the existence and uniqueness of both renormalized and entropy solutions. Moreover, we conclude that the entropy and renormalized solutions for this equation are equivalent. Our results cover a variety of problems, including those with Orlicz growth, variable exponents, and double-phase growth.

math.AP↗

arXiv2024

Identifying and bounding the probability of necessity for causes of effects with ordinal outcomes

Although the existing causal inference literature focuses on the forward-looking perspective by estimating effects of causes, the backward-looking perspective can provide insights into causes of effects. In backward-looking causal inference, the probability of necessity measures the probability that a certain event is caused by the treatment given the observed treatment and outcome. Most existing results focus on binary outcomes. Motivated by applications with ordinal outcomes, we propose a general definition of the probability of necessity. However, identifying the probability of necessity is challenging because it involves the joint distribution of the potential outcomes. We propose a novel assumption of monotonic incremental treatment effect to identify the probability of necessity with ordinal outcomes. We also discuss the testable implications of this key identification assumption. When it fails, we derive explicit formulas of the sharp large-sample bounds on the probability of necessity.

math.ST↗

arXiv2024

Aligning Large Language Models with Representation Editing: A Control Perspective

Aligning large language models (LLMs) with human objectives is crucial for real-world applications. However, fine-tuning LLMs for alignment often suffers from unstable training and requires substantial computing resources. Test-time alignment techniques, such as prompting and guided decoding, do not modify the underlying model, and their performance remains dependent on the original model's capabilities. To address these challenges, we propose aligning LLMs through representation editing. The core of our method is to view a pre-trained autoregressive LLM as a discrete-time stochastic dynamical system. To achieve alignment for specific objectives, we introduce external control signals into the state space of this language dynamical system. We train a value function directly on the hidden states according to the Bellman equation, enabling gradient-based optimization to obtain the optimal control signals at test time. Our experiments demonstrate that our method outperforms existing test-time alignment techniques while requiring significantly fewer resources compared to fine-tuning methods. Our code is available at https://github.com/Lingkai-Kong/RE-Control.

cs.AI↗

arXiv2024

Large Language Models for Generative Information Extraction: A Survey

Information extraction (IE) aims to extract structural knowledge from plain natural language texts. Recently, generative Large Language Models (LLMs) have demonstrated remarkable capabilities in text understanding and generation. As a result, numerous works have been proposed to integrate LLMs for IE tasks based on a generative paradigm. To conduct a comprehensive systematic review and exploration of LLM efforts for IE tasks, in this study, we survey the most recent advancements in this field. We first present an extensive overview by categorizing these works in terms of various IE subtasks and techniques, and then we empirically analyze the most advanced methods and discover the emerging trend of IE tasks with LLMs. Based on a thorough review conducted, we identify several insights in technique and promising research directions that deserve further exploration in future studies. We maintain a public repository and consistently update related works and resources on GitHub (\href{https://github.com/quqxui/Awesome-LLM4IE-Papers}{LLM4IE repository})

cs.CL↗

arXiv2024

Even-parity stability of hairy black holes in $U(1)$ gauge-invariant scalar-vector-tensor theories

The $U(1)$ gauge-invariant scalar-vector-tensor theories, which catches five degrees of freedom, are valuable for its implications to inflation problems, generation of primordial magnetic fields, new black hole (BH) and neutron star solutions, etc. In this paper, we derive conditions for the absence of ghosts and Laplacian instabilities of nontrivial BH solutions dressed with scalar hair against both odd- and even-parity perturbations on top of the static and spherically symmetric background in the most general $U(1)$ gauge-invariant scalar-vector-tensor theories with second-order equations of motion. In addition to some general discussions, several typical concrete models are investigated. Specially, we show that the stability against even-parity perturbations is ensured outside the event horizon under certain constraints to these models. This is a crucial step to check the self-consistency of the theories and to shed light on the physically accessible models of such theories for future studies.

gr-qc↗