arXiv subjects

Cheng Zhang

Publications and source records attributed to Cheng Zhang.

At least 289 records · Page 16Linked to original sources

arXiv2024

EyeEcho: Continuous and Low-power Facial Expression Tracking on Glasses

In this paper, we introduce EyeEcho, a minimally-obtrusive acoustic sensing system designed to enable glasses to continuously monitor facial expressions. It utilizes two pairs of speakers and microphones mounted on glasses, to emit encoded inaudible acoustic signals directed towards the face, capturing subtle skin deformations associated with facial expressions. The reflected signals are processed through a customized machine-learning pipeline to estimate full facial movements. EyeEcho samples at 83.3 Hz with a relatively low power consumption of 167 mW. Our user study involving 12 participants demonstrates that, with just four minutes of training data, EyeEcho achieves highly accurate tracking performance across different real-world scenarios, including sitting, walking, and after remounting the devices. Additionally, a semi-in-the-wild study involving 10 participants further validates EyeEcho's performance in naturalistic scenarios while participants engage in various daily activities. Finally, we showcase EyeEcho's potential to be deployed on a commercial-off-the-shelf (COTS) smartphone, offering real-time facial expression tracking.

cs.HC↗

arXiv2024

GazeTrak: Exploring Acoustic-based Eye Tracking on a Glass Frame

In this paper, we present GazeTrak, the first acoustic-based eye tracking system on glasses. Our system only needs one speaker and four microphones attached to each side of the glasses. These acoustic sensors capture the formations of the eyeballs and the surrounding areas by emitting encoded inaudible sound towards eyeballs and receiving the reflected signals. These reflected signals are further processed to calculate the echo profiles, which are fed to a customized deep learning pipeline to continuously infer the gaze position. In a user study with 20 participants, GazeTrak achieves an accuracy of 3.6° within the same remounting session and 4.9° across different sessions with a refreshing rate of 83.3 Hz and a power signature of 287.9 mW. Furthermore, we report the performance of our gaze tracking system fully implemented on an MCU with a low-power CNN accelerator (MAX78002). In this configuration, the system runs at up to 83.3 Hz and has a total power signature of 95.4 mW with a 30 Hz FPS.

cs.HC↗

arXiv2024

Revisiting azimuthal angular asymmetries in diffractive di-jet production

We explore the impact of initial state soft gluon radiations on the azimuthal angle asymmetries in photo-production of hard di-jet via coherent diffraction in ultraperipheral heavy ion collisions, as well as in electron-proton ($ep$) and electron-nucleus ($eA$) collisions. The primary production mechanism is identified as the diffractive production of two hard jets, accompanied by a collinear gluon emission along the beam direction. In contrast, the diffractive exclusive di-jet production, where the initial state radiation is absent, is suppressed due to color transparency. Our analysis shows that azimuthal asymmetries, traditionally attributed to final state gluon emissions, are reduced by the presence of initial state radiations. The sensitivity of azimuthal asymmetries to both initial and final state radiations suggests that they could provide novel insights into the mechanisms of di-jet production in diffractive processes.

hep-ph↗

arXiv2024

Neural-PBIR Reconstruction of Shape, Material, and Illumination

Reconstructing the shape and spatially varying surface appearances of a physical-world object as well as its surrounding illumination based on 2D images (e.g., photographs) of the object has been a long-standing problem in computer vision and graphics. In this paper, we introduce an accurate and highly efficient object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR). Our pipeline firstly leverages a neural SDF based shape reconstruction to produce high-quality but potentially imperfect object shape. Then, we introduce a neural material and lighting distillation stage to achieve high-quality predictions for material and illumination. In the last stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination. Experimental results demonstrate our pipeline significantly outperforms existing methods quality-wise and performance-wise.

cs.CV↗

arXiv2024

Multi-modal News Understanding with Professionally Labelled Videos (ReutersViLNews)

While progress has been made in the domain of video-language understanding, current state-of-the-art algorithms are still limited in their ability to understand videos at high levels of abstraction, such as news-oriented videos. Alternatively, humans easily amalgamate information from video and language to infer information beyond what is visually observable in the pixels. An example of this is watching a news story, where the context of the event can play as big of a role in understanding the story as the event itself. Towards a solution for designing this ability in algorithms, we present a large-scale analysis on an in-house dataset collected by the Reuters News Agency, called Reuters Video-Language News (ReutersViLNews) dataset which focuses on high-level video-language understanding with an emphasis on long-form news. The ReutersViLNews Dataset consists of long-form news videos collected and labeled by news industry professionals over several years and contains prominent news reporting from around the world. Each video involves a single story and contains action shots of the actual event, interviews with people associated with the event, footage from nearby areas, and more. ReutersViLNews dataset contains videos from seven subject categories: disaster, finance, entertainment, health, politics, sports, and miscellaneous with annotations from high-level to low-level, title caption, visual video description, high-level story description, keywords, and location. We first present an analysis of the dataset statistics of ReutersViLNews compared to previous datasets. Then we benchmark state-of-the-art approaches for four different video-language tasks. The results suggest that news-oriented videos are a substantial challenge for current video-language understanding algorithms and we conclude by providing future directions in designing approaches to solve the ReutersViLNews dataset.

cs.CV↗

arXiv2024

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

We present TextureDreamer, a novel image-guided texture synthesis method to transfer relightable textures from a small number of input images (3 to 5) to target 3D shapes across arbitrary categories. Texture creation is a pivotal challenge in vision and graphics. Industrial companies hire experienced artists to manually craft textures for 3D assets. Classical methods require densely sampled views and accurately aligned geometry, while learning-based methods are confined to category-specific shapes within the dataset. In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation. Our core idea, personalized geometry-aware score distillation (PGSD), draws inspiration from recent advancements in diffuse models, including personalized modeling for texture information extraction, variational score distillation for detailed appearance synthesis, and explicit geometry guidance with ControlNet. Our integration and several essential modifications substantially improve the texture quality. Experiments on real images spanning different categories show that TextureDreamer can successfully transfer highly realistic, semantic meaningful texture to arbitrary objects, surpassing the visual quality of previous state-of-the-art.

cs.CV↗

arXiv2024

Interleaved Training for Massive MIMO Downlink via Exploring Spatial Correlation

Interleaved training has been studied for single-user and multi-user massive MIMO downlink with either fully-digital or hybrid beamforming. However, the impact of channel correlation on its average training overhead is rarely addressed. In this paper, we explore the channel correlation to improve the interleaved training for single-user massive MIMO downlink. For the beam-domain interleaved training, we propose a modified scheme by optimizing the beam training codebook. The basic antenna-domain interleaved training is also improved by dynamically adjusting the training order of the base station (BS) antennas during the training process based on the values of the already trained channels. Exact and simplified approximate expressions of the average training length are derived in closed-form for the basic and modified beam-domain schemes and the basic antenna-domain scheme in correlated channels. For the modified antenna-domain scheme, a deep neural network (DNN)-based approximation is provided for fast performance evaluation. Analytical results and simulations verify the accuracy of our derived training length expressions and explicitly reveal the impact of system parameters on the average training length. In addition, the modified beam/antenna-domain schemes are shown to have a shorter average training length compared to the basic schemes.

cs.IT↗

arXiv2024

Joint Port Selection Based Channel Acquisition for FDD Cell-Free Massive MIMO

In frequency division duplexing (FDD) cell-free massive MIMO, the acquisition of the channel state information (CSI) is very challenging because of the large overhead required for the training and feedback of the downlink channels of multiple cooperating base stations (BSs). In this paper, for systems with partial uplink-downlink channel reciprocity, and a general spatial domain channel model with variations in the average port power and correlation among port coefficients, we propose a joint-port-selection-based CSI acquisition and feedback scheme for the downlink transmission with zero-forcing precoding. The scheme uses an eigenvalue-decomposition-based transformation to reduce the feedback overhead by exploring the port correlation. We derive the sum-rate of the system for any port selection. Based on the sum-rate result, we propose a low-complexity greedy-search-based joint port selection (GS-JPS) algorithm. Moreover, to adapt to fast time-varying scenarios, a supervised deep learning-enhanced joint port selection (DL-JPS) algorithm is proposed. Simulations verify the effectiveness of our proposed schemes and their advantage over existing port-selection channel acquisition schemes.

cs.IT↗

arXiv2024

ProAgent: Building Proactive Cooperative Agents with Large Language Models

Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely primarily on learning-based methods, whose policy generalization depends heavily on the diversity of teammates they interact with during the training phase. Such reliance, however, constrains the agents' capacity for strategic adaptation when cooperating with unfamiliar teammates, which becomes a significant challenge in zero-shot coordination scenarios. To address this challenge, we propose ProAgent, a novel framework that harnesses large language models (LLMs) to create proactive agents capable of dynamically adapting their behavior to enhance cooperation with teammates. ProAgent can analyze the present state, and infer the intentions of teammates from observations. It then updates its beliefs in alignment with the teammates' subsequent actual behaviors. Moreover, ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various of coordination scenarios. Experimental evaluations conducted within the Overcooked-AI environment unveil the remarkable performance superiority of ProAgent, outperforming five methods based on self-play and population-based training when cooperating with AI agents. Furthermore, in partnered with human proxy models, its performance exhibits an average improvement exceeding 10% compared to the current state-of-the-art method. For more information about our project, please visit~\url{https://pku-proagent.github.io}.

cs.AI↗

arXiv2024

Three-state coherent control using narrowband and passband sequences

In this work, we propose a comprehensive design for narrowband and passband composite pulse sequences by involving the dynamics of all states in the three-state system. The design is quite universal as all pulse parameters can be freely employed to modify the coefficients of error terms. Two modulation techniques, the strength and phase modulations, are used to achieve arbitrary population transfer with a desired excitation profile, while the system keeps minimal leakage to the third state. Furthermore, the current sequences are capable of tolerating inaccurate waveforms, detunings errors, and work well when rotating wave approximation is not strictly justified. Therefore, this work provides versatile adaptability for shaping various excitation profiles in both narrowband and passband sequences.

quant-ph↗

arXiv2024

Entropy-based Probing Beam Selection and Beam Prediction via Deep Learning

Hierarchical beam search in mmWave communications incurs substantial training overhead, necessitating deep learning-enabled beam predictions to effectively leverage channel priors and mitigate this overhead. In this study, we introduce a comprehensive probabilistic model of power distribution in beamspace, and formulate the joint optimization problem of probing beam selection and probabilistic beam prediction as an entropy minimization problem. Then, we propose a greedy scheme to iteratively and alternately solve this problem, where a transformer-based beam predictor is trained to estimate the conditional power distribution based on the probing beams and user location within each iteration, and the trained predictor selects an unmeasured beam that minimizes the entropy of remaining beams. To further reduce the number of interactions and the computational complexity of the iterative scheme, we propose a two-stage probing beam selection scheme. Firstly, probing beams are selected from a location-specific codebook designed by an entropy-based criterion, and predictions are made with corresponding feedback. Secondly, the optimal beam is identified using additional probing beams with the highest predicted power values. Simulation results demonstrate the superiority of the proposed schemes compared to hierarchical beam search and beam prediction with uniform probing beams.

cs.IT↗

arXiv2023

Properties of new even and odd nonlinear coherent states with different parameters

We construct a class of nonlinear coherent states (NLCSs) by introducing a more general nonlinear function and study their non-classical properties, specifically the second-order correlation function $g^{(2)}(0)$, Mandel parameter $Q$, squeezing, amplitude squared squeezing and Wigner function of the optical field. The results indicate that the non-classical properties of the new types of even and odd NLCSs crucially depend on nonlinear functions. More concretely, we find that the new even NLCSs could exhibit the photon-bunching effect whereas the new odd NLCSs could show photon-antibunching effect. The degree of squeezing is also significantly affected by the parameter selection of these NLCSs. By employing various forms of nonlinear functions, it becomes possible to construct NLCSs with diverse properties, thereby providing a theoretical foundation for corresponding experimental investigations.

quant-ph↗

arXiv2023

Tail-Learning: Adaptive Learning Method for Mitigating Tail Latency in Autonomous Edge Systems

In the realm of edge computing, the increasing demand for high Quality of Service (QoS), particularly in dynamic multimedia streaming applications (e.g., Augmented Reality/Virtual Reality and online gaming), has prompted the need for effective solutions. Nevertheless, adopting an edge paradigm grounded in distributed computing has exacerbated the issue of tail latency. Given a limited variety of multimedia services supported by edge servers and the dynamic nature of user requests, employing traditional queuing methods to model tail latency in distributed edge computing is challenging, substantially exacerbating head-of-line (HoL) blocking. In response to this challenge, we have developed a learning-based scheduling method to mitigate the overall tail latency, which adaptively selects appropriate edge servers for execution as incoming distributed tasks vary with unknown size. To optimize the utilization of the edge computing paradigm, we leverage Laplace transform techniques to theoretically derive an upper bound for the response time of edge servers. Subsequently, we integrate this upper bound into reinforcement learning to facilitate tail learning and enable informed decisions for autonomous distributed scheduling. The experiment results demonstrate the efficiency in reducing tail latency compared to existing methods.

cs.DC↗

arXiv2023

BayesDAG: Gradient-Based Posterior Inference for Causal Discovery

Bayesian causal discovery aims to infer the posterior distribution over causal models from observed data, quantifying epistemic uncertainty and benefiting downstream tasks. However, computational challenges arise due to joint inference over combinatorial space of Directed Acyclic Graphs (DAGs) and nonlinear functions. Despite recent progress towards efficient posterior inference over DAGs, existing methods are either limited to variational inference on node permutation matrices for linear causal models, leading to compromised inference accuracy, or continuous relaxation of adjacency matrices constrained by a DAG regularizer, which cannot ensure resulting graphs are DAGs. In this work, we introduce a scalable Bayesian causal discovery framework based on a combination of stochastic gradient Markov Chain Monte Carlo (SG-MCMC) and Variational Inference (VI) that overcomes these limitations. Our approach directly samples DAGs from the posterior without requiring any DAG regularization, simultaneously draws function parameter samples and is applicable to both linear and nonlinear causal models. To enable our approach, we derive a novel equivalence to the permutation-based DAG learning, which opens up possibilities of using any relaxed gradient estimator defined over permutations. To our knowledge, this is the first framework applying gradient-based MCMC sampling for causal discovery. Empirical evaluation on synthetic and real-world datasets demonstrate our approach's effectiveness compared to state-of-the-art baselines.

cs.LG↗

arXiv2023

Vanishing of the anomalous Hall effect and enhanced carrier mobility in the spin-gapless ferromagnetic Mn2CoGa1-xAlx alloys

Spin gapless semiconductor (SGS) has attracted long attention since its theoretical prediction, while concrete experimental hints are still lack in the relevant Heusler alloys. Here in this work, by preparing the series alloys of Mn2CoGa1-xAlx (x=0, 0.25, 0.5, 0.75 and 1), we identified the vanishing of anomalous Hall effect in the ferromagnetic Mn2CoGa (or x=0.25) alloy in a wide temperature interval, accompanying with growing contribution from the ordinary Hall effect. As a result, comparatively low carrier density (1020 cm-3) and high carrier mobility (150 cm2/Vs) are obtained in Mn2CoGa (or x=0.25) alloy in the temperature range of 10-200K. These also lead to a large dip in the related magnetoresistance at low fields. While in high Al content, despite the magnetization behavior is not altered significantly, the Hall resistivity is instead dominated by the anomalous one, just analogous to that widely reported in Mn2CoAl. The distinct electrical transport behavior of x=0 and x=0.75 (or 1) is presently understood by their possible different scattering mechanism of the anomalous Hall effect due to the differences in atomic order and conductivity. Our work can expand the existing understanding of the SGS properties and offer a better SGS candidate with higher carrier mobility that can facilitate the application in the spin-injected related devices.

cond-mat.mtrl-sci↗

arXiv2023

Digital Twin-Enhanced Deep Reinforcement Learning for Resource Management in Networks Slicing

Network slicing-based communication systems can dynamically and efficiently allocate resources for diversified services. However, due to the limitation of the network interface on channel access and the complexity of the resource allocation, it is challenging to achieve an acceptable solution in the practical system without precise prior knowledge of the dynamics probability model of the service requests. Existing work attempts to solve this problem using deep reinforcement learning (DRL), however, such methods usually require a lot of interaction with the real environment in order to achieve good results. In this paper, a framework consisting of a digital twin and reinforcement learning agents is present to handle the issue. Specifically, we propose to use the historical data and the neural networks to build a digital twin model to simulate the state variation law of the real environment. Then, we use the data generated by the network slicing environment to calibrate the digital twin so that it is in sync with the real environment. Finally, DRL for slice optimization optimizes its own performance in this virtual pre-verification environment. We conducted an exhaustive verification of the proposed digital twin framework to confirm its scalability. Specifically, we propose to use loss landscapes to visualize the generalization of DRL solutions. We explore a distillation-based optimization scheme for lightweight slicing strategies. In addition, we also extend the framework to offline reinforcement learning, where solutions can be used to obtain intelligent decisions based solely on historical data. Numerical simulation experiments show that the proposed digital twin can significantly improve the performance of the slice optimization strategy.

cs.NI↗

arXiv2023

Learn the Time to Learn: Replay Scheduling in Continual Learning

Replay methods are known to be successful at mitigating catastrophic forgetting in continual learning scenarios despite having limited access to historical data. However, storing historical data is cheap in many real-world settings, yet replaying all historical data is often prohibited due to processing time constraints. In such settings, we propose that continual learning systems should learn the time to learn and schedule which tasks to replay at different time steps. We first demonstrate the benefits of our proposal by using Monte Carlo tree search to find a proper replay schedule, and show that the found replay schedules can outperform fixed scheduling policies when combined with various replay methods in different continual learning settings. Additionally, we propose a framework for learning replay scheduling policies with reinforcement learning. We show that the learned policies can generalize better in new continual learning scenarios compared to equally replaying all seen tasks, without added computational cost. Our study reveals the importance of learning the time to learn in continual learning, which brings current research closer to real-world needs.

cs.LG↗

arXiv2023

Absence of metallicity and bias-dependent resistivity in low-carrier-density EuCd2As2

EuCd2As2 was theoretically predicted to be a minimal model of Weyl semimetals with a single pair of Weyl points in the ferromagnet state. However, the heavily p-doped EuCd2As2 crystals in previous experiments prevent direct identification of the semimetal hypothesis. Here we present a comprehensive magneto-transport study of high-quality EuCd2As2 crystals with ultralow bulk carrier density (10^13 cm-3). In contrast to the general expectation of a Weyl semimetal phase, EuCd2As2 shows insulating behavior in both antiferromagnetic and ferromagnetic states as well as surface-dominated conduction from band bending. Moreover, the application of a dc bias current can dramatically modulate the resistance by over one order of magnitude, and induce a periodic resistance oscillation due to the geometric resonance. Such nonlinear transport results from the highly nonequilibrium state induced by electrical field near the band edge. Our results suggest an insulating phase in EuCd2As2 and put a strong constraint on the underlying mechanism of anomalous transport properties in this system.

cond-mat.mes-hall↗