arXiv subjects

Hao Zhang

Publications and source records attributed to Hao Zhang.

At least 829 records · Page 46Linked to original sources

arXiv2023

Text-Guided Generation and Editing of Compositional 3D Avatars

Our goal is to create a realistic 3D facial avatar with hair and accessories using only a text description. While this challenge has attracted significant recent interest, existing methods either lack realism, produce unrealistic shapes, or do not support editing, such as modifications to the hairstyle. We argue that existing methods are limited because they employ a monolithic modeling approach, using a single representation for the head, face, hair, and accessories. Our observation is that the hair and face, for example, have very different structural qualities that benefit from different representations. Building on this insight, we generate avatars with a compositional model, in which the head, face, and upper body are represented with traditional 3D meshes, and the hair, clothing, and accessories with neural radiance fields (NeRF). The model-based mesh representation provides a strong geometric prior for the face region, improving realism while enabling editing of the person's appearance. By using NeRFs to represent the remaining components, our method is able to model and synthesize parts with complex geometry and appearance, such as curly hair and fluffy scarves. Our novel system synthesizes these high-quality compositional avatars from text descriptions. The experimental results demonstrate that our method, Text-guided generation and Editing of Compositional Avatars (TECA), produces avatars that are more realistic than those of recent methods while being editable because of their compositional nature. For example, our TECA enables the seamless transfer of compositional features like hairstyles, scarves, and other accessories between avatars. This capability supports applications such as virtual try-on.

cs.CV↗

arXiv2023

Efficient Memory Management for Large Language Model Serving with PagedAttention

High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically. When managed inefficiently, this memory can be significantly wasted by fragmentation and redundant duplication, limiting the batch size. To address this problem, we propose PagedAttention, an attention algorithm inspired by the classical virtual memory and paging techniques in operating systems. On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage. Our evaluations show that vLLM improves the throughput of popular LLMs by 2-4$\times$ with the same level of latency compared to the state-of-the-art systems, such as FasterTransformer and Orca. The improvement is more pronounced with longer sequences, larger models, and more complex decoding algorithms. vLLM's source code is publicly available at https://github.com/vllm-project/vllm

cs.LG↗

arXiv2023

Ballistic PbTe Nanowire Devices

Disorder is the primary obstacle in current Majorana nanowire experiments. Reducing disorder or achieving ballistic transport is thus of paramount importance. In clean and ballistic nanowire devices, quantized conductance is expected with plateau quality serving as a benchmark for disorder assessment. Here, we introduce ballistic PbTe nanowire devices grown using the selective-area-growth (SAG) technique. Quantized conductance plateaus in units of $2e^2/h$ are observed at zero magnetic field. This observation represents an advancement in diminishing disorder within SAG nanowires, as none of the previously studied SAG nanowires (InSb or InAs) exhibit zero-field ballistic transport. Notably, the plateau values indicate that the ubiquitous valley degeneracy in PbTe is lifted in nanowire devices. This degeneracy lifting addresses an additional concern in the pursuit of Majorana realization. Moreover, these ballistic PbTe nanowires may enable the search for clean signatures of the spin-orbit helical gap in future devices.

cond-mat.mes-hall↗

arXiv2023

FuseFPS: Accelerating Farthest Point Sampling with Fusing KD-tree Construction for Point Clouds

Point cloud analytics has become a critical workload for embedded and mobile platforms across various applications. Farthest point sampling (FPS) is a fundamental and widely used kernel in point cloud processing. However, the heavy external memory access makes FPS a performance bottleneck for real-time point cloud processing. Although bucket-based farthest point sampling can significantly reduce unnecessary memory accesses during the point sampling stage, the KD-tree construction stage becomes the predominant contributor to execution time. In this paper, we present FuseFPS, an architecture and algorithm co-design for bucket-based farthest point sampling. We first propose a hardware-friendly sampling-driven KD-tree construction algorithm. The algorithm fuses the KD-tree construction stage into the point sampling stage, further reducing memory accesses. Then, we design an efficient accelerator for bucket-based point sampling. The accelerator can offload the entire bucket-based FPS kernel at a low hardware cost. Finally, we evaluate our approach on various point cloud datasets. The detailed experiments show that compared to the state-of-the-art accelerator QuickFPS, FuseFPS achieves about 4.3$\times$ and about 6.1$\times$ improvements on speed and power efficiency, respectively.

cs.AR↗

arXiv2023

Renormalized Classical Theory of Quantum Magnets

We derive a renormalized classical spin (RCS) theory for $S > 1/2$ quantum magnets by constraining a generalized classical theory that includes all multipolar fluctuations to a reduced CP$^1$ phase space of dipolar SU($2$) coherent states. When the spin Hamiltonian $\hat{\cal{H}}^{S}$ is linear in the spin operators $\hat{\boldsymbol{S}}_j$ for each lattice site $j$, the RCS Hamiltonian $\tilde{\cal{H}}_{\rm cl}$ coincides with the usual classical model $\cal{H}_{\rm cl} = \lim_{S\rightarrow\infty} \hat{\cal{H}}^S$. In the presence of non-linear terms, however, the RCS theory is more accurate than $\cal{H}_{\rm cl}$. For the many materials modeled by spin Hamiltonians with (non-linear) single-ion anisotropy terms, the use of the RCS theory is essential to accurately model phase diagrams and to extract the correct Hamiltonian parameters from neutron scattering data

cond-mat.str-el↗

arXiv2023

Hard superconducting gap in PbTe nanowires

Semiconductor nanowires coupled to a superconductor provide a powerful testbed for quantum device physics such as Majorana zero modes and gate-tunable hybrid qubits. The performance of these quantum devices heavily relies on the quality of the induced superconducting gap. A hard gap, evident as vanishing subgap conductance in tunneling spectroscopy, is both necessary and desired. Previously, a hard gap has been achieved and extensively studied in III-V semiconductor nanowires (InAs and InSb). In this study, we present the observation of a hard superconducting gap in PbTe nanowires coupled to a superconductor Pb. The gap size ($Δ$) is $\sim$ 1 meV (maximally 1.3 meV in one device). Additionally, subgap Andreev bound states can also be created and controlled through gate tuning. Tuning a device into the open regime can reveal Andreev enhancement of the subgap conductance, suggesting a remarkable transparent superconductor-semiconductor interface, with a transparency of $\sim$ 0.96. These results pave the way for diverse superconducting quantum devices based on PbTe nanowires.

cond-mat.mes-hall↗

arXiv2023

Structure of large quadratic character sums

In this article, we study the distribution of large quadratic character sums. Based on the recent work of Lamzouri~\cite{La2022}, we obtain the structure results of quadratic characters with large character sums.

math.NT↗

arXiv2023

Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Model

In this paper, we rethink the low-light image enhancement task and propose a physically explainable and generative diffusion model for low-light image enhancement, termed as Diff-Retinex. We aim to integrate the advantages of the physical model and the generative network. Furthermore, we hope to supplement and even deduce the information missing in the low-light image through the generative network. Therefore, Diff-Retinex formulates the low-light image enhancement problem into Retinex decomposition and conditional image generation. In the Retinex decomposition, we integrate the superiority of attention in Transformer and meticulously design a Retinex Transformer decomposition network (TDN) to decompose the image into illumination and reflectance maps. Then, we design multi-path generative diffusion networks to reconstruct the normal-light Retinex probability distribution and solve the various degradations in these components respectively, including dark illumination, noise, color deviation, loss of scene contents, etc. Owing to generative diffusion model, Diff-Retinex puts the restoration of low-light subtle detail into practice. Extensive experiments conducted on real-world low-light datasets qualitatively and quantitatively demonstrate the effectiveness, superiority, and generalization of the proposed method.

cs.CV↗

arXiv2023

Advances in Data-Driven Analysis and Synthesis of 3D Indoor Scenes

This report surveys advances in deep learning-based modeling techniques that address four different 3D indoor scene analysis tasks, as well as synthesis of 3D indoor scenes. We describe different kinds of representations for indoor scenes, various indoor scene datasets available for research in the aforementioned areas, and discuss notable works employing machine learning models for such scene modeling tasks based on these representations. Specifically, we focus on the analysis and synthesis of 3D indoor scenes. With respect to analysis, we focus on four basic scene understanding tasks -- 3D object detection, 3D scene segmentation, 3D scene reconstruction and 3D scene similarity. And for synthesis, we mainly discuss neural scene synthesis works, though also highlighting model-driven methods that allow for human-centric, progressive scene synthesis. We identify the challenges involved in modeling scenes for these tasks and the kind of machinery that needs to be developed to adapt to the data representation, and the task setting in general. For each of these tasks, we provide a comprehensive summary of the state-of-the-art works across different axes such as the choice of data representation, backbone, evaluation metric, input, output, etc., providing an organized review of the literature. Towards the end, we discuss some interesting research directions that have the potential to make a direct impact on the way users interact and engage with these virtual scene models, making them an integral part of the metaverse.

cs.GR↗

arXiv2023

Deep AHS: A Deep Learning Approach to Acoustic Howling Suppression

In this paper, we formulate acoustic howling suppression (AHS) as a supervised learning problem and propose a deep learning approach, called Deep AHS, to address it. Deep AHS is trained in a teacher forcing way which converts the recurrent howling suppression process into an instantaneous speech separation process to simplify the problem and accelerate the model training. The proposed method utilizes properly designed features and trains an attention based recurrent neural network (RNN) to extract the target signal from the microphone recording, thus attenuating the playback signal that may lead to howling. Different training strategies are investigated and a streaming inference method implemented in a recurrent mode used to evaluate the performance of the proposed method for real-time howling suppression. Deep AHS avoids howling detection and intrinsically prohibits howling from happening, allowing for more flexibility in the design of audio systems. Experimental results show the effectiveness of the proposed method for howling suppression under different scenarios.

eess.AS↗

arXiv2023

Nowhere to Go: Benchmarking Multi-robot Collaboration in Target Trapping Environment

Collaboration is one of the most important factors in multi-robot systems. Considering certain real-world applications and to further promote its development, we propose a new benchmark to evaluate multi-robot collaboration in Target Trapping Environment (T2E). In T2E, two kinds of robots (called captor robot and target robot) share the same space. The captors aim to catch the target collaboratively, while the target will try to escape from the trap. Both the trapping and escaping process can use the environment layout to help achieve the corresponding objective, which requires high collaboration between robots and the utilization of the environment. For the benchmark, we present and evaluate multiple learning-based baselines in T2E, and provide insights into regimes of multi-robot collaboration. We also make our benchmark publicly available and encourage researchers from related robotics disciplines to propose, evaluate, and compare their solutions in this benchmark. Our project is released at https://github.com/Dr-Xiaogaren/T2E.

cs.RO↗

arXiv2023

Adaptive Split-Fusion Transformer

Neural networks for visual content understanding have recently evolved from convolutional ones (CNNs) to transformers. The prior (CNN) relies on small-windowed kernels to capture the regional clues, demonstrating solid local expressiveness. On the contrary, the latter (transformer) establishes long-range global connections between localities for holistic learning. Inspired by this complementary nature, there is a growing interest in designing hybrid models to best utilize each technique. Current hybrids merely replace convolutions as simple approximations of linear projection or juxtapose a convolution branch with attention, without concerning the importance of local/global modeling. To tackle this, we propose a new hybrid named Adaptive Split-Fusion Transformer (ASF-former) to treat convolutional and attention branches differently with adaptive weights. Specifically, an ASF-former encoder equally splits feature channels into half to fit dual-path inputs. Then, the outputs of dual-path are fused with weighting scalars calculated from visual cues. We also design the convolutional path compactly for efficiency concerns. Extensive experiments on standard benchmarks, such as ImageNet-1K, CIFAR-10, and CIFAR-100, show that our ASF-former outperforms its CNN, transformer counterparts, and hybrid pilots in terms of accuracy (83.9% on ImageNet-1K), under similar conditions (12.9G MACs/56.7M Params, without large-scale pre-training). The code is available at: https://github.com/szx503045266/ASF-former.

cs.CV↗

arXiv2023

Spin wave Hamiltonian and anomalous scattering in NiPS$_3$

We report a comprehensive spin wave analysis of the semiconducting honeycomb van der Waal antiferromagnet NiPS$_3$. Using single crystal inelastic neutron scattering, we map out the full Brillouin zone and fit the observed modes to a spin wave model with rigorously defined uncertainty. We find that the third neighbor exchange $J_3$ dominates the Hamiltonian, a feature which we fully account for by ab-initio density functional theory calculations. We also quantify the degree to which the three-fold rotation symmetry is broken and account for the $Q=0$ excitations observed in other measurements, yielding a spin exchange model which is consistent across multiple experimental probes. We also identify a strongly reduced static ordered moment and reduced low-energy intensity relative to the linear spin wave calculations, signaling unexplained features in the magnetism which requires going beyond the linear spin wave approximation.

cond-mat.str-el↗

arXiv2023

Text Injection for Capitalization and Turn-Taking Prediction in Speech Models

Text injection for automatic speech recognition (ASR), wherein unpaired text-only data is used to supplement paired audio-text data, has shown promising improvements for word error rate. This study examines the use of text injection for auxiliary tasks, which are the non-ASR tasks often performed by an E2E model. In this work, we use joint end-to-end and internal language model training (JEIT) as our text injection algorithm to train an ASR model which performs two auxiliary tasks. The first is capitalization, which is a de-normalization task. The second is turn-taking prediction, which attempts to identify whether a user has completed their conversation turn in a digital assistant interaction. We show results demonstrating that our text injection method boosts capitalization performance for long-tail data, and improves turn-taking detection recall.

cs.CL↗

arXiv2023

Contrastive Attraction and Contrastive Repulsion for Representation Learning

Contrastive learning (CL) methods effectively learn data representations in a self-supervision manner, where the encoder contrasts each positive sample over multiple negative samples via a one-vs-many softmax cross-entropy loss. By leveraging large amounts of unlabeled image data, recent CL methods have achieved promising results when pretrained on large-scale datasets, such as ImageNet. However, most of them consider the augmented views from the same instance are positive pairs, while views from other instances are negative ones. Such binary partition insufficiently considers the relation between samples and tends to yield worse performance when generalized on images in the wild. In this paper, to further improve the performance of CL and enhance its robustness on various datasets, {we propose a doubly CL strategy that separately compares positive and negative samples within their own groups, and then proceeds with a contrast between positive and negative groups}. We realize this strategy with contrastive attraction and contrastive repulsion (CACR), which makes the query not only exert a greater force to attract more distant positive samples but also do so to repel closer negative samples. Theoretical analysis reveals that CACR generalizes CL's behavior by positive attraction and negative repulsion, and it further considers the intra-contrastive relation within the positive and negative pairs to narrow the gap between the sampled and true distribution, which is important when datasets are less curated. With our extensive experiments, CACR not only demonstrates good performance on CL benchmarks, but also shows better robustness when generalized on imbalanced image datasets. Code and pre-trained checkpoints are available at https://github.com/JegZheng/CACR-SSL.

cs.LG↗

arXiv2023

Instabilities of heavy magnons in an anisotropic magnet

The search for new elementary particles is one of the most basic pursuits in physics, spanning from subatomic physics to quantum materials. Magnons are the ubiquitous elementary quasiparticle to describe the excitations of fully-ordered magnetic systems. But other possibilities exist, including fractional and multipolar excitations. Here, we demonstrate that strong quantum interactions exist between three flavors of elementary quasiparticles in the uniaxial spin-one magnet FeI2. Using neutron scattering in an applied magnetic field, we observe spontaneous decay between conventional and heavy magnons and the recombination of these quasiparticles into a super-heavy bound-state. Akin to other contemporary problems in quantum materials, the microscopic origin for new physics in FeI2 is the quasi-flat nature of excitation bands and the presence of Kitaev anisotropic magnetic exchange interactions.

cond-mat.str-el↗

arXiv2023

Quantum scaling atomic superheterodyne receiver

Measurement sensitivity is one of the critical indicators for Rydberg atomic radio receivers. This work quantitatively studies the relationship between the atomic superheterodyne receiver's sensitivity and the number of atoms involved in the measurement. The atom number is changed by adjusting the length of the interaction area. The results show that for the ideal case, the sensitivity of the atomic superheterodyne receiver exhibits a quantum scaling: the amplitude of its output signal is proportional to the atom number, and the amplitude of its read-out noise is proportional to the square root of the atom number. Hence, its sensitivity is inversely proportional to the square root of the atom number. This work also gives a detailed discussion of the properties of transit noise in atomic receivers and the influence of some non-ideal factors on sensitivity scaling. This work is significant in the field of atom-based quantum precision measurements.

quant-ph↗

arXiv2023

DFA3D: 3D Deformable Attention For 2D-to-3D Feature Lifting

In this paper, we propose a new operator, called 3D DeFormable Attention (DFA3D), for 2D-to-3D feature lifting, which transforms multi-view 2D image features into a unified 3D space for 3D object detection. Existing feature lifting approaches, such as Lift-Splat-based and 2D attention-based, either use estimated depth to get pseudo LiDAR features and then splat them to a 3D space, which is a one-pass operation without feature refinement, or ignore depth and lift features by 2D attention mechanisms, which achieve finer semantics while suffering from a depth ambiguity problem. In contrast, our DFA3D-based method first leverages the estimated depth to expand each view's 2D feature map to 3D and then utilizes DFA3D to aggregate features from the expanded 3D feature maps. With the help of DFA3D, the depth ambiguity problem can be effectively alleviated from the root, and the lifted features can be progressively refined layer by layer, thanks to the Transformer-like architecture. In addition, we propose a mathematically equivalent implementation of DFA3D which can significantly improve its memory efficiency and computational speed. We integrate DFA3D into several methods that use 2D attention-based feature lifting with only a few modifications in code and evaluate on the nuScenes dataset. The experiment results show a consistent improvement of +1.41\% mAP on average, and up to +15.1\% mAP improvement when high-quality depth information is available, demonstrating the superiority, applicability, and huge potential of DFA3D. The code is available at https://github.com/IDEA-Research/3D-deformable-attention.git.

cs.CV↗