arXiv subjects

Chao Zhang

Publications and source records attributed to Chao Zhang.

At least 613 records · Page 34Linked to original sources

arXiv2023

GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework

Particle-based Variational Inference (ParVI) methods approximate the target distribution by iteratively evolving finite weighted particle systems. Recent advances of ParVI methods reveal the benefits of accelerated position update strategies and dynamic weight adjustment approaches. In this paper, we propose the first ParVI framework that possesses both accelerated position update and dynamical weight adjustment simultaneously, named the General Accelerated Dynamic-Weight Particle-based Variational Inference (GAD-PVI) framework. Generally, GAD-PVI simulates the semi-Hamiltonian gradient flow on a novel Information-Fisher-Rao space, which yields an additional decrease on the local functional dissipation. GAD-PVI is compatible with different dissimilarity functionals and associated smoothing approaches under three information metrics. Experiments on both synthetic and real-world data demonstrate the faster convergence and reduced approximation error of GAD-PVI methods over the state-of-the-art.

cs.LG↗

arXiv2023

How Far Have We Gone in Vulnerability Detection Using Large Language Models

As software becomes increasingly complex and prone to vulnerabilities, automated vulnerability detection is critically important, yet challenging. Given the significant successes of large language models (LLMs) in various tasks, there is growing anticipation of their efficacy in vulnerability detection. However, a quantitative understanding of their potential in vulnerability detection is still missing. To bridge this gap, we introduce a comprehensive vulnerability benchmark VulBench. This benchmark aggregates high-quality data from a wide range of CTF (Capture-the-Flag) challenges and real-world applications, with annotations for each vulnerable function detailing the vulnerability type and its root cause. Through our experiments encompassing 16 LLMs and 6 state-of-the-art (SOTA) deep learning-based models and static analyzers, we find that several LLMs outperform traditional deep learning approaches in vulnerability detection, revealing an untapped potential in LLMs. This work contributes to the understanding and utilization of LLMs for enhanced software security.

cs.AI↗

arXiv2023

Revisiting DETR Pre-training for Object Detection

Motivated by the remarkable achievements of DETR-based approaches on COCO object detection and segmentation benchmarks, recent endeavors have been directed towards elevating their performance through self-supervised pre-training of Transformers while preserving a frozen backbone. Noteworthy advancements in accuracy have been documented in certain studies. Our investigation delved deeply into a representative approach, DETReg, and its performance assessment in the context of emerging models like $\mathcal{H}$-Deformable-DETR. Regrettably, DETReg proves inadequate in enhancing the performance of robust DETR-based models under full data conditions. To dissect the underlying causes, we conduct extensive experiments on COCO and PASCAL VOC probing elements such as the selection of pre-training datasets and strategies for pre-training target generation. By contrast, we employ an optimized approach named Simple Self-training which leads to marked enhancements through the combination of an improved box predictor and the Objects$365$ benchmark. The culmination of these endeavors results in a remarkable AP score of $59.3\%$ on the COCO val set, outperforming $\mathcal{H}$-Deformable-DETR + Swin-L without pre-training by $1.4\%$. Moreover, a series of synthetic pre-training datasets, generated by merging contemporary image-to-text(LLaVA) and text-to-image (SDXL) models, significantly amplifies object detection capabilities.

cs.CV↗

arXiv2023

Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection

Anomaly detection (AD) is a fundamental task in computer vision. It aims to identify incorrect image data patterns which deviate from the normal ones. Conventional methods generally address AD by preparing augmented negative samples to enforce self-supervised learning. However, these techniques typically do not consider semantics during augmentation, leading to the generation of unrealistic or invalid negative samples. Consequently, the feature extraction network can be hindered from embedding critical features. In this study, inspired by visual attention learning approaches, we propose CutSwap, which leverages saliency guidance to incorporate semantic cues for augmentation. Specifically, we first employ LayerCAM to extract multilevel image features as saliency maps and then perform clustering to obtain multiple centroids. To fully exploit saliency guidance, on each map, we select a pixel pair from the cluster with the highest centroid saliency to form a patch pair. Such a patch pair includes highly similar context information with dense semantic correlations. The resulting negative sample is created by swapping the locations of the patch pair. Compared to prior augmentation methods, CutSwap generates more subtle yet realistic negative samples to facilitate quality feature learning. Extensive experimental and ablative evaluations demonstrate that our method achieves state-of-the-art AD performance on two mainstream AD benchmark datasets.

cs.CV↗

arXiv2023

LanGWM: Language Grounded World Model

Recent advances in deep reinforcement learning have showcased its potential in tackling complex tasks. However, experiments on visual control tasks have revealed that state-of-the-art reinforcement learning models struggle with out-of-distribution generalization. Conversely, expressing higher-level concepts and global contexts is relatively easy using language. Building upon recent success of the large language models, our main objective is to improve the state abstraction technique in reinforcement learning by leveraging language for robust action selection. Specifically, we focus on learning language-grounded visual features to enhance the world model learning, a model-based reinforcement learning technique. To enforce our hypothesis explicitly, we mask out the bounding boxes of a few objects in the image observation and provide the text prompt as descriptions for these masked objects. Subsequently, we predict the masked objects along with the surrounding regions as pixel reconstruction, similar to the transformer-based masked autoencoder approach. Our proposed LanGWM: Language Grounded World Model achieves state-of-the-art performance in out-of-distribution test at the 100K interaction steps benchmarks of iGibson point navigation tasks. Furthermore, our proposed technique of explicit language-grounded visual representation learning has the potential to improve models for human-robot interaction because our extracted visual features are language grounded.

cs.LG↗

arXiv2023

Numerical testing of mirror diffusion of cosmic rays

The tension between recent observations and theories on cosmic ray (CR) diffusion necessitates exploration of new CR diffusion mechanisms. We perform the first numerical study on the mirror diffusion of CRs that is recently proposed by Lazarian & Xu (2021). We demonstrate that the perpendicular superdiffusion of turbulent magnetic fields and magnetic mirroring that naturally arise in magnetohydrodynamic (MHD) turbulence are the two essential physical ingredients for the mirror diffusion to happen. In supersonic, subsonic, and incompressible MHD turbulence, with the pitch angles of CRs repeatedly crossing $90^\circ$ due to the mirror reflection, we find that the mirror diffusion strongly enhances the confinement of CRs, and their pitch-angle-dependent parallel mean free path can be much smaller than the injection scale of turbulence. With the stochastic change of pitch angles due to gyroresonant scattering, CRs stochastically undergo slow mirror diffusion at relatively large pitch angles and fast scattering diffusion at smaller pitch angles, resulting in a Lévy-flight-like propagation.

astro-ph.HE↗

arXiv2023

Ultrahigh-inductance materials from spinodal decomposition

Disordered superconducting nitrides with kinetic inductance have long been considered a leading material candidate for high-inductance quantum-circuit applications. Despite continuing efforts in reducing material dimensions to increase the kinetic inductance and the corresponding circuit impedance, it becomes a fundamental challenge to improve further without compromising material qualities. To this end, we propose a method to drastically increase the kinetic inductance of superconducting materials via spinodal decomposition while keeping a low microwave loss. We use epitaxial Ti\textsubscript{0.48}Al\textsubscript{0.52}N as a model system, and for the first time demonstrate the utilization of spinodal decomposition to trigger the insulator-to-superconductor transition with a drastically enhanced material disorder. The measured kinetic inductance has increased by 2-3 orders of magnitude compared with all the best reported disordered superconducting nitrides. Our work paves the way for substantially enhancing and deterministically controlling the inductance for advanced superconducting quantum circuits.

quant-ph↗

arXiv2023

The first Ka-band (26.1-35 GHz) blind line survey towards Orion KL

We conducted a Ka-band (26.1--35 GHz) line survey towards Orion KL using the TianMa 65-m Radio Telescope (TMRT). It is the first blind line survey in the Ka band, and achieves a sensitivity of mK level (1--3 mK at a spectral resolution of $\sim$1 km s$^{-1}$). In total, 592 Gaussian features are extracted. Among them, 257 radio recombination lines (RRLs) are identified. The maximum $Δn$ of RRLs of H, He and C are 20, 15, and 5, respectively. Through stacking, we have detected the $β$ lines of ion RRLs (RRLs of C$^+$ with possible contribution of other ions like O$^+$) for the first time, and tentative signal of the $γ$ lines of ion RRLs can also be seen on the stacked spectrum. Besides, 318 other line features were assigned to 37 molecular species, and ten of these species were not detected in the Q-band survey of TMRT. The vibrationally excited states of nine species were also detected. Emission of most species can be modeled under LTE. A number of transitions of E-CH3OH ($J_2-J_1$) display maser effects, which are confirmed by our modeling, and besides the bumping peak at $J\sim 6$ there is another peak at $J\sim 13$. Methylcyanoacetylene (CH$_3$C$_3$N) is detected in Orion KL for the first time. This work emphasizes that the Ka band, which was long-ignored for spectral line surveys, is very useful for surveying RRLs and molecular lines simultaneously.

astro-ph.GA↗

arXiv2023

Salt effects on the mechanical properties of ionic conductive polymer: a molecular dynamics study

Functoinal polymers can be used as electrolyte and binder materials in solid-state batteries. This often requires performance targets in terms of both transport and mechanical properties. In this work, a model ionic conductive polymer system, i.e., poly(ethylene oxide)-LiTFSI, was used to study the impact of salt concentrations on mechanical properties, including different types of elastic moduli and the visoelasticity with both non-equilibrium and equilibrium molecular dynamics simulations. We found an encouragingly good agreement between experiments and simulations regarding the Young's modulus, bulk modulus and viscosity. In addition, we identified an intermediate salt concentration at which the system shows high ionic conductivity, high Young's modulus and short elastic restoration time. Therefore, this study laid down the groundwork for investigating ionic conductive polymer binders with self-healing functionality from molecular dynamics simulations.

cond-mat.soft↗

arXiv2023

Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning

Large language models (LLMs) have shown remarkable capabilities in various natural language understanding tasks. With only a few demonstration examples, these LLMs can quickly adapt to target tasks without expensive gradient updates. Common strategies to boost such 'in-context' learning ability are to ensemble multiple model decoded results and require the model to generate an explanation along with the prediction. However, these models often treat different class predictions equally and neglect the potential discrepancy between the explanations and predictions. To fully unleash the power of explanations, we propose EASE, an Explanation-Aware Soft Ensemble framework to empower in-context learning with LLMs. We design two techniques, explanation-guided ensemble, and soft probability aggregation, to mitigate the effect of unreliable explanations and improve the consistency between explanations and final predictions. Experiments on seven natural language understanding tasks and four varying-size LLMs demonstrate the effectiveness of our proposed framework.

cs.CL↗

arXiv2023

Speech-based Slot Filling using Large Language Models

Recently, advancements in large language models (LLMs) have shown an unprecedented ability across various language tasks. This paper investigates the potential application of LLMs to slot filling with noisy ASR transcriptions, via both in-context learning and task-specific fine-tuning. Dedicated prompt designs and fine-tuning approaches are proposed to improve the robustness of LLMs for slot filling with noisy ASR transcriptions. Moreover, a linearised knowledge injection (LKI) scheme is also proposed to integrate dynamic external knowledge into LLMs. Experiments were performed on SLURP to quantify the performance of LLMs, including GPT-3.5-turbo, GPT-4, LLaMA-13B and Vicuna-13B (v1.1 and v1.5) with different ASR error rates. The use of the proposed fine-tuning together with the LKI scheme for LLaMA-13B achieved an 8.3% absolute SLU-F1 improvement compared to the strong Flan-T5-base baseline system on a limited data setup.

cs.CL↗

arXiv2023

PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literature

Scientific information extraction (SciIE), which aims to automatically extract information from scientific literature, is becoming more important than ever. However, there are no existing SciIE datasets for polymer materials, which is an important class of materials used ubiquitously in our daily lives. To bridge this gap, we introduce POLYIE, a new SciIE dataset for polymer materials. POLYIE is curated from 146 full-length polymer scholarly articles, which are annotated with different named entities (i.e., materials, properties, values, conditions) as well as their N-ary relations by domain experts. POLYIE presents several unique challenges due to diverse lexical formats of entities, ambiguity between entities, and variable-length relations. We evaluate state-of-the-art named entity extraction and relation extraction models on POLYIE, analyze their strengths and weaknesses, and highlight some difficult cases for these models. To the best of our knowledge, POLYIE is the first SciIE benchmark for polymer materials, and we hope it will lead to more research efforts from the community on this challenging task. Our code and data are available on: https://github.com/jerry3027/PolyIE.

cs.CL↗

arXiv2023

The edge-girth-regularity of Wenger graphs

Let $n\ge 1$ be an integer and $\mathbb{F}_q$ be a finite field of characteristic $p$ with $q$ elements. In this paper, it is proved that the Wenger graph $W_n(q)$ and linearized Wenger graph $L_m(q)$ are edge-girth-regular $(v,k,g,λ)$-graphs, and the parameter $λ$ of graphs $W_n(q)$ and $L_m(q)$ is completely determined. Here, an edge-girth-regular graph $egr(v,k,g,λ)$ means a $k$-regular graph of order $v$ and girth $g$ satisfying that any edge is contained in $λ$ distinct $g$-cycles. As a direct corollary, we obtain the number of girth cycles of graph $W_n(q)$, and the lower bounds on the generalized Turán numbers $ex(n, C_{6}, \mathscr{C}_{5})$ and $ex(n, C_{8}, \mathscr{C}_{7})$, where $C_k$ is the cycle of length $k$ and $\mathscr{C}_k = \{C_3, C_4, \dots , C_k\}$.Moreover, there exist a family of $egr(2q^3,q,8,(q-1)^3(q-2))$-graphs for $q$ odd, and the order of graph $W_2(q)$ and extremal $egr(v,q,8,(q-1)^3(q-2))$-graph have same asymptotic order for $q$ odd.

math.CO↗

arXiv2023

Rank-DETR for High Quality Object Detection

Modern detection transformers (DETRs) use a set of object queries to predict a list of bounding boxes, sort them by their classification confidence scores, and select the top-ranked predictions as the final detection results for the given input image. A highly performant object detector requires accurate ranking for the bounding box predictions. For DETR-based detectors, the top-ranked bounding boxes suffer from less accurate localization quality due to the misalignment between classification scores and localization accuracy, thus impeding the construction of high-quality detectors. In this work, we introduce a simple and highly performant DETR-based object detector by proposing a series of rank-oriented designs, combinedly called Rank-DETR. Our key contributions include: (i) a rank-oriented architecture design that can prompt positive predictions and suppress the negative ones to ensure lower false positive rates, as well as (ii) a rank-oriented loss function and matching cost design that prioritizes predictions of more accurate localization accuracy during ranking to boost the AP under high IoU thresholds. We apply our method to improve the recent SOTA methods (e.g., H-DETR and DINO-DETR) and report strong COCO object detection results when using different backbones such as ResNet-$50$, Swin-T, and Swin-L, demonstrating the effectiveness of our approach. Code is available at \url{https://github.com/LeapLabTHU/Rank-DETR}.

cs.CV↗

arXiv2023

On the solutions of nonlocal 1-Laplacian equation with $L^1$-data

We study the solutions to a nonlocal 1-Laplacian equation given by $$ 2\text{P.V.}\int_{\mathbb{R}^N}\frac{u(x)-u(y)}{|u(x)-u(y)|} \frac{dy}{|x-y|^{N+s}}=f(x) \quad \textmd{for } x\in Ω, $$ with Dirichlet boundary condition $u(x)=0$ in $\mathbb R^N\backslash Ω$ and nonnegative $L^1$-data. By investigating the asymptotic behaviour of renormalized solutions $u_p$ to the nonlocal $p$-Laplacian equations as $p$ goes to $1^+$, we introduce a suitable definition of solutions and prove that the limit function $u$ of $\{u_p\}$ is a solution of the nonlocal $1$-Laplacian equation above.

math.AP↗

arXiv2023

Improving MIMO channel estimation via receive power feedback

Estimating the channel state is known to be an important problem in wireless networks. To this end, it matters to exploit all the available information to improve channel estimation accuracy as much as possible. It turns out that the problem of exploiting the information associated with the receive power feedback (e.g., the received signal strength indicator -RSSI-) has not been identified and solved; in this setup, the transmitter is assumed to receive feedback from all the receivers in presence. As shown in this paper, to solve this problem, classical estimation tools can be used. Using the corresponding MMSE is shown to be always beneficial, whereas the relevance of using the MAP estimator would depend on the operating SNR.

eess.SP↗

arXiv2023

Entropy solutions to the fully non-local diffusion equations

We consider the fully non-local diffusion equations with non-negative $L^1$-data. Based on the approximation and energy methods, we prove the existence and uniqueness of non-negative entropy solutions for such problems. In particular, our results are valid for the time-space fractional Laplacian equations.

math.AP↗

arXiv2023

SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process

Transformer Hawkes process models have shown to be successful in modeling event sequence data. However, most of the existing training methods rely on maximizing the likelihood of event sequences, which involves calculating some intractable integral. Moreover, the existing methods fail to provide uncertainty quantification for model predictions, e.g., confidence intervals for the predicted event's arrival time. To address these issues, we propose SMURF-THP, a score-based method for learning Transformer Hawkes process and quantifying prediction uncertainty. Specifically, SMURF-THP learns the score function of events' arrival time based on a score-matching objective that avoids the intractable computation. With such a learned score function, we can sample arrival time of events from the predictive distribution. This naturally allows for the quantification of uncertainty by computing confidence intervals over the generated samples. We conduct extensive experiments in both event type prediction and uncertainty quantification of arrival time. In all the experiments, SMURF-THP outperforms existing likelihood-based methods in confidence calibration while exhibiting comparable prediction accuracy.

cs.LG↗