arXiv subjects

Chang Liu

Publications and source records attributed to Chang Liu.

At least 757 records · Page 42Linked to original sources

arXiv2023

Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning

Contrastive learning-based video-language representation learning approaches, e.g., CLIP, have achieved outstanding performance, which pursue semantic interaction upon pre-defined video-text pairs. To clarify this coarse-grained global interaction and move a step further, we have to encounter challenging shell-breaking interactions for fine-grained cross-modal learning. In this paper, we creatively model video-text as game players with multivariate cooperative game theory to wisely handle the uncertainty during fine-grained semantic interaction with diverse granularity, flexible combination, and vague intensity. Concretely, we propose Hierarchical Banzhaf Interaction (HBI) to value possible correspondence between video frames and text words for sensitive and explainable cross-modal contrast. To efficiently realize the cooperative game of multiple video frames and multiple text words, the proposed method clusters the original video frames (text words) and computes the Banzhaf Interaction between the merged tokens. By stacking token merge modules, we achieve cooperative games at different semantic levels. Extensive experiments on commonly used text-video retrieval and video-question answering benchmarks with superior performances justify the efficacy of our HBI. More encouragingly, it can also serve as a visualization tool to promote the understanding of cross-modal interaction, which have a far-reaching impact on the community. Project page is available at https://jpthu17.github.io/HBI/.

cs.CV↗

arXiv2023

Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentation

Interactive segmentation enables users to segment as needed by providing cues of objects, which introduces human-computer interaction for many fields, such as image editing and medical image analysis. Typically, massive and expansive pixel-level annotations are spent to train deep models by object-oriented interactions with manually labeled object masks. In this work, we reveal that informative interactions can be made by simulation with semantic-consistent yet diverse region exploration in an unsupervised paradigm. Concretely, we introduce a Multi-granularity Interaction Simulation (MIS) approach to open up a promising direction for unsupervised interactive segmentation. Drawing on the high-quality dense features produced by recent self-supervised models, we propose to gradually merge patches or regions with similar features to form more extensive regions and thus, every merged region serves as a semantic-meaningful multi-granularity proposal. By randomly sampling these proposals and simulating possible interactions based on them, we provide meaningful interaction at multiple granularities to teach the model to understand interactions. Our MIS significantly outperforms non-deep learning unsupervised methods and is even comparable with some previous deep-supervised methods without any annotation.

cs.CV↗

arXiv2023

$L_2$BN: Enhancing Batch Normalization by Equalizing the $L_2$ Norms of Features

In this paper, we analyze batch normalization from the perspective of discriminability and find the disadvantages ignored by previous studies: the difference in $l_2$ norms of sample features can hinder batch normalization from obtaining more distinguished inter-class features and more compact intra-class features. To address this issue, we propose a simple yet effective method to equalize the $l_2$ norms of sample features. Concretely, we $l_2$-normalize each sample feature before feeding them into batch normalization, and therefore the features are of the same magnitude. Since the proposed method combines the $l_2$ normalization and batch normalization, we name our method $L_2$BN. The $L_2$BN can strengthen the compactness of intra-class features and enlarge the discrepancy of inter-class features. The $L_2$BN is easy to implement and can exert its effect without any additional parameters or hyper-parameters. We evaluate the effectiveness of $L_2$BN through extensive experiments with various models on image classification and acoustic scene classification tasks. The results demonstrate that the $L_2$BN can boost the generalization ability of various neural network models and achieve considerable performance improvements.

cs.CV↗

arXiv2023

A structured input-output approach to characterizing optimal perturbations in wall-bounded shear flows

This work builds upon recent work exploiting the notion of structured singular values to capture nonlinear interactions in the analysis of wall-bounded shear flows. In this context, the structured uncertainty can be interpreted in terms of the flow structures most likely to be amplified (the optimal perturbations). Here we further analyze these perturbations through a problem reformulation that decomposes this uncertainty into three components associated with the streamwise, wall-normal and spanwise velocity correlations. We then demonstrate that the structural features of these correlations are consistent with nonlinear optimal perturbations and results from secondary stability analysis associated with streamwise streaks. These results indicate the potential of structured input-output analysis for gaining insight into both linear and nonlinear behavior that can be used to inform sensing and control strategies for transitional wall-bounded shear flows.

physics.flu-dyn↗

arXiv2023

Self-driven photo-polarized water molecule triggered graphene-based photodetector

Flowing water can be used as an energy source for generators, providing a major part of the energy for daily life. However, water is rarely used for information or electronic devices. Herein, we present the feasibility of a polarized liquid-triggered photodetector in which polarized water is sandwiched between graphene and semiconductor. Due to the polarization and depolarization processes of water molecular under the driven of photogenerated carriers, a photo-sensitive current can be repeatably produced, resulting in a high-performance photodetector. The response wavelength of the photodetector can be finely tuned as a result of the free choice of semiconductors as there is no requirement of lattice match between graphene and the semiconductors. Under zero voltage bias, the responsivity and specific detectivity of Gr/NaCl (0.5 M)/N-GaN reaches values of 130.7 mA/W and 2.8*10^12 Jones under 350 nm illumination. Meanwhile, using a polar liquid photodetector can successfully read the photoplethysmography signals to produce accurate oxygen blood saturation and heart rate. Compared with the commercial pulse oximetry sensor, the average errors of oxygen saturation and heart rate in the designed photoplethysmography sensor are~1.9% and ~2.1%, respectively. This study transfers the message that water can be used as high-performance photodetector in informative industries.

physics.app-ph↗

arXiv2023

Mechanical motion tuned carrier transport characteristic of dynamic diode

Since the invention of dynamic diode, its physical properties and potential applications have attracted wide attentions. A lot of attempts have been made to harvest the rebounding current and voltage of dynamic diode. However, the underlying physical mechanism of its carrier transport characteristic was rarely explored carefully. Here, the electrical transport properties of the dynamic diode are systematically investigated with a mechanical motion tuned method, where the dynamic current-voltage curve shows a gentler growth trend compared to the static curve. The rebounding current increases with motion velocity and contact force, resulting in a reduced current with the same bias voltage and an oscillation current with a changing velocity and force. Besides, we propose a circuit model with an accurate mathematical formula expression to describe the oscillation current, where an imaginary parameter n0 is creatively added to the exponential growth term. This work shows a physical picture of adjust microscopic carrier motion with macroscopic mechanical motion, which provides strong theoretical support for designing dynamic diode devices with better performance in the future.

physics.app-ph↗

arXiv2023

Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation is typically inspired by class activation maps, which serve as pseudo masks with class-discriminative regions highlighted. Although tremendous efforts have been made to recall precise and complete locations for each class, existing methods still commonly suffer from the unsolicited Out-of-Candidate (OC) error predictions that not belongs to the label candidates, which could be avoidable since the contradiction with image-level class tags is easy to be detected. In this paper, we develop a group ranking-based Out-of-Candidate Rectification (OCR) mechanism in a plug-and-play fashion. Firstly, we adaptively split the semantic categories into In-Candidate (IC) and OC groups for each OC pixel according to their prior annotation correlation and posterior prediction correlation. Then, we derive a differentiable rectification loss to force OC pixels to shift to the IC group. Incorporating our OCR with seminal baselines (e.g., AffinityNet, SEAM, MCTformer), we can achieve remarkable performance gains on both Pascal VOC (+3.2%, +3.3%, +0.8% mIoU) and MS COCO (+1.0%, +1.3%, +0.5% mIoU) datasets with negligible extra training overhead, which justifies the effectiveness and generality of our OCR.

cs.CV↗

arXiv2023

Parallel Vertex Diffusion for Unified Visual Grounding

Unified visual grounding pursues a simple and generic technical route to leverage multi-task data with less task-specific design. The most advanced methods typically present boxes and masks as vertex sequences to model referring detection and segmentation as an autoregressive sequential vertex generation paradigm. However, generating high-dimensional vertex sequences sequentially is error-prone because the upstream of the sequence remains static and cannot be refined based on downstream vertex information, even if there is a significant location gap. Besides, with limited vertexes, the inferior fitting of objects with complex contours restricts the performance upper bound. To deal with this dilemma, we propose a parallel vertex generation paradigm for superior high-dimension scalability with a diffusion model by simply modifying the noise dimension. An intuitive materialization of our paradigm is Parallel Vertex Diffusion (PVD) to directly set vertex coordinates as the generation target and use a diffusion model to train and infer. We claim that it has two flaws: (1) unnormalized coordinate caused a high variance of loss value; (2) the original training objective of PVD only considers point consistency but ignores geometry consistency. To solve the first flaw, Center Anchor Mechanism (CAM) is designed to convert coordinates as normalized offset values to stabilize the training loss value. For the second flaw, Angle summation loss (ASL) is designed to constrain the geometry difference of prediction and ground truth vertexes for geometry-level consistency. Empirical results show that our PVD achieves state-of-the-art in both referring detection and segmentation, and our paradigm is more scalable and efficient than sequential vertex generation with high-dimension data.

cs.CV↗

arXiv2023

Self-consistent simulation of compressional Alfvén eigenmodes excited by runaway electrons

Alfvénic modes in the current quench (CQ) stage of the tokamak disruption have been observed in experiments. In DIII-D the excitation of these modes is associated with the presence of high-energy runaway electrons, and a strong mode excitation is often associated with the failure of RE plateau formation. In this work we present results of self-consistent kinetic-MHD simulations of RE-driven compressional Alfvén eigenmodes (CAEs) in DIII-D disruption scenarios, providing an explanation of the CQ modes. Simulation results reveal that high energy trapped REs can have resonance with the Alfvén mode through their precession motion, and the resonance frequency is proportional to the energy of REs. The mode frequencies and their relationship with the RE energy are consistent with experimental observation. The perturbed magnetic fields from the modes can lead to spatial diffusion of runaway electrons including the nonresonant passing ones, thus providing the theoretical basis for a potential approach for runaway electron mitigation.

physics.plasm-ph↗

arXiv2023

Confusion noise from Galactic binaries for Taiji

Gravitational waves (GWs) from tens of millions of compact binaries in our Milky Way enter the milli-Hertz band of space-based detection. The majority of them cannot be resolved individually, resulting in a foreground confusion noise for Laser Interferometer Space Antenna (LISA). The concept of Taiji mission is similar to LISA's with slightly better sensitivity, which means that the galactic GW signals will also affect the detection with Taiji. Here we generate the GW signals from 29.8 million galactic binaries for Taiji and subtract the `resolvable' sources. The confusion noise is estimated and fitted in an analytic form with 6-month, 1-year, 2-year and 4-year observation time. We find that the full sensitivity curve is slightly lower for Taiji than for LISA at frequencies of $\leq 0.8$ mHz and around 2~mHz. For a 4-year lifetime, more than 29 thousand sources are resolvable with Taiji. Compared to LISA, Taiji can subtract $\sim 20 \%$ more sources and the distribution of them in our Milky Way is consistent with that of the resolvable sources with LISA. At frequencies around 2~mHz or with the chirp masses ranging from $0.2 M_\odot$ to $0.4 M_\odot$, more sources become resolvable with Taiji.

astro-ph.IM↗

arXiv2023

Image as Set of Points

What is an image and how to extract latent features? Convolutional Networks (ConvNets) consider an image as organized pixels in a rectangular shape and extract features via convolutional operation in local region; Vision Transformers (ViTs) treat an image as a sequence of patches and extract features via attention mechanism in a global range. In this work, we introduce a straightforward and promising paradigm for visual representation, which is called Context Clusters. Context clusters (CoCs) view an image as a set of unorganized points and extract features via simplified clustering algorithm. In detail, each point includes the raw feature (e.g., color) and positional information (e.g., coordinates), and a simplified clustering algorithm is employed to group and extract deep features hierarchically. Our CoCs are convolution- and attention-free, and only rely on clustering algorithm for spatial interaction. Owing to the simple design, we show CoCs endow gratifying interpretability via the visualization of clustering process. Our CoCs aim at providing a new perspective on image and visual representation, which may enjoy broad applications in different domains and exhibit profound insights. Even though we are not targeting SOTA performance, COCs still achieve comparable or even better results than ConvNets or ViTs on several benchmarks. Codes are available at: https://github.com/ma-xu/Context-Cluster.

cs.CV↗

arXiv2023

Revocable Deep Reinforcement Learning with Affinity Regularization for Outlier-Robust Graph Matching

Graph matching (GM) has been a building block in various areas including computer vision and pattern recognition. Despite recent impressive progress, existing deep GM methods often have obvious difficulty in handling outliers, which are ubiquitous in practice. We propose a deep reinforcement learning based approach RGM, whose sequential node matching scheme naturally fits the strategy for selective inlier matching against outliers. A revocable action framework is devised to improve the agent's flexibility against the complex constrained GM. Moreover, we propose a quadratic approximation technique to regularize the affinity score, in the presence of outliers. As such, the agent can finish inlier matching timely when the affinity score stops growing, for which otherwise an additional parameter i.e. the number of inliers is needed to avoid matching outliers. In this paper, we focus on learning the back-end solver under the most general form of GM: the Lawler's QAP, whose input is the affinity matrix. Especially, our approach can also boost existing GM methods that use such input. Experiments on multiple real-world datasets demonstrate its performance regarding both accuracy and robustness.

cs.CV↗

arXiv2023

A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking

The robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts, which becomes an important research problem in the development of deep learning. Although new deep learning methods and robustness improvement techniques have been constantly proposed, the robustness evaluations of existing methods are often inadequate due to their rapid development, diverse noise patterns, and simple evaluation metrics. Without thorough robustness evaluations, it is hard to understand the advances in the field and identify the effective methods. In this paper, we establish a comprehensive robustness benchmark called \textbf{ARES-Bench} on the image classification task. In our benchmark, we evaluate the robustness of 55 typical deep learning models on ImageNet with diverse architectures (e.g., CNNs, Transformers) and learning algorithms (e.g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets. Using robustness curves as the major evaluation criteria, we conduct large-scale experiments and draw several important findings, including: 1) there is an inherent trade-off between adversarial and natural robustness for the same model architecture; 2) adversarial training effectively improves adversarial robustness, especially when performed on Transformer architectures; 3) pre-training significantly improves natural robustness based on more training data or self-supervised learning. Based on ARES-Bench, we further analyze the training tricks in large-scale adversarial training on ImageNet. By designing the training settings accordingly, we achieve the new state-of-the-art adversarial robustness. We have made the benchmarking results and code platform publicly available.

cs.CV↗

arXiv2023

Ultra-low Precision Multiplication-free Training for Deep Neural Networks

The training for deep neural networks (DNNs) demands immense energy consumption, which restricts the development of deep learning as well as increases carbon emissions. Thus, the study of energy-efficient training for DNNs is essential. In training, the linear layers consume the most energy because of the intense use of energy-consuming full-precision (FP32) multiplication in multiply-accumulate (MAC). The energy-efficient works try to decrease the precision of multiplication or replace the multiplication with energy-efficient operations such as addition or bitwise shift, to reduce the energy consumption of FP32 multiplications. However, the existing energy-efficient works cannot replace all of the FP32 multiplications during both forward and backward propagation with low-precision energy-efficient operations. In this work, we propose an Adaptive Layer-wise Scaling PoT Quantization (ALS-POTQ) method and a Multiplication-Free MAC (MF-MAC) to replace all of the FP32 multiplications with the INT4 additions and 1-bit XOR operations. In addition, we propose Weight Bias Correction and Parameterized Ratio Clipping techniques for stable training and improving accuracy. In our training scheme, all of the above methods do not introduce extra multiplications, so we reduce up to 95.8% of the energy consumption in linear layers during training. Experimentally, we achieve an accuracy degradation of less than 1% for CNN models on ImageNet and Transformer model on the WMT En-De task. In summary, we significantly outperform the existing methods for both energy efficiency and accuracy.

cs.LG↗

arXiv2023

Particle-based Online Bayesian Sampling

Online optimization has gained increasing interest due to its capability of tracking real-world streaming data. Although online optimization methods have been widely studied in the setting of frequentist statistics, few works have considered online optimization with the Bayesian sampling problem. In this paper, we study an Online Particle-based Variational Inference (OPVI) algorithm that uses a set of particles to represent the approximating distribution. To reduce the gradient error caused by the use of stochastic approximation, we include a sublinear increasing batch-size method to reduce the variance. To track the performance of the OPVI algorithm with respect to a sequence of dynamically changing target posterior, we provide a detailed theoretical analysis from the perspective of Wasserstein gradient flow with a dynamic regret. Synthetic and Bayesian Neural Network experiments show that the proposed algorithm achieves better results than naively applying existing Bayesian sampling methods in the online setting.

cs.LG↗

arXiv2023

2D-XY ferromagnetism with high transition temperature in Janus monolayer V$_{2}$XN (X = P, As)

Two-dimensional (2D) XY magnets with easy magnetization planes support the nontrivial topological spin textures whose dissipationless transport is highly desirable for 2D spintronic devices. Here, we predicted that Janus monolayer V$_{2}$XN (X = P, As) with a square lattice are 2D-XY ferromagnets by first-principles calculations. Both the magnetocrystalline anisotropy and magnetic shape anisotropy favor an in-plane magnetization, leading to an easy magnetization $xy$-plane in Janus monolayer V$_{2}$XN. Resting on the Monte Carlo simulations, we observed the Berezinskii-Kosterlitz-Thouless (BKT) phase transition in monolayer V$_{2}$XN with transition temperature $T_{\rm BKT}$ being above the room temperature. Especially, monolayer V$_{2}$AsN has a magnetic anisotropy energy (MAE) of 292.0 $μ$eV per V atom and a $T_{\rm BKT}$ of 434 K, which is larger than that of monolayer V$_{2}$PN. Moreover, a tensile strain of 5\% can further improve the $T_{\rm BKT}$ of monolayer V$_{2}$XN to be above 500 K. Our results indicated that Janus monolayer V$_{2}$XN (X = P, As) were candidate materials to realize high-temperature 2D-XY ferromagnetism for spintronics applications.

cond-mat.mtrl-sci↗

arXiv2023

SN 2021zny: an early flux excess combined with late-time oxygen emission suggests a double white dwarf merger event

We present a photometric and spectroscopic analysis of the ultra-luminous and slowly evolving 03fg-like Type Ia SN 2021zny. Our observational campaign starts from $\sim5.3$ hours after explosion (making SN 2021zny one of the earliest observed members of its class), with dense multi-wavelength coverage from a variety of ground- and space-based telescopes, and is concluded with a nebular spectrum $\sim10$ months after peak brightness. SN 2021zny displayed several characteristics of its class, such as the peak brightness ($M_{B}=-19.95$ mag), the slow decline ($Δm_{15}(B) = 0.62$ mag), the blue early-time colours, the low ejecta velocities and the presence of significant unburned material above the photosphere. However, a flux excess for the first $\sim1.5$ days after explosion is observed in four photometric bands, making SN 2021zny the third 03fg-like event with this distinct behavior, while its $+313$ d spectrum shows prominent [O I] lines, a very unusual characteristic of thermonuclear SNe. The early flux excess can be explained as the outcome of the interaction of the ejecta with $\sim0.04\:\mathrm{M_{\odot}}$ of H/He-poor circumstellar material at a distance of $\sim10^{12}$ cm, while the low ionization state of the late-time spectrum reveals low abundances of stable iron-peak elements. All our observations are in accordance with a progenitor system of two carbon/oxygen white dwarfs that undergo a merger event, with the disrupted white dwarf ejecting carbon-rich circumstellar material prior to the primary white dwarf detonation.

astro-ph.HE↗

arXiv2023

A unified gas-kinetic particle method for frequency-dependent radiative transfer equations with isotropic scattering process on unstructured mesh

In this paper, we extend the unified kinetic particle (UGKP) method to the frequency-dependent radiative transfer equation with both absorption-emission and scattering processes. The extended UGKP method could not only capture the diffusion and free transport limit, but also provide a smooth transition in the physical and frequency space in the regime between the above two limits. The proposed scheme has the properties of asymptotic-preserving, regime-adaptive, and entropy-preserving, which make it an accurate and efficient scheme in the simulation of multiscale photon transport problems. The methodology of scheme construction is a coupled evolution of macroscopic energy equation and the microscopic radiant intensity equation, where the numerical flux in macroscopic energy equation and the closure in microscopic radiant intensity equation are constructed based on the integral solution. Both numerical dissipation and computational complexity are well controlled especially in the optical thick regime. A 2D multi-thread code on a general unstructured mesh has been developed. Several numerical tests have been simulated to verify the numerical scheme and code, covering a wide range of flow regimes. The numerical scheme and code that we developed are highly demanded and widely applicable in the high energy density engineering applications.

physics.comp-ph↗