Search arXiv⌕ Search

arXiv subjects

Kai Zhang

Publications and source records attributed to Kai Zhang.

At least 469 records · Page 26Linked to original sources
arXiv2023

ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching

Graph Neural Networks (GNNs) with attention have been successfully applied for learning visual feature matching. However, current methods learn with complete graphs, resulting in a quadratic complexity in the number of features. Motivated by a prior observation that self- and cross- attention matrices converge to a sparse representation, we propose ClusterGNN, an attentional GNN architecture which operates on clusters for learning the feature matching task. Using a progressive clustering module we adaptively divide keypoints into different subgraphs to reduce redundant connectivity, and employ a coarse-to-fine paradigm for mitigating miss-classification within images. Our approach yields a 59.7% reduction in runtime and 58.4% reduction in memory consumption for dense detection, compared to current state-of-the-art GNN-based matching, while achieving a competitive performance on various computer vision tasks.

cs.CV↗
arXiv2023

QVRF: A Quantization-error-aware Variable Rate Framework for Learned Image Compression

Learned image compression has exhibited promising compression performance, but variable bitrates over a wide range remain a challenge. State-of-the-art variable rate methods compromise the loss of model performance and require numerous additional parameters. In this paper, we present a Quantization-error-aware Variable Rate Framework (QVRF) that utilizes a univariate quantization regulator a to achieve wide-range variable rates within a single model. Specifically, QVRF defines a quantization regulator vector coupled with predefined Lagrange multipliers to control quantization error of all latent representation for discrete variable rates. Additionally, the reparameterization method makes QVRF compatible with a round quantizer. Exhaustive experiments demonstrate that existing fixed-rate VAE-based methods equipped with QVRF can achieve wide-range continuous variable rates within a single model without significant performance degradation. Furthermore, QVRF outperforms contemporary variable-rate methods in rate-distortion performance with minimal additional parameters.

eess.IV↗
arXiv2023

Probing the electronic topological transitions of WTe2 under pressure using ultrafast spectroscopy

We investigate the nonequilibrium photocarrier dynamics of WTe2 under pressure using the optical pump-probe spectroscopy. The pressure dependences of the electronic relaxation manifest anomalous changes around 0.8, 3.5, and 6 GPa, indicating the abruptions in the electron-phonon interactions. In addition, the coherent phonon oscillations originating from shear mode suddenly disappears above 3.5 GPa, which marks the onset of Td-1T' structural phase transition. Supported by the theoretical calculation, we unveil the electronic topological transitions (ETTs), especially an emergence of a new type-II Weyl point for Td-WTe2 under pressure. Our work demonstrates a novel route to probe the ETTs under pressure.

cond-mat.str-el↗
arXiv2023

Securing Biomedical Images from Unauthorized Training with Anti-Learning Perturbation

The volume of open-source biomedical data has been essential to the development of various spheres of the healthcare community since more `free' data can provide individual researchers more chances to contribute. However, institutions often hesitate to share their data with the public due to the risk of data exploitation by unauthorized third parties for another commercial usage (e.g., training AI models). This phenomenon might hinder the development of the whole healthcare research community. To address this concern, we propose a novel approach termed `unlearnable biomedical image' for protecting biomedical data by injecting imperceptible but delusive noises into the data, making them unexploitable for AI models. We formulate the problem as a bi-level optimization and propose three kinds of anti-learning perturbation generation approaches to solve the problem. Our method is an important step toward encouraging more institutions to contribute their data for the long-term development of the research community.

cs.LG↗
arXiv2023

LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval

Retrieval models based on dense representations in semantic space have become an indispensable branch for first-stage retrieval. These retrievers benefit from surging advances in representation learning towards compressive global sequence-level embeddings. However, they are prone to overlook local salient phrases and entity mentions in texts, which usually play pivot roles in first-stage retrieval. To mitigate this weakness, we propose to make a dense retriever align a well-performing lexicon-aware representation model. The alignment is achieved by weakened knowledge distillations to enlighten the retriever via two aspects -- 1) a lexicon-augmented contrastive objective to challenge the dense encoder and 2) a pair-wise rank-consistent regularization to make dense model's behavior incline to the other. We evaluate our model on three public benchmarks, which shows that with a comparable lexicon-aware retriever as the teacher, our proposed dense one can bring consistent and significant improvements, and even outdo its teacher. In addition, we found our improvement on the dense retriever is complementary to the standard ranker distillation, which can further lift state-of-the-art performance.

cs.CL↗
arXiv2023

Backdoor Attacks to Pre-trained Unified Foundation Models

The rise of pre-trained unified foundation models breaks down the barriers between different modalities and tasks, providing comprehensive support to users with unified architectures. However, the backdoor attack on pre-trained models poses a serious threat to their security. Previous research on backdoor attacks has been limited to uni-modal tasks or single tasks across modalities, making it inapplicable to unified foundation models. In this paper, we make proof-of-concept level research on the backdoor attack for pre-trained unified foundation models. Through preliminary experiments on NLP and CV classification tasks, we reveal the vulnerability of these models and suggest future research directions for enhancing the attack approach.

cs.CR↗
arXiv2023

Building Shortcuts between Distant Nodes with Biaffine Mapping for Graph Convolutional Networks

Multiple recent studies show a paradox in graph convolutional networks (GCNs), that is, shallow architectures limit the capability of learning information from high-order neighbors, while deep architectures suffer from over-smoothing or over-squashing. To enjoy the simplicity of shallow architectures and overcome their limits of neighborhood extension, in this work, we introduce Biaffine technique to improve the expressiveness of graph convolutional networks with a shallow architecture. The core design of our method is to learn direct dependency on long-distance neighbors for nodes, with which only one-hop message passing is capable of capturing rich information for node representation. Besides, we propose a multi-view contrastive learning method to exploit the representations learned from long-distance dependencies. Extensive experiments on nine graph benchmark datasets suggest that the shallow biaffine graph convolutional networks (BAGCN) significantly outperforms state-of-the-art GCNs (with deep or shallow architectures) on semi-supervised node classification. We further verify the effectiveness of biaffine design in node representation learning and the performance consistency on different sizes of training data.

cs.LG↗
arXiv2023

Aerodynamic performance enhancement and noise reduction for Darrieus vertical axis wind turbines: V-shaped blades and trailing-edge serrations

The Darrieus vertical axis wind turbines (VAWTs) are one of the mainstream devices for wind energy utilization in the urban areas. The market's pursuit of high wind energy conversion efficiency promotes the research on improving the wind turbine power coefficient while reducing noise. This paper makes further investigation on the aerodynamics and aeroacoustics of a small VAWT with V-shaped blades and trailing-edge serrations. The feasibility of utilizing the Reynolds-Averaged Navier-Stokes SST k-omega turbulence model and the FW-H method is verified against experiments. The studied V-shaped blades can effectively improve the power performance of VAWT over a wide range of tip speed ratios under normal wind speed conditions, and the trailing-edge serrations will also slightly increase the power output of V-bladed VAWT at the optimal tip speed ratio. The power coefficient of the V-bladed wind turbine with trailing-edge serrations is about 28.3% higher than that of the original turbine. In addition, a dumbbell-shaped noise directivity distribution was first discovered in the VAWT compared with the traditional elliptical distribution. The V-bladed VAWT generated less low-frequency noise and the trailing-edge serrations realized the expected noise reduction effect. Practically, this study proposes a feasible solution for the design of high-efficiency and low-noise wind turbines.

physics.flu-dyn↗
arXiv2023

Fairly Predicting Graft Failure in Liver Transplant for Organ Assigning

Liver transplant is an essential therapy performed for severe liver diseases. The fact of scarce liver resources makes the organ assigning crucial. Model for End-stage Liver Disease (MELD) score is a widely adopted criterion when making organ distribution decisions. However, it ignores post-transplant outcomes and organ/donor features. These limitations motivate the emergence of machine learning (ML) models. Unfortunately, ML models could be unfair and trigger bias against certain groups of people. To tackle this problem, this work proposes a fair machine learning framework targeting graft failure prediction in liver transplant. Specifically, knowledge distillation is employed to handle dense and sparse features by combining the advantages of tree models and neural networks. A two-step debiasing method is tailored for this framework to enhance fairness. Experiments are conducted to analyze unfairness issues in existing models and demonstrate the superiority of our method in both prediction and fairness performance.

cs.AI↗
arXiv2023

Aerodynamic characterization of two tandem wind turbines under yaw misalignment control using actuator line model

Yaw control has proven to be promising in alleviating the wake effects that plague the efficiency of wind farms. In this work, the actuator line modeling (ALM) method is adopted to simulate the flows over two tandem turbines distanced by $3 - 7$ rotor diameters, with the yaw angle of the upstream rotor varying from $γ_1=0^{\circ}$ to $50^{\circ}$. The aim is to provide a comprehensive aerodynamic characterization of a simple wind farm under yaw misalignment. With increasing yaw angle, the power generated by the downstream rotor increases, compensating the power loss in the upstream rotor, and resulting in higher total power of the two turbines than that without yaw control. The maximum power output is achieved as the upstream wake of the yawed rotor is redirected away from the downstream rotor plane. Behind the downstream rotor, the secondary steering phenomenon is observed, where the wake is also redirected from the centerline. The use of the actuator line model also reveal unsteady aerodynamic characteristics that can not be captured by lower-fidelity models. For the upstream rotor, the yaw misalignment results in time-varying change in the local angle of attack on the blade, giving rise to unsteady loading. The downstream rotor is partially submerged in the deflected wake incurred by the yawed upstream rotor. As the blade revolves into and out of the wake deficit, the blade experiences cyclic loading, leading to even stronger fluctuations in the aerodynamic loads than the upstream rotor. These analysis provides a comprehensive understanding of the yaw control effects on the two tandem rotors from the perspectives of aerodynamic performance, wake profiles, and unsteady characteristics. The insights gained from the study can aid the design of collective yaw control strategies of wind farms, and lay the foundation for assessing the fatigue damage associated with yaw misalignment.

physics.flu-dyn↗
arXiv2023

Scalable Causal Structure Learning: Scoping Review of Traditional and Deep Learning Algorithms and New Opportunities in Biomedicine

Causal structure learning refers to a process of identifying causal structures from observational data, and it can have multiple applications in biomedicine and health care. This paper provides a practical review and tutorial on scalable causal structure learning models with examples of real-world data to help health care audiences understand and apply them. We reviewed traditional (combinatorial and score-based methods) for causal structure discovery and machine learning-based schemes. We also highlighted recent developments in biomedicine where causal structure learning can be applied to discover structures such as gene networks, brain connectivity networks, and those in cancer epidemiology. We also compared the performance of traditional and machine learning-based algorithms for causal discovery over some benchmark data sets. Machine learning-based approaches, including deep learning, have many advantages over traditional approaches, such as scalability, including a greater number of variables, and potentially being applied in a wide range of biomedical applications, such as genetics, if sufficient data are available. Furthermore, these models are more flexible than traditional models and are poised to positively affect many applications in the future.

cs.LG↗
arXiv2022

GreenEyes: An Air Quality Evaluating Model based on WaveNet

Accompanying rapid industrialization, humans are suffering from serious air pollution problems. The demand for air quality prediction is becoming more and more important to the government's policy-making and people's daily life. In this paper, We propose GreenEyes -- a deep neural network model, which consists of a WaveNet-based backbone block for learning representations of sequences and an LSTM with a Temporal Attention module for capturing the hidden interactions between features of multi-channel inputs. To evaluate the effectiveness of our proposed method, we carry out several experiments including an ablation study on our collected and preprocessed air quality data near HKUST. The experimental results show our model can effectively predict the air quality level of the next timestamp given any segment of the air quality data from the data set. We have also released our standalone dataset at https://github.com/AI-Huang/IAQI_Dataset The model and code for this paper are publicly available at https://github.com/AI-Huang/AirEvaluation

cs.LG↗
arXiv2022

Spectral patterns of elastic transmission eigenfunctions: boundary localisation, surface resonance and stress concentration

We present a comprehensive study of new discoveries on the spectral patterns of elastic transmission eigenfunctions, including boundary localisation, surface resonance, and stress concentration. In the case where the domain is radial and the underlying parameters are constant, we give rigorous justifications and derive a thorough understanding of those intriguing geometric and physical patterns. We also present numerical examples to verify that the same results hold in general geometric and parameter setups.

math.AP↗
arXiv2022

Pairwise Nonlinear Dependence Analysis of Genomic Data

In The Cancer Genome Atlas (TCGA) data set, there are many interesting nonlinear dependencies between pairs of genes that reveal important relationships and subtypes of cancer. Such genomic data analysis requires a rapid, powerful and interpretable detection process, especially in a high-dimensional environment. We study the nonlinear patterns among the expression of pairs of genes from TCGA using a powerful tool called Binary Expansion Testing. We find many nonlinear patterns, some of which are driven by known cancer subtypes, some of which are novel.

stat.AP↗
arXiv2022

Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) predicts sentiment polarity towards a specific aspect in the given sentence. While pre-trained language models such as BERT have achieved great success, incorporating dynamic semantic changes into ABSA remains challenging. To this end, in this paper, we propose to address this problem by Dynamic Re-weighting BERT (DR-BERT), a novel method designed to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT layers as a primary encoder to grasp the overall semantic of the sentence and then fine-tune it by incorporating a lightweight Dynamic Re-weighting Adapter (DRA). Note that the DRA can pay close attention to a small region of the sentences at each step and re-weigh the vitally important words for better aspect-aware sentiment understanding. Finally, experimental results on three benchmark datasets demonstrate the effectiveness and the rationality of our proposed model and provide good interpretable insights for future semantic modeling.

cs.CL↗
arXiv2022

Recurrent Video Restoration Transformer with Guided Deformable Attention

Video restoration aims at restoring multiple high-quality frames from multiple low-quality frames. Existing video restoration methods generally fall into two extreme cases, i.e., they either restore all frames in parallel or restore the video frame by frame in a recurrent way, which would result in different merits and drawbacks. Typically, the former has the advantage of temporal information fusion. However, it suffers from large model size and intensive memory consumption; the latter has a relatively small model size as it shares parameters across frames; however, it lacks long-range dependency modeling ability and parallelizability. In this paper, we attempt to integrate the advantages of the two cases by proposing a recurrent video restoration transformer, namely RVRT. RVRT processes local neighboring frames in parallel within a globally recurrent framework which can achieve a good trade-off between model size, effectiveness, and efficiency. Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature. Within each clip, different frame features are jointly updated with implicit feature aggregation. Across different clips, the guided deformable attention is designed for clip-to-clip alignment, which predicts multiple relevant locations from the whole inferred clip and aggregates their features by the attention mechanism. Extensive experiments on video super-resolution, deblurring, and denoising show that the proposed RVRT achieves state-of-the-art performance on benchmark datasets with balanced model size, testing memory and runtime.

cs.CV↗
arXiv2022

Constraining galaxy-halo connection with high-order statistics

We investigate using three-point statistics in constraining the galaxy-halo connection. We show that for some galaxy samples, the constraints on the halo occupation distribution parameters are dominated by the three-point function signal (over its two-point counterpart). We demonstrate this on mock catalogs corresponding to the Luminous Red Galaxies (LRGs), Emission-Line Galaxies (ELG), and quasars (QSOs) targeted by the Dark Energy Spectroscopic Instrument (DESI) Survey. The projected three-point function for triangle sides less up to 20$h^{-1}$ Mpc measured from a cubic Gpc of data can constrain the characteristic minimum mass of the LRGs with a precision of $0.46$ %. For comparison, similar constraints from the projected two-point function are $1.55$ %. The improvements for the ELGs and QSOs targets are more modest. In the case of the QSOs it is caused by the high shot-noise of the sample, and in the case of the ELGs, this is caused by the range of halo masses of the host halos. The most time-consuming part of our pipeline is the measurement of the three-point functions. We adopt a tabulation method, proposed in earlier works for the two-point function, to reduce significantly the required compute time for the three-point analysis.

astro-ph.CO↗