Search arXiv⌕ Search

arXiv subjects

Lei Wang

Publications and source records attributed to Lei Wang.

At least 721 records · Page 40Linked to original sources
arXiv2023

Beamforming Design and Performance Evaluation for RIS-aided Localization using LEO Satellite Signals

The growing availability of low-Earth orbit (LEO) satellites, coupled with the anticipated widespread deployment of reconfigurable intelligent surfaces (RISs), opens up promising prospects for new localization paradigms. This paper studies RIS-aided localization using LEO satellite signals. The Cramér-Rao bound of the considered localization problem is derived, based on which an optimal RIS beamforming design that minimizes the derived bound is proposed. Numerical results demonstrate the superiority of the proposed beamforming scheme over benchmark alternatives, while also revealing that the synergy between LEO satellites and RISs holds the promise of achieving localization accuracy at the meter or even sub-meter level.

eess.SP↗
arXiv2023

Enhancing Sample Utilization through Sample Adaptive Augmentation in Semi-Supervised Learning

In semi-supervised learning, unlabeled samples can be utilized through augmentation and consistency regularization. However, we observed certain samples, even undergoing strong augmentation, are still correctly classified with high confidence, resulting in a loss close to zero. It indicates that these samples have been already learned well and do not provide any additional optimization benefits to the model. We refer to these samples as ``naive samples". Unfortunately, existing SSL models overlook the characteristics of naive samples, and they just apply the same learning strategy to all samples. To further optimize the SSL model, we emphasize the importance of giving attention to naive samples and augmenting them in a more diverse manner. Sample adaptive augmentation (SAA) is proposed for this stated purpose and consists of two modules: 1) sample selection module; 2) sample augmentation module. Specifically, the sample selection module picks out {naive samples} based on historical training information at each epoch, then the naive samples will be augmented in a more diverse manner in the sample augmentation module. Thanks to the extreme ease of implementation of the above modules, SAA is advantageous for being simple and lightweight. We add SAA on top of FixMatch and FlexMatch respectively, and experiments demonstrate SAA can significantly improve the models. For example, SAA helped improve the accuracy of FixMatch from 92.50% to 94.76% and that of FlexMatch from 95.01% to 95.31% on CIFAR-10 with 40 labels.

cs.CV↗
arXiv2023

Trion states and quantum criticality of attractive SU(3) Dirac fermions

We perform the projector quantum Monte Carlo (QMC) simulation to study the trion formation and quantum phase transition in the half-filled attractive SU(3) Hubbard model on a honeycomb lattice. With increasing attractive Hubbard interaction, our simulations demonstrate a continuous quantum phase transition from the semimetal to charge density wave (CDW) at the critical coupling $U_c/t=-1.52(2)$. The critical exponents $ν=0.82(3)$ and $η=0.58(4)$ determined by the QMC simulation remarkably disagree with those of the $N=3$ chiral Ising universality class suggested by the effective Gross-Neveu-Yukawa (GNY) theory, but coincide with the $N=1$ chiral Ising universality class. In the CDW phase, we show that on-site and off-site trions coexist and the off-site trion forms a local bond state. Our work not only illustrates the formation of off-site trions in two-dimensional Hubbard model, but also raises doubts about the extent of applicability of GNY model on the attractive SU(3) Dirac fermions.

cond-mat.quant-gas↗
arXiv2023

HybPSF: Hybrid PSF reconstruction for the observed JWST NIRCam image

The James Webb Space Telescope (JWST) ushers in a new era of astronomical observation and discovery, offering unprecedented precision in a variety of measurements such as photometry, astrometry, morphology, and shear measurement. Accurate point spread function (PSF) models are crucial for many of these measurements. In this paper, we introduce a hybrid PSF construction method called HybPSF for JWST NIRCam imaging data. HybPSF combines the WebbPSF software, which simulates the PSF for JWST, with observed data to produce more accurate and reliable PSF models. We apply this method to the SMACS J0723 imaging data and construct supplementary structures from residuals obtained by subtracting the WebbPSF PSF model from the data. Our results show that HybPSF significantly reduces discrepancies between the PSF model and the data compared to WebbPSF. Specifically, the PSF shape parameter ellipticity and size comparisons indicate that HybPSF improves precision by a factor of approximately 10 for \$R^2\$ and \$50\%\$ for \$e\$. This improvement has important implications for astronomical measurements using JWST NIRCam imaging data.

astro-ph.IM↗
arXiv2023

Riemannian Optimistic Algorithms

In this paper, we consider Riemannian online convex optimization with dynamic regret. First, we propose two novel algorithms, namely the Riemannian Online Optimistic Gradient Descent (R-OOGD) and the Riemannian Adaptive Online Optimistic Gradient Descent (R-AOOGD), which combine the advantages of classical optimistic algorithms with the rich geometric properties of Riemannian manifolds. We analyze the dynamic regrets of the R-OOGD and R-AOOGD in terms of regularity of the sequence of cost functions and comparators. Next, we apply the R-OOGD to Riemannian zero-sum games, leading to the Riemannian Optimistic Gradient Descent Ascent algorithm (R-OGDA). We analyze the average iterate and best-iterate of the R-OGDA in seeking Nash equilibrium for a two-player, zero-sum, g-convex-concave games. We also prove the last-iterate convergence of the R-OGDA for g-strongly convex-strongly concave problems. Our theoretical analysis shows that all proposed algorithms achieve results in regret and convergence that match their counterparts in Euclidean spaces. Finally, we conduct several experiments to verify our theoretical findings.

math.OC↗
arXiv2023

Defensive Few-shot Learning

This paper investigates a new challenging problem called defensive few-shot learning in order to learn a robust few-shot model against adversarial attacks. Simply applying the existing adversarial defense methods to few-shot learning cannot effectively solve this problem. This is because the commonly assumed sample-level distribution consistency between the training and test sets can no longer be met in the few-shot setting. To address this situation, we develop a general defensive few-shot learning (DFSL) framework to answer the following two key questions: (1) how to transfer adversarial defense knowledge from one sample distribution to another? (2) how to narrow the distribution gap between clean and adversarial examples under the few-shot setting? To answer the first question, we propose an episode-based adversarial training mechanism by assuming a task-level distribution consistency to better transfer the adversarial defense knowledge. As for the second question, within each few-shot task, we design two kinds of distribution consistency criteria to narrow the distribution gap between clean and adversarial examples from the feature-wise and prediction-wise perspectives, respectively. Extensive experiments demonstrate that the proposed framework can effectively make the existing few-shot models robust against adversarial attacks. Code is available at https://github.com/WenbinLee/DefensiveFSL.git.

cs.CV↗
arXiv2023

A Multilayer Framework for Online Metric Learning

Online metric learning has been widely applied in classification and retrieval. It can automatically learn a suitable metric from data by restricting similar instances to be separated from dissimilar instances with a given margin. However, the existing online metric learning algorithms have limited performance in real-world classifications, especially when data distributions are complex. To this end, this paper proposes a multilayer framework for online metric learning to capture the nonlinear similarities among instances. Different from the traditional online metric learning, which can only learn one metric space, the proposed Multi-Layer Online Metric Learning (MLOML) takes an online metric learning algorithm as a metric layer and learns multiple hierarchical metric spaces, where each metric layer follows a nonlinear layers for the complicated data distribution. Moreover, the forward propagation (FP) strategy and backward propagation (BP) strategy are employed to train the hierarchical metric layers. To build a metric layer of the proposed MLOML, a new Mahalanobis-based Online Metric Learning (MOML) algorithm is presented based on the passive-aggressive strategy and one-pass triplet construction strategy. Furthermore, in a progressively and nonlinearly learning way, MLOML has a stronger learning ability than traditional online metric learning in the case of limited available training data. To make the learning process more explainable and theoretically guaranteed, theoretical analysis is provided. The proposed MLOML enjoys several nice properties, indeed learns a metric progressively, and performs better on the benchmark datasets. Extensive experiments with different settings have been conducted to verify these properties of the proposed MLOML.

cs.CV↗
arXiv2023

Trip-ROMA: Self-Supervised Learning with Triplets and Random Mappings

Contrastive self-supervised learning (SSL) methods, such as MoCo and SimCLR, have achieved great success in unsupervised visual representation learning. They rely on a large number of negative pairs and thus require either large memory banks or large batches. Some recent non-contrastive SSL methods, such as BYOL and SimSiam, attempt to discard negative pairs and have also shown remarkable performance. To avoid collapsed solutions caused by not using negative pairs, these methods require non-trivial asymmetry designs. However, in small data regimes, we can not obtain a sufficient number of negative pairs or effectively avoid the over-fitting problem when negatives are not used at all. To address this situation, we argue that negative pairs are still important but one is generally sufficient for each positive pair. We show that a simple Triplet-based loss (Trip) can achieve surprisingly good performance without requiring large batches or asymmetry designs. Moreover, to alleviate the over-fitting problem in small data regimes and further enhance the effect of Trip, we propose a simple plug-and-play RandOm MApping (ROMA) strategy by randomly mapping samples into other spaces and requiring these randomly projected samples to satisfy the same relationship indicated by the triplets. Integrating the triplet-based loss with random mapping, we obtain the proposed method Trip-ROMA. Extensive experiments, including unsupervised representation learning and unsupervised few-shot learning, have been conducted on ImageNet-1K and seven small datasets. They successfully demonstrate the effectiveness of Trip-ROMA and consistently show that ROMA can further effectively boost other SSL methods. Code is available at https://github.com/WenbinLee/Trip-ROMA.

cs.CV↗
arXiv2023

Head-Tail Cooperative Learning Network for Unbiased Scene Graph Generation

Scene Graph Generation (SGG) as a critical task in image understanding, facing the challenge of head-biased prediction caused by the long-tail distribution of predicates. However, current unbiased SGG methods can easily prioritize improving the prediction of tail predicates while ignoring the substantial sacrifice in the prediction of head predicates, leading to a shift from head bias to tail bias. To address this issue, we propose a model-agnostic Head-Tail Collaborative Learning (HTCL) network that includes head-prefer and tail-prefer feature representation branches that collaborate to achieve accurate recognition of both head and tail predicates. We also propose a self-supervised learning approach to enhance the prediction ability of the tail-prefer feature representation branch by constraining tail-prefer predicate features. Specifically, self-supervised learning converges head predicate features to their class centers while dispersing tail predicate features as much as possible through contrast learning and head center loss. We demonstrate the effectiveness of our HTCL by applying it to various SGG models on VG150, Open Images V6 and GQA200 datasets. The results show that our method achieves higher mean Recall with a minimal sacrifice in Recall and achieves a new state-of-the-art overall performance. Our code is available at https://github.com/wanglei0618/HTCL.

cs.CV↗
arXiv2023

ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction

Large language models (LLMs), such as GPT-3 and ChatGPT, have demonstrated remarkable results in various natural language processing (NLP) tasks with in-context learning, which involves inference based on a few demonstration examples. Despite their successes in NLP tasks, no investigation has been conducted to assess the ability of LLMs to perform document information extraction (DIE) using in-context learning. Applying LLMs to DIE poses two challenges: the modality and task gap. To this end, we propose a simple but effective in-context learning framework called ICL-D3IE, which enables LLMs to perform DIE with different types of demonstration examples. Specifically, we extract the most difficult and distinct segments from hard training documents as hard demonstrations for benefiting all test instances. We design demonstrations describing relationships that enable LLMs to understand positional relationships. We introduce formatting demonstrations for easy answer extraction. Additionally, the framework improves diverse demonstrations by updating them iteratively. Our experiments on three widely used benchmark datasets demonstrate that the ICL-D3IE framework enables Davinci-003/ChatGPT to achieve superior performance when compared to previous pre-trained methods fine-tuned with full training in both the in-distribution (ID) setting and in the out-of-distribution (OOD) setting. Code is available at https://github.com/MAEHCM/ICL-D3IE.

cs.CL↗
arXiv2023

Joint User Association and Transmission Scheduling in Integrated mmWave Access and Terahertz Backhaul Networks

Terahertz wireless backhaul is expected to meet the high-speed backhaul requirements of future ultra-dense networks using millimeter-wave (mmWave) base stations (BSs). In order to achieve higher network capacity with limited resources and meet the quality of service (QoS) requirements of more users in the integrated mmWave access and terahertz backhaul network, this paper formulates a problem of maximizing the number of users successfully served in both the access and backhaul links. Since the problem is a non-linear integer optimization problem, a minimum rate ratio user association and transmission scheduling algorithm is proposed to obtain a suboptimal solution. The proposed algorithm takes the minimum rate ratio as the user association criterion and schedules first the users with fewer backhaul transmission slots. In addition, the algorithm will update the number of access transmission slots allocated to users and the access scheduling results after the backhaul scheduling phase. Numerical results show that the proposed algorithm outperforms several benchmark algorithms in terms of the number of served users and system throughput, and it can cope with a large number of bursty user requests.

cs.IT↗
arXiv2023

Towards Semi-supervised Learning with Non-random Missing Labels

Semi-supervised learning (SSL) tackles the label missing problem by enabling the effective usage of unlabeled data. While existing SSL methods focus on the traditional setting, a practical and challenging scenario called label Missing Not At Random (MNAR) is usually ignored. In MNAR, the labeled and unlabeled data fall into different class distributions resulting in biased label imputation, which deteriorates the performance of SSL models. In this work, class transition tracking based Pseudo-Rectifying Guidance (PRG) is devised for MNAR. We explore the class-level guidance information obtained by the Markov random walk, which is modeled on a dynamically created graph built over the class tracking matrix. PRG unifies the historical information of class distribution and class transitions caused by the pseudo-rectifying procedure to maintain the model's unbiased enthusiasm towards assigning pseudo-labels to all classes, so as the quality of pseudo-labels on both popular classes and rare classes in MNAR could be improved. Finally, we show the superior performance of PRG across a variety of MNAR scenarios, outperforming the latest SSL approaches combining bias removal solutions by a large margin. Code and model weights are available at https://github.com/NJUyued/PRG4SSL-MNAR.

cs.LG↗
arXiv2023

Syntax-Guided Domain Adaptation for Aspect-based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) aims at extracting opinionated aspect terms in review texts and determining their sentiment polarities, which is widely studied in both academia and industry. As a fine-grained classification task, the annotation cost is extremely high. Domain adaptation is a popular solution to alleviate the data deficiency issue in new domains by transferring common knowledge across domains. Most cross-domain ABSA studies are based on structure correspondence learning (SCL), and use pivot features to construct auxiliary tasks for narrowing down the gap between domains. However, their pivot-based auxiliary tasks can only transfer knowledge of aspect terms but not sentiment, limiting the performance of existing models. In this work, we propose a novel Syntax-guided Domain Adaptation Model, named SDAM, for more effective cross-domain ABSA. SDAM exploits syntactic structure similarities for building pseudo training instances, during which aspect terms of target domain are explicitly related to sentiment polarities. Besides, we propose a syntax-based BERT mask language model for further capturing domain-invariant features. Finally, to alleviate the sentiment inconsistency issue in multi-gram aspect terms, we introduce a span-based joint aspect term and sentiment analysis module into the cross-domain End2End ABSA. Experiments on five benchmark datasets show that our model consistently outperforms the state-of-the-art baselines with respect to Micro-F1 metric for the cross-domain End2End ABSA task.

cs.AI↗
arXiv2023

MoCoSA: Momentum Contrast for Knowledge Graph Completion with Structure-Augmented Pre-trained Language Models

Knowledge Graph Completion (KGC) aims to conduct reasoning on the facts within knowledge graphs and automatically infer missing links. Existing methods can mainly be categorized into structure-based or description-based. On the one hand, structure-based methods effectively represent relational facts in knowledge graphs using entity embeddings. However, they struggle with semantically rich real-world entities due to limited structural information and fail to generalize to unseen entities. On the other hand, description-based methods leverage pre-trained language models (PLMs) to understand textual information. They exhibit strong robustness towards unseen entities. However, they have difficulty with larger negative sampling and often lag behind structure-based methods. To address these issues, in this paper, we propose Momentum Contrast for knowledge graph completion with Structure-Augmented pre-trained language models (MoCoSA), which allows the PLM to perceive the structural information by the adaptable structure encoder. To improve learning efficiency, we proposed momentum hard negative and intra-relation negative sampling. Experimental results demonstrate that our approach achieves state-of-the-art performance in terms of mean reciprocal rank (MRR), with improvements of 2.5% on WN18RR and 21% on OpenBG500.

cs.CL↗
arXiv2023

New extended method for $ψ^{\prime}$ scaling function of inclusive electron scattering

Scaling analyses have been successfully applied to study the inclusive electron scattering $(\textit{e}, \textit{e}^{\prime})$ over the past few decades. In this paper, we utilize the $ψ^{\prime}$ scaling function in momentum space to analyze the $(\textit{e}, \textit{e}^{\prime})$ cross sections, where the nucleon momentum distributions are derived from self-consistent mean-field calculations. By further introducing the energy and momentum conservation in the scaling analysis, an improved $ψ^{\prime}$ scaling function is proposed to investigate the high-momentum part of the momentum distributions. Using the proposed scaling function, we systematically explore the effects of the nucleon-nucleon short-range correlation ($\textit{NN}$-SRC) on the $(\textit{e}, \textit{e}^{\prime})$ cross sections. From the experimental $(\textit{e}, \textit{e}^{\prime})$ data, the \textit{NN}-SRC strength is further extracted within the framework of the improved $ψ^{\prime}$ scaling function. The studies in this paper offer a new method to investigate the nucleon momentum distributions and the $\textit{NN}$-SRC effects in nuclei.

nucl-th↗
arXiv2023

Does AI for science need another ImageNet Or totally different benchmarks? A case study of machine learning force fields

AI for science (AI4S) is an emerging research field that aims to enhance the accuracy and speed of scientific computing tasks using machine learning methods. Traditional AI benchmarking methods struggle to adapt to the unique challenges posed by AI4S because they assume data in training, testing, and future real-world queries are independent and identically distributed, while AI4S workloads anticipate out-of-distribution problem instances. This paper investigates the need for a novel approach to effectively benchmark AI for science, using the machine learning force field (MLFF) as a case study. MLFF is a method to accelerate molecular dynamics (MD) simulation with low computational cost and high accuracy. We identify various missed opportunities in scientifically meaningful benchmarking and propose solutions to evaluate MLFF models, specifically in the aspects of sample efficiency, time domain sensitivity, and cross-dataset generalization capabilities. By setting up the problem instantiation similar to the actual scientific applications, more meaningful performance metrics from the benchmark can be achieved. This suite of metrics has demonstrated a better ability to assess a model's performance in real-world scientific applications, in contrast to traditional AI benchmarking methodologies. This work is a component of the SAIBench project, an AI4S benchmarking suite. The project homepage is https://www.computercouncil.org/SAIBench.

cs.LG↗
arXiv2023

Automatic Radiology Report Generation by Learning with Increasingly Hard Negatives

Automatic radiology report generation is challenging as medical images or reports are usually similar to each other due to the common content of anatomy. This makes a model hard to capture the uniqueness of individual images and is prone to producing undesired generic or mismatched reports. This situation calls for learning more discriminative features that could capture even fine-grained mismatches between images and reports. To achieve this, this paper proposes a novel framework to learn discriminative image and report features by distinguishing them from their closest peers, i.e., hard negatives. Especially, to attain more discriminative features, we gradually raise the difficulty of such a learning task by creating increasingly hard negative reports for each image in the feature space during training, respectively. By treating the increasingly hard negatives as auxiliary variables, we formulate this process as a min-max alternating optimisation problem. At each iteration, conditioned on a given set of hard negative reports, image and report features are learned as usual by minimising the loss functions related to report generation. After that, a new set of harder negative reports will be created by maximising a loss reflecting image-report alignment. By solving this optimisation, we attain a model that can generate more specific and accurate reports. It is noteworthy that our framework enhances discriminative feature learning without introducing extra network weights. Also, in contrast to the existing way of generating hard negatives, our framework extends beyond the granularity of the dataset by generating harder samples out of the training set. Experimental study on benchmark datasets verifies the efficacy of our framework and shows that it can serve as a plug-in to readily improve existing medical report generation models.

cs.CV↗
arXiv2023

Differentiable programming tensor networks for Kitaev magnets

We present a general computational framework to investigate ground state properties of quantum spin models on infinite two-dimensional lattices using automatic differentiation-based gradient optimization of infinite projected entangled-pair states. The approach exploits the variational uniform matrix product states to contract infinite tensor networks with unit-cell structure and incorporates automatic differentiation to optimize the local tensors. We applied this framework to the Kitaev-type model, which involves complex interactions and competing ground states. To evaluate the accuracy of this method, we compared the results with exact solutions for the Kitaev model and found that it has a better agreement for various observables compared to previous tensor network calculations based on imaginary-time projection. Additionally, by finding out the ground state with lower variational energy compared to previous studies, we provided convincing evidence for the existence of nematic paramagnetic phases and 18-site configuration in the phase diagram of the $K$-$Γ$ model. Furthermore, in the case of the realistic $K$-$J$-$Γ$-$Γ'$ model for the Kitaev material $α$-RuCl$_3$, we discovered a non-colinear zigzag ground state. Lastly, we also find that the strength of the critical out-of-plane magnetic field that suppresses such a zigzag state has a lower transition field value than the previous finite-cylinder calculations. The framework is versatile and will be useful for a quick scan of phase diagrams for a broad class of quantum spin models.

physics.comp-ph↗