arXiv subjects

Xin Wang

Publications and source records attributed to Xin Wang.

At least 1,081 records · Page 60Linked to original sources

arXiv2023

Outlier Robust Adversarial Training

Supervised learning models are challenged by the intrinsic complexities of training data such as outliers and minority subpopulations and intentional attacks at inference time with adversarial samples. While traditional robust learning methods and the recent adversarial training approaches are designed to handle each of the two challenges, to date, no work has been done to develop models that are robust with regard to the low-quality training data and the potential adversarial attack at inference time simultaneously. It is for this reason that we introduce Outlier Robust Adversarial Training (ORAT) in this work. ORAT is based on a bi-level optimization formulation of adversarial training with a robust rank-based loss function. Theoretically, we show that the learning objective of ORAT satisfies the $\mathcal{H}$-consistency in binary classification, which establishes it as a proper surrogate to adversarial 0/1 loss. Furthermore, we analyze its generalization ability and provide uniform convergence rates in high probability. ORAT can be optimized with a simple algorithm. Experimental evaluations on three benchmark datasets demonstrate the effectiveness and robustness of ORAT in handling outliers and adversarial attacks. Our code is available at https://github.com/discovershu/ORAT.

cs.LG↗

arXiv2023

Control-Oriented Modeling and Layer-to-Layer Spatial Control of Powder Bed Fusion Processes

Powder Bed Fusion (PBF) is an important Additive Manufacturing (AM) process that is seeing widespread utilization. However, due to inherent process variability, it is still very costly and time consuming to certify the process and the part. This has led researchers to conduct numerous studies in process modeling, in-situ monitoring and feedback control to better understand the PBF process and decrease variations, thereby making the process more repeatable. In this study, we develop a layer-to-layer, spatial, control-oriented thermal PBF model. This model enables a framework for capturing spatially-driven thermal effects and constructing layer-to-layer spatial controllers that do not suffer from inherent temporal delays. Further, this framework is amenable to voxel-level monitoring and characterization efforts. System output controllability is analyzed and output controllability conditions are determined. A spatial Iterative Learning Controller (ILC), constructed using the spatial modeling framework, is implemented in two experiments, one where the path and part geometry are layer-invariant and another where the path and part geometry change each layer. The results illustrate the ability of the controller to thermally regulate the entire part, even at corners that tend to overheat and even as the path and part geometry change each layer.

eess.SY↗

arXiv2023

Trion states and quantum criticality of attractive SU(3) Dirac fermions

We perform the projector quantum Monte Carlo (QMC) simulation to study the trion formation and quantum phase transition in the half-filled attractive SU(3) Hubbard model on a honeycomb lattice. With increasing attractive Hubbard interaction, our simulations demonstrate a continuous quantum phase transition from the semimetal to charge density wave (CDW) at the critical coupling $U_c/t=-1.52(2)$. The critical exponents $ν=0.82(3)$ and $η=0.58(4)$ determined by the QMC simulation remarkably disagree with those of the $N=3$ chiral Ising universality class suggested by the effective Gross-Neveu-Yukawa (GNY) theory, but coincide with the $N=1$ chiral Ising universality class. In the CDW phase, we show that on-site and off-site trions coexist and the off-site trion forms a local bond state. Our work not only illustrates the formation of off-site trions in two-dimensional Hubbard model, but also raises doubts about the extent of applicability of GNY model on the attractive SU(3) Dirac fermions.

cond-mat.quant-gas↗

arXiv2023

Revisiting Adversarial Attacks on Graph Neural Networks for Graph Classification

Graph neural networks (GNNs) have achieved tremendous success in the task of graph classification and its diverse downstream real-world applications. Despite the huge success in learning graph representations, current GNN models have demonstrated their vulnerability to potentially existent adversarial examples on graph-structured data. Existing approaches are either limited to structure attacks or restricted to local information, urging for the design of a more general attack framework on graph classification, which faces significant challenges due to the complexity of generating local-node-level adversarial examples using the global-graph-level information. To address this "global-to-local" attack challenge, we present a novel and general framework to generate adversarial examples via manipulating graph structure and node features. Specifically, we make use of Graph Class Activation Mapping and its variant to produce node-level importance corresponding to the graph classification task. Then through a heuristic design of algorithms, we can perform both feature and structure attacks under unnoticeable perturbation budgets with the help of both node-level and subgraph-level importance. Experiments towards attacking four state-of-the-art graph classification models on six real-world benchmarks verify the flexibility and effectiveness of our framework.

cs.SI↗

arXiv2023

DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction

Click-Through Rate (CTR) prediction is a pivotal task in product and content recommendation, where learning effective feature embeddings is of great significance. However, traditional methods typically learn fixed feature representations without dynamically refining feature representations according to the context information, leading to suboptimal performance. Some recent approaches attempt to address this issue by learning bit-wise weights or augmented embeddings for feature representations, but suffer from uninformative or redundant features in the context. To tackle this problem, inspired by the Global Workspace Theory in conscious processing, which posits that only a specific subset of the product features are pertinent while the rest can be noisy and even detrimental to human-click behaviors, we propose a CTR model that enables Dynamic Embedding Learning with Truncated Conscious Attention for CTR prediction, termed DELTA. DELTA contains two key components: (I) conscious truncation module (CTM), which utilizes curriculum learning to apply adaptive truncation on attention weights to select the most critical feature in the context; (II) explicit embedding optimization (EEO), which applies an auxiliary task during training that directly and independently propagates the gradient from the loss layer to the embedding layer, thereby optimizing the embedding explicitly via linear feature crossing. Extensive experiments on five challenging CTR datasets demonstrate that DELTA achieves new state-of-art performance among current CTR methods.

cs.IR↗

arXiv2023

DRAG: Divergence-based Adaptive Aggregation in Federated learning on Non-IID Data

Local stochastic gradient descent (SGD) is a fundamental approach in achieving communication efficiency in Federated Learning (FL) by allowing individual workers to perform local updates. However, the presence of heterogeneous data distributions across working nodes causes each worker to update its local model towards a local optimum, leading to the phenomenon known as ``client-drift" and resulting in slowed convergence. To address this issue, previous works have explored methods that either introduce communication overhead or suffer from unsteady performance. In this work, we introduce a novel metric called ``degree of divergence," quantifying the angle between the local gradient and the global reference direction. Leveraging this metric, we propose the divergence-based adaptive aggregation (DRAG) algorithm, which dynamically ``drags" the received local updates toward the reference direction in each round without requiring extra communication overhead. Furthermore, we establish a rigorous convergence analysis for DRAG, proving its ability to achieve a sublinear convergence rate. Compelling experimental results are presented to illustrate DRAG's superior performance compared to state-of-the-art algorithms in effectively managing the client-drift phenomenon. Additionally, DRAG exhibits remarkable resilience against certain Byzantine attacks. By securely sharing a small sample of the client's data with the FL server, DRAG effectively counters these attacks, as demonstrated through comprehensive experiments.

cs.LG↗

arXiv2023

BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks

To handle graphs in which features or connectivities are evolving over time, a series of temporal graph neural networks (TGNNs) have been proposed. Despite the success of these TGNNs, the previous TGNN evaluations reveal several limitations regarding four critical issues: 1) inconsistent datasets, 2) inconsistent evaluation pipelines, 3) lacking workload diversity, and 4) lacking efficient comparison. Overall, there lacks an empirical study that puts TGNN models onto the same ground and compares them comprehensively. To this end, we propose BenchTemp, a general benchmark for evaluating TGNN models on various workloads. BenchTemp provides a set of benchmark datasets so that different TGNN models can be fairly compared. Further, BenchTemp engineers a standard pipeline that unifies the TGNN evaluation. With BenchTemp, we extensively compare the representative TGNN models on different tasks (e.g., link prediction and node classification) and settings (transductive and inductive), w.r.t. both effectiveness and efficiency metrics. We have made BenchTemp publicly available at https://github.com/qianghuangwhu/benchtemp.

cs.LG↗

arXiv2023

Near-term Efficient Quantum Algorithms for Entanglement Analysis

Entanglement plays a crucial role in quantum physics and is the key resource in quantum information processing. However, entanglement detection and quantification are believed to be hard due to the operational impracticality of existing methods. This work proposes three near-term efficient algorithms exploiting the hybrid quantum-classical technique to address this difficulty. The first algorithm finds the Schmidt decomposition--a powerful tool to analyze the properties and structure of entanglement--for bipartite pure states. While the logarithm negativity can be calculated from the Schmidt decomposition, we propose the second algorithm to estimate the logarithm negativity for bipartite pure states, where the width of the parameterized quantum circuits is further reduced. Finally, we generalize our framework for mixed states, leading to our third algorithm which detects entanglement on specific families of states, and determines disdillability in general. All three algorithms share a similar framework where the optimizations are accomplished by maximizing a cost function utilizing local parameterized quantum circuits, with better hardware efficiency and practicality compared to existing methods. The experimental implementation on Quantum Leaf using the IoP CAS superconducting quantum processor exhibits the validity and practicality of our methods for analyzing and quantifying entanglement on near-term quantum devices.

quant-ph↗

arXiv2023

Resolution-independent generative models based on operator learning for physics-constrained Bayesian inverse problems

The Bayesian inference approach is widely used to tackle inverse problems due to its versatile and natural ability to handle ill-posedness. However, it often faces challenges when dealing with situations involving continuous fields or large-resolution discrete representations (high-dimensional). Moreover, the prior distribution of unknown parameters is commonly difficult to be determined. In this study, an Operator Learning-based Generative Adversarial Network (OL-GAN) is proposed and integrated into the Bayesian inference framework to handle these issues. Unlike most Bayesian approaches, the distinctive characteristic of the proposed method is to learn the joint distribution of parameters and responses. By leveraging the trained generative model, the posteriors of the unknown parameters can theoretically be approximated by any sampling algorithm (e.g., Markov Chain Monte Carlo, MCMC) in a low-dimensional latent space shared by the components of the joint distribution. The latent space is typically a simple and easy-to-sample distribution (e.g., Gaussian, uniform), which significantly reduces the computational cost associated with the Bayesian inference while avoiding prior selection concerns. Furthermore, incorporating operator learning enables resolution-independent in the generator. Predictions can be obtained at desired coordinates, and inversions can be performed even if the observation data are misaligned with the training data. Finally, the effectiveness of the proposed method is validated through several numerical experiments.

math.NA↗

arXiv2023

Semiquantum key distribution using initial states in only one basis without the classical user measuring

From the perspective of resource theory, it is interesting to achieve the same quantum task using as few quantum resources as possible. Semiquantum key distribution (SQKD), which allows a quantum user to share a confidential key with a classical user who prepares and operates qubits in only one basis, is an important example for studying this issue. To further limit the quantum resources used by users, in this paper, we constructed the first SQKD protocol which restricts the quantum user to prepare quantum states in only one basis and removes the classical user's measurement capability. Furthermore, we prove that the constructed protocol is unconditionally secure by deriving a key rate expression of the error rate in the asymptotic scenario. The work of this paper provides inspiration for achieving quantum superiority with minimal quantum resources.

quant-ph↗

arXiv2023

Unsupervised Multiplex Graph Learning with Complementary and Consistent Information

Unsupervised multiplex graph learning (UMGL) has been shown to achieve significant effectiveness for different downstream tasks by exploring both complementary information and consistent information among multiple graphs. However, previous methods usually overlook the issues in practical applications, i.e., the out-of-sample issue and the noise issue. To address the above issues, in this paper, we propose an effective and efficient UMGL method to explore both complementary and consistent information. To do this, our method employs multiple MLP encoders rather than graph convolutional network (GCN) to conduct representation learning with two constraints, i.e., preserving the local graph structure among nodes to handle the out-of-sample issue, and maximizing the correlation of multiple node representations to handle the noise issue. Comprehensive experiments demonstrate that our proposed method achieves superior effectiveness and efficiency over the comparison methods and effectively tackles those two issues. Code is available at https://github.com/LarryUESTC/CoCoMG.

cs.LG↗

arXiv2023

Hawking effect can generate physically inaccessible genuine tripartite nonlocality

We explore the acceleration effect on the genuine tripartite nonlocality (GTN) for one or two accelerated detector(s) coupled to the vacuum field with initial mixed tripartite states. We show that the Hawking radiation degrades the physically accessible GTN, which suffers from "sudden death" at certain critical Hawking temperature. An novel phenomenon has been observed first time that the Hawking effect can generate the physically inaccessible GTN for fermion fields in curved spacetime, the "sudden birth" of the physically inaccessible GTN. This result shows that the GTN can pass through the event horizon of black hole for certain mixed initial states. We also derived analytically the tradeoff relations of genuine tripartite entanglement (GTE) and quantum coherence under the influence of Hawking effect.

gr-qc↗

arXiv2023

Environmental dependence of the mass-metallicity relation in cosmological hydrodynamical simulations

We investigate the environmental dependence of the gas-phase metallicity for galaxies at $z=0$ to $z\gtrsim 2$ and the underlying physical mechanisms driving this dependence using state-of-the-art cosmological hydrodynamical simulations. We find that, at fixed stellar mass, central galaxies in massive halos have lower gas-phase metallicity than those in low-mass halos. On the contrary, satellite galaxies residing in more massive halos are more metal-rich. The combined effect is that massive galaxies are more metal-poor in massive halos, and low-mass galaxies are more metal-rich in massive halos. By inspecting the environmental dependence of other galaxy properties, we identify that the accretion of low-metallicity gas is responsible for the environmental dependence of central galaxies at high $z$, whereas the AGN feedback processes play a crucial role at low $z$. For satellite galaxies, we find that both the suppression of gas accretion and the stripping of existing gas are responsible for their environmental dependence, with negligible effect from the AGN feedback. Finally, we show that the difference of gas-phase metallicity as a function of stellar mass between protocluster and field galaxies agrees with recent observational results, for example from the MAMMOTH-Grism survey.

astro-ph.GA↗

arXiv2023

DETReg: Unsupervised Pretraining with Region Priors for Object Detection

Recent self-supervised pretraining methods for object detection largely focus on pretraining the backbone of the object detector, neglecting key parts of detection architecture. Instead, we introduce DETReg, a new self-supervised method that pretrains the entire object detection network, including the object localization and embedding components. During pretraining, DETReg predicts object localizations to match the localizations from an unsupervised region proposal generator and simultaneously aligns the corresponding feature embeddings with embeddings from a self-supervised image encoder. We implement DETReg using the DETR family of detectors and show that it improves over competitive baselines when finetuned on COCO, PASCAL VOC, and Airbus Ship benchmarks. In low-data regimes DETReg achieves improved performance, e.g., when training with only 1% of the labels and in the few-shot learning settings.

cs.CV↗

arXiv2023

SphereNet: Learning a Noise-Robust and General Descriptor for Point Cloud Registration

Point cloud registration is to estimate a transformation to align point clouds collected in different perspectives. In learning-based point cloud registration, a robust descriptor is vital for high-accuracy registration. However, most methods are susceptible to noise and have poor generalization ability on unseen datasets. Motivated by this, we introduce SphereNet to learn a noise-robust and unseen-general descriptor for point cloud registration. In our method, first, the spheroid generator builds a geometric domain based on spherical voxelization to encode initial features. Then, the spherical interpolation of the sphere is introduced to realize robustness against noise. Finally, a new spherical convolutional neural network with spherical integrity padding completes the extraction of descriptors, which reduces the loss of features and fully captures the geometric features. To evaluate our methods, a new benchmark 3DMatch-noise with strong noise is introduced. Extensive experiments are carried out on both indoor and outdoor datasets. Under high-intensity noise, SphereNet increases the feature matching recall by more than 25 percentage points on 3DMatch-noise. In addition, it sets a new state-of-the-art performance for the 3DMatch and 3DLoMatch benchmarks with 93.5\% and 75.6\% registration recall and also has the best generalization ability on unseen datasets.

cs.CV↗

arXiv2023

Rank-based Decomposable Losses in Machine Learning: A Survey

Recent works have revealed an essential paradigm in designing loss functions that differentiate individual losses vs. aggregate losses. The individual loss measures the quality of the model on a sample, while the aggregate loss combines individual losses/scores over each training sample. Both have a common procedure that aggregates a set of individual values to a single numerical value. The ranking order reflects the most fundamental relation among individual values in designing losses. In addition, decomposability, in which a loss can be decomposed into an ensemble of individual terms, becomes a significant property of organizing losses/scores. This survey provides a systematic and comprehensive review of rank-based decomposable losses in machine learning. Specifically, we provide a new taxonomy of loss functions that follows the perspectives of aggregate loss and individual loss. We identify the aggregator to form such losses, which are examples of set functions. We organize the rank-based decomposable losses into eight categories. Following these categories, we review the literature on rank-based aggregate losses and rank-based individual losses. We describe general formulas for these losses and connect them with existing research topics. We also suggest future research directions spanning unexplored, remaining, and emerging issues in rank-based decomposable losses.

cs.LG↗

arXiv2023

TOAST: Transfer Learning via Attention Steering

Transfer learning involves adapting a pre-trained model to novel downstream tasks. However, we observe that current transfer learning methods often fail to focus on task-relevant features. In this work, we explore refocusing model attention for transfer learning. We introduce Top-Down Attention Steering (TOAST), a novel transfer learning algorithm that keeps the pre-trained backbone frozen, selects task-relevant features in the output, and feeds those features back to the model to steer the attention to the task-specific features. By refocusing the attention only, TOAST achieves state-of-the-art results on a number of transfer learning benchmarks, while having a small number of tunable parameters. Compared to fully fine-tuning, LoRA, and prompt tuning, TOAST substantially improves performance across a range of fine-grained visual classification datasets (e.g., 81.1% -> 86.2% on FGVC). TOAST also outperforms the fully fine-tuned Alpaca and Vicuna models on instruction-following language generation. Code is available at https://github.com/bfshi/TOAST.

cs.CV↗

arXiv2023

DisAsymNet: Disentanglement of Asymmetrical Abnormality on Bilateral Mammograms using Self-adversarial Learning

Asymmetry is a crucial characteristic of bilateral mammograms (Bi-MG) when abnormalities are developing. It is widely utilized by radiologists for diagnosis. The question of 'what the symmetrical Bi-MG would look like when the asymmetrical abnormalities have been removed ?' has not yet received strong attention in the development of algorithms on mammograms. Addressing this question could provide valuable insights into mammographic anatomy and aid in diagnostic interpretation. Hence, we propose a novel framework, DisAsymNet, which utilizes asymmetrical abnormality transformer guided self-adversarial learning for disentangling abnormalities and symmetric Bi-MG. At the same time, our proposed method is partially guided by randomly synthesized abnormalities. We conduct experiments on three public and one in-house dataset, and demonstrate that our method outperforms existing methods in abnormality classification, segmentation, and localization tasks. Additionally, reconstructed normal mammograms can provide insights toward better interpretable visual cues for clinical diagnosis. The code will be accessible to the public.

cs.CV↗