Search arXiv⌕ Search

arXiv subjects

Shuo Yang

Publications and source records attributed to Shuo Yang.

At least 307 records · Page 17Linked to original sources
arXiv2023

A Lightweight Approach for Network Intrusion Detection based on Self-Knowledge Distillation

Network Intrusion Detection (NID) works as a kernel technology for the security network environment, obtaining extensive research and application. Despite enormous efforts by researchers, NID still faces challenges in deploying on resource-constrained devices. To improve detection accuracy while reducing computational costs and model storage simultaneously, we propose a lightweight intrusion detection approach based on self-knowledge distillation, namely LNet-SKD, which achieves the trade-off between accuracy and efficiency. Specifically, we carefully design the DeepMax block to extract compact representation efficiently and construct the LNet by stacking DeepMax blocks. Furthermore, considering compensating for performance degradation caused by the lightweight network, we adopt batch-wise self-knowledge distillation to provide the regularization of training consistency. Experiments on benchmark datasets demonstrate the effectiveness of our proposed LNet-SKD, which outperforms existing state-of-the-art techniques with fewer parameters and lower computation loads.

cs.CR↗
arXiv2023

Real-space construction of crystalline topological superconductors and insulators in 2D interacting fermionic systems

The construction and classification of crystalline symmetry protected topological (SPT) phases in interacting bosonic and fermionic systems have been intensively studied in the past few years. Crystalline SPT phases are not only of conceptual importance, but also provide great opportunities towards experimental realization since space group symmetries naturally exist for any realistic material. In this paper, we systematically classify the crystalline topological superconductors (TSC) and topological insulators (TI) in 2D interacting fermionic systems by using an explicit real-space construction. In particular, we discover an intriguing fermionic crystalline topological superconductor that can only be realized in interacting fermionic systems (i.e., not in free-fermion or bosonic SPT systems). Moreover, we also verify the recently conjectured crystalline equivalence principle for generic 2D interacting fermionic systems.

cond-mat.str-el↗
arXiv2023

BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency

As one of the most fundamental techniques in multimodal learning, cross-modal matching aims to project various sensory modalities into a shared feature space. To achieve this, massive and correctly aligned data pairs are required for model training. However, unlike unimodal datasets, multimodal datasets are extremely harder to collect and annotate precisely. As an alternative, the co-occurred data pairs (e.g., image-text pairs) collected from the Internet have been widely exploited in the area. Unfortunately, the cheaply collected dataset unavoidably contains many mismatched data pairs, which have been proven to be harmful to the model's performance. To address this, we propose a general framework called BiCro (Bidirectional Cross-modal similarity consistency), which can be easily integrated into existing cross-modal matching models and improve their robustness against noisy data. Specifically, BiCro aims to estimate soft labels for noisy data pairs to reflect their true correspondence degree. The basic idea of BiCro is motivated by that -- taking image-text matching as an example -- similar images should have similar textual descriptions and vice versa. Then the consistency of these two similarities can be recast as the estimated soft labels to train the matching model. The experiments on three popular cross-modal matching datasets demonstrate that our method significantly improves the noise-robustness of various matching models, and surpass the state-of-the-art by a clear margin.

cs.CV↗
arXiv2023

Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach

Matrix manifolds, such as manifolds of Symmetric Positive Definite (SPD) matrices and Grassmann manifolds, appear in many applications. Recently, by applying the theory of gyrogroups and gyrovector spaces that is a powerful framework for studying hyperbolic geometry, some works have attempted to build principled generalizations of Euclidean neural networks on matrix manifolds. However, due to the lack of many concepts in gyrovector spaces for the considered manifolds, e.g., the inner product and gyroangles, techniques and mathematical tools provided by these works are still limited compared to those developed for studying hyperbolic geometry. In this paper, we generalize some notions in gyrovector spaces for SPD and Grassmann manifolds, and propose new models and layers for building neural networks on these manifolds. We show the effectiveness of our approach in two applications, i.e., human action recognition and knowledge graph completion.

stat.ML↗
arXiv2023

A Benchmark Comparison of Imitation Learning-based Control Policies for Autonomous Racing

Autonomous racing with scaled race cars has gained increasing attention as an effective approach for developing perception, planning and control algorithms for safe autonomous driving at the limits of the vehicle's handling. To train agile control policies for autonomous racing, learning-based approaches largely utilize reinforcement learning, albeit with mixed results. In this study, we benchmark a variety of imitation learning policies for racing vehicles that are applied directly or for bootstrapping reinforcement learning both in simulation and on scaled real-world environments. We show that interactive imitation learning techniques outperform traditional imitation learning methods and can greatly improve the performance of reinforcement learning policies by bootstrapping thanks to its better sample efficiency. Our benchmarks provide a foundation for future research on autonomous racing using Imitation Learning and Reinforcement Learning.

cs.RO↗
arXiv2023

Towards Understanding Crypto Money Laundering in Web3 Through the Lenses of Ethereum Heists

With the overall momentum of the blockchain industry, crypto-based crimes are becoming more and more prevalent. After committing a crime, the main goal of cybercriminals is to obfuscate the source of the illicit funds in order to convert them into cash and get away with it. Many studies have analyzed money laundering in the field of the traditional financial sector and blockchain-based Bitcoin. But so far, little is known about the characteristics of crypto money laundering in the blockchain-based Web3 ecosystem. To fill this gap, and considering that Ethereum is the largest platform on Web3, in this paper, we systematically study the behavioral characteristics and economic impact of money laundering accounts through the lenses of Ethereum heists. Based on a very small number of tagged accounts of exchange hackers, DeFi exploiters, and scammers, we mine untagged money laundering groups through heuristic transaction tracking methods, to carve out a full picture of security incidents. By analyzing account characteristics and transaction networks, we obtain many interesting findings about crypto money laundering in Web3, observing the escalating money laundering methods such as creating counterfeit tokens and masquerading as speculators. Finally, based on these findings we provide inspiration for anti-money laundering to promote the healthy development of the Web3 ecosystem.

cs.CR↗
arXiv2023

You Don't Know When I Will Arrive: Unpredictable Controller Synthesis for Temporal Logic Tasks

In this paper, we investigate the problem of synthesizing controllers for temporal logic specifications under security constraint. We assume that there exists a passive intruder (eavesdropper) that can partially observe the behavior of the system. For the purpose of security, we require that the system's behaviors are unpredictable in the sense that the intruder cannot determine for sure that the system will exactly accomplish the task in $K$ steps ahead. This problem is particularly challenging since future information is involved in the synthesis process. We propose a novel information structure that predicts the effect of control in the future. A sound and complete algorithm is developed to synthesize a controller which ensures both task completion and security guarantee. The proposed approach is illustrated by a case study of robot task planning.

eess.SY↗
arXiv2023

Construction of Non-Hermitian Parent Hamiltonian from Matrix Product States

There are various research strategies used for non-Hermitian systems, which typically involve introducing non-Hermitian terms to pre-existing Hermitian Hamiltonians. It can be challenging to directly design non-Hermitian many-body models that exhibit unique features not found in Hermitian systems. In this Letter, we propose a new method to construct non-Hermitian many-body systems by generalizing the parent Hamiltonian method into non-Hermitian regimes. This allows us to build a local Hamiltonian using given matrix product states as its left and right ground states. We demonstrate this method by constructing a non-Hermitian spin-$1$ model from the asymmetric Affleck-Kennedy-Lieb-Tasaki (AKLT) state, which preserves both chiral order and symmetry-protected topological order. Our approach opens up a new paradigm for systematically constructing and studying non-Hermitian many-body systems, providing guiding principles to explore new properties and phenomena in non-Hermitian physics.

quant-ph↗
arXiv2023

Rethink Long-tailed Recognition with Vision Transformers

In the real world, data tends to follow long-tailed distributions w.r.t. class or attribution, motivating the challenging Long-Tailed Recognition (LTR) problem. In this paper, we revisit recent LTR methods with promising Vision Transformers (ViT). We figure out that 1) ViT is hard to train with long-tailed data. 2) ViT learns generalized features in an unsupervised manner, like mask generative training, either on long-tailed or balanced datasets. Hence, we propose to adopt unsupervised learning to utilize long-tailed data. Furthermore, we propose the Predictive Distribution Calibration (PDC) as a novel metric for LTR, where the model tends to simply classify inputs into common classes. Our PDC can measure the model calibration of predictive preferences quantitatively. On this basis, we find many LTR approaches alleviate it slightly, despite the accuracy improvement. Extensive experiments on benchmark datasets validate that PDC reflects the model's predictive preference precisely, which is consistent with the visualization.

cs.CV↗
arXiv2023

Learning Imbalanced Data with Vision Transformers

The real-world data tends to be heavily imbalanced and severely skew the data-driven deep neural networks, which makes Long-Tailed Recognition (LTR) a massive challenging task. Existing LTR methods seldom train Vision Transformers (ViTs) with Long-Tailed (LT) data, while the off-the-shelf pretrain weight of ViTs always leads to unfair comparisons. In this paper, we systematically investigate the ViTs' performance in LTR and propose LiVT to train ViTs from scratch only with LT data. With the observation that ViTs suffer more severe LTR problems, we conduct Masked Generative Pretraining (MGP) to learn generalized features. With ample and solid evidence, we show that MGP is more robust than supervised manners. In addition, Binary Cross Entropy (BCE) loss, which shows conspicuous performance with ViTs, encounters predicaments in LTR. We further propose the balanced BCE to ameliorate it with strong theoretical groundings. Specially, we derive the unbiased extension of Sigmoid and compensate extra logit margins to deploy it. Our Bal-BCE contributes to the quick convergence of ViTs in just a few epochs. Extensive experiments demonstrate that with MGP and Bal-BCE, LiVT successfully trains ViTs well without any additional data and outperforms comparable state-of-the-art methods significantly, e.g., our ViT-B achieves 81.0% Top-1 accuracy in iNaturalist 2018 without bells and whistles. Code is available at https://github.com/XuZhengzhuo/LiVT.

cs.CV↗
arXiv2023

Dataset Pruning: Reducing Training Data by Examining Generalization Influence

The great success of deep learning heavily relies on increasingly larger training data, which comes at a price of huge computational and infrastructural costs. This poses crucial questions that, do all training data contribute to model's performance? How much does each individual training sample or a sub-training-set affect the model's generalization, and how to construct the smallest subset from the entire training data as a proxy training set without significantly sacrificing the model's performance? To answer these, we propose dataset pruning, an optimization-based sample selection method that can (1) examine the influence of removing a particular set of training samples on model's generalization ability with theoretical guarantee, and (2) construct the smallest subset of training data that yields strictly constrained generalization gap. The empirically observed generalization gap of dataset pruning is substantially consistent with our theoretical expectations. Furthermore, the proposed method prunes 40% training examples on the CIFAR-10 dataset, halves the convergence time with only 1.3% test accuracy decrease, which is superior to previous score-based sample selection methods.

cs.LG↗
arXiv2023

Chebyshev pseudosite matrix product state approach for the spectral functions of electron-phonon coupling systems

The electron-phonon ($e$-ph) coupling system often has a large number of phonon degrees of freedom, whose spectral functions are numerically difficult to compute using matrix product state (MPS) formalisms. To solve this problem, we propose a new and practical method that combines the Chebyshev MPS and the pseudosite density matrix renormalization group (DMRG) algorithm. The Chebyshev vector is represented by a pseudosite MPS with global $U(1)$ fermion symmetry, which maps $2^{N_p}$ bosonic degrees of freedom onto $N_p$ pseudosites, each with two states. This approach can handle arbitrary $e$-ph coupling Hamiltonians where pseudosite DMRG performs efficiently. We use this method to study the spectral functions of the doped extended Hubbard-Holstein model in a regime of strong Coulomb repulsion, which has not been studied extensively before. Key features of the excitation spectra are captured at a modest computational cost. Our results show that weak extended $e$-ph couplings can increase the spectral weight of the holon-folding branch at low phonon frequencies, in agreement with angle-resolved photoemission observations on one-dimensional (1D) cuprates.

cond-mat.str-el↗
arXiv2023

Differentiable Safe Controller Design through Control Barrier Functions

Learning-based controllers, such as neural network (NN) controllers, can show high empirical performance but lack formal safety guarantees. To address this issue, control barrier functions (CBFs) have been applied as a safety filter to monitor and modify the outputs of learning-based controllers in order to guarantee the safety of the closed-loop system. However, such modification can be myopic with unpredictable long-term effects. In this work, we propose a safe-by-construction NN controller which employs differentiable CBF-based safety layers, and investigate the performance of safe-by-construction NN controllers in learning-based control. Specifically, two formulations of controllers are compared: one is projection-based and the other relies on our proposed set-theoretic parameterization. Both methods demonstrate improved closed-loop performance over using CBF as a separate safety filter in numerical experiments.

eess.SY↗
arXiv2023

Fast Contact-Implicit Model-Predictive Control

We present a general approach for controlling robotic systems that make and break contact with their environments. Contact-implicit model predictive control (CI-MPC) generalizes linear MPC to contact-rich settings by utilizing a bi-level planning formulation with lower-level contact dynamics formulated as time-varying linear complementarity problems (LCPs) computed using strategic Taylor approximations about a reference trajectory. These dynamics enable the upper-level planning problem to reason about contact timing and forces, and generate entirely new contact-mode sequences online. To achieve reliable and fast numerical convergence, we devise a structure-exploiting interior-point solver for these LCP contact dynamics and a custom trajectory optimizer for the tracking problem. We demonstrate real-time solution rates for CI-MPC and the ability to generate and track non-periodic behaviours in hardware experiments on a quadrupedal robot. We also show that the controller is robust to model mismatch and can respond to disturbances by discovering and exploiting new contact modes across a variety of robotic systems in simulation, including a pushbot, planar hopper, planar quadruped, and planar biped.

cs.RO↗
arXiv2023

Noise effects on purity and quantum entanglement in terms of physical implementability

Quantum decoherence due to imperfect manipulation of quantum devices is a key issue in the noisy intermediate-scale quantum (NISQ) era. Standard analyses in quantum information and quantum computation use error rates to parameterize quantum noise channels. However, there is no explicit relation between the decoherence effect induced by a noise channel and its error rate. In this work, we propose to characterize the decoherence effect of a noise channel by the physical implementability of its inverse, which is a universal parameter quantifying the difficulty to simulate the noise inverse with accessible quantum channels. We establish two concise inequalities connecting the decrease of the state purity and logarithmic negativity after a noise channel to the physical implementability of the noise inverse, which is required to be decomposed as mutually orthogonal unitaries or product channels respectively. Our results are numerically demonstrated on several commonly adopted two-qubit noise models. We believe that these relations contribute to the theoretical research on the entanglement properties of noise channels and provide guiding principles for quantum circuit design.

quant-ph↗
arXiv2022

PartImageNet: A Large, High-Quality Dataset of Parts

It is natural to represent objects in terms of their parts. This has the potential to improve the performance of algorithms for object recognition and segmentation but can also help for downstream tasks like activity recognition. Research on part-based models, however, is hindered by the lack of datasets with per-pixel part annotations. This is partly due to the difficulty and high cost of annotating object parts so it has rarely been done except for humans (where there exists a big literature on part-based models). To help address this problem, we propose PartImageNet, a large, high-quality dataset with part segmentation annotations. It consists of $158$ classes from ImageNet with approximately $24,000$ images. PartImageNet is unique because it offers part-level annotations on a general set of classes including non-rigid, articulated objects, while having an order of magnitude larger size compared to existing part datasets (excluding datasets of humans). It can be utilized for many vision tasks including Object Segmentation, Semantic Part Segmentation, Few-shot Learning and Part Discovery. We conduct comprehensive experiments which study these tasks and set up a set of baselines. The dataset and scripts are released at https://github.com/TACJu/PartImageNet.

cs.CV↗
arXiv2022

CPMLHO:Hyperparameter Tuning via Cutting Plane and Mixed-Level Optimization

The hyperparameter optimization of neural network can be expressed as a bilevel optimization problem. The bilevel optimization is used to automatically update the hyperparameter, and the gradient of the hyperparameter is the approximate gradient based on the best response function. Finding the best response function is very time consuming. In this paper we propose CPMLHO, a new hyperparameter optimization method using cutting plane method and mixed-level objective function.The cutting plane is added to the inner layer to constrain the space of the response function. To obtain more accurate hypergradient,the mixed-level can flexibly adjust the loss function by using the loss of the training set and the verification set. Compared to existing methods, the experimental results show that our method can automatically update the hyperparameters in the training process, and can find more superior hyperparameters with higher accuracy and faster convergence.

cs.LG↗
arXiv2022

Towards Explainability in Modular Autonomous Vehicle Software

Safety-critical Autonomous Systems require trustworthy and transparent decision-making process to be deployable in the real world. The advancement of Machine Learning introduces high performance but largely through black-box algorithms. We focus the discussion of explainability specifically with Autonomous Vehicles (AVs). As a safety-critical system, AVs provide the unique opportunity to utilize cutting-edge Machine Learning techniques while requiring transparency in decision making. Interpretability in every action the AV takes becomes crucial in post-hoc analysis where blame assignment might be necessary. In this paper, we provide positioning on how researchers could consider incorporating explainability and interpretability into design and optimization of separate Autonomous Vehicle modules including Perception, Planning, and Control.

cs.RO↗