arXiv subjects

Fei Wang

Publications and source records attributed to Fei Wang.

At least 541 records · Page 30Linked to original sources

arXiv2022

A review of knowledge graph application scenarios in cyber security

Facing the dynamic complex cyber environments, internal and external cyber threat intelligence, and the increasing risk of cyber-attack, knowledge graphs show great application potential in the cyber security area because of their capabilities in knowledge aggregation, representation, management, and reasoning. However, while most research has focused on how to develop a complete knowledge graph, it remains unclear how to apply the knowledge graph to solve industrial real challenges in cyber-attack and defense scenarios. In this review, we provide a brief overview of the basic concepts, schema, and construction approaches for the cyber security knowledge graph. To facilitate future research on cyber security knowledge graphs, we also present a curated collection of datasets and open-source libraries on the knowledge construction and information extraction task. In the major part of this article, we conduct a comparative review of the different works that elaborate on the recent progress in the application scenarios of the cyber security knowledge graph. Furthermore, a novel comprehensive classification framework is created to describe the connected works from nine primary categories and eighteen subcategories. Finally, we have a thorough outlook on several promising research directions based on the discussion of existing research flaws.

cs.CR↗

arXiv2022

Multimodal Machine Learning in Precision Health

As machine learning and artificial intelligence are more frequently being leveraged to tackle problems in the health sector, there has been increased interest in utilizing them in clinical decision-support. This has historically been the case in single modal data such as electronic health record data. Attempts to improve prediction and resemble the multimodal nature of clinical expert decision-making this has been met in the computational field of machine learning by a fusion of disparate data. This review was conducted to summarize this field and identify topics ripe for future research. We conducted this review in accordance with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) extension for Scoping Reviews to characterize multi-modal data fusion in health. We used a combination of content analysis and literature searches to establish search strings and databases of PubMed, Google Scholar, and IEEEXplore from 2011 to 2021. A final set of 125 articles were included in the analysis. The most common health areas utilizing multi-modal methods were neurology and oncology. However, there exist a wide breadth of current applications. The most common form of information fusion was early fusion. Notably, there was an improvement in predictive performance performing heterogeneous data fusion. Lacking from the papers were clear clinical deployment strategies and pursuit of FDA-approved tools. These findings provide a map of the current literature on multimodal data fusion as applied to health diagnosis/prognosis problems. Multi-modal machine learning, while more robust in its estimations over unimodal methods, has drawbacks in its scalability and the time-consuming nature of information concatenation.

cs.LG↗

arXiv2022

TR-MOT: Multi-Object Tracking by Reference

Multi-object Tracking (MOT) generally can be split into two sub-tasks, i.e., detection and association. Many previous methods follow the tracking by detection paradigm, which first obtain detections at each frame and then associate them between adjacent frames. Though with an impressive performance by utilizing a strong detector, it will degrade their detection and association performance under scenes with many occlusions and large motion if not using temporal information. In this paper, we propose a novel Reference Search (RS) module to provide a more reliable association based on the deformable transformer structure, which is natural to learn the feature alignment for each object among frames. RS takes previous detected results as references to aggregate the corresponding features from the combined features of the adjacent frames and makes a one-to-one track state prediction for each reference in parallel. Therefore, RS can attain a reliable association coping with unexpected motions by leveraging visual temporal features while maintaining the strong detection performance by decoupling from the detector. Our RS module can also be compatible with the structure of the other tracking by detection frameworks. Furthermore, we propose a joint training strategy and an effective matching pipeline for our online MOT framework with the RS module. Our method achieves competitive results on MOT17 and MOT20 datasets.

cs.CV↗

arXiv2022

MogFace: Towards a Deeper Appreciation on Face Detection

Benefiting from the pioneering design of generic object detectors, significant achievements have been made in the field of face detection. Typically, the architectures of the backbone, feature pyramid layer, and detection head module within the face detector all assimilate the excellent experience from general object detectors. However, several effective methods, including label assignment and scale-level data augmentation strategy, fail to maintain consistent superiority when applying on the face detector directly. Concretely, the former strategy involves a vast body of hyper-parameters and the latter one suffers from the challenge of scale distribution bias between different detection tasks, which both limit their generalization abilities. Furthermore, in order to provide accurate face bounding boxes for facial down-stream tasks, the face detector imperatively requires the elimination of false alarms. As a result, practical solutions on label assignment, scale-level data augmentation, and reducing false alarms are necessary for advancing face detectors. In this paper, we focus on resolving three aforementioned challenges that exiting methods are difficult to finish off and present a novel face detector, termed MogFace. In our Mogface, three key components, Adaptive Online Incremental Anchor Mining Strategy, Selective Scale Enhancement Strategy and Hierarchical Context-Aware Module, are separately proposed to boost the performance of face detectors. Finally, to the best of our knowledge, our MogFace is the best face detector on the Wider Face leader-board, achieving all champions across different testing scenarios. The code is available at \url{https://github.com/damo-cv/MogFace}.

cs.CV↗

arXiv2022

Learning Where to Learn in Cross-View Self-Supervised Learning

Self-supervised learning (SSL) has made enormous progress and largely narrowed the gap with the supervised ones, where the representation learning is mainly guided by a projection into an embedding space. During the projection, current methods simply adopt uniform aggregation of pixels for embedding; however, this risks involving object-irrelevant nuisances and spatial misalignment for different augmentations. In this paper, we present a new approach, Learning Where to Learn (LEWEL), to adaptively aggregate spatial information of features, so that the projected embeddings could be exactly aligned and thus guide the feature learning better. Concretely, we reinterpret the projection head in SSL as a per-pixel projection and predict a set of spatial alignment maps from the original features by this weight-sharing projection head. A spectrum of aligned embeddings is thus obtained by aggregating the features with spatial weighting according to these alignment maps. As a result of this adaptive alignment, we observe substantial improvements on both image-level prediction and dense prediction at the same time: LEWEL improves MoCov2 by 1.6%/1.3%/0.5%/0.4% points, improves BYOL by 1.3%/1.3%/0.7%/0.6% points, on ImageNet linear/semi-supervised classification, Pascal VOC semantic segmentation, and object detection, respectively.

cs.CV↗

arXiv2022

Searching for Network Width with Bilaterally Coupled Network

Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constraints. To fulfill the searching, a one-shot supernet is usually leveraged to efficiently evaluate the performance \wrt~different network widths. However, current methods mainly follow a \textit{unilaterally augmented} (UA) principle for the evaluation of each width, which induces the training unfairness of channels in supernet. In this paper, we introduce a new supernet called Bilaterally Coupled Network (BCNet) to address this issue. In BCNet, each channel is fairly trained and responsible for the same amount of network widths, thus each network width can be evaluated more accurately. Besides, we propose to reduce the redundant search space and present the BCNetV2 as the enhanced supernet to ensure rigorous training fairness over channels. Furthermore, we leverage a stochastic complementary strategy for training the BCNet, and propose a prior initial population sampling method to boost the performance of the evolutionary search. We also propose the first open-source width benchmark on macro structures named Channel-Bench-Macro for the better comparison of width search algorithms. Extensive experiments on benchmark CIFAR-10 and ImageNet datasets indicate that our method can achieve state-of-the-art or competing performance over other baseline methods. Moreover, our method turns out to further boost the performance of NAS models by refining their network widths. For example, with the same FLOPs budget, our obtained EfficientNet-B0 achieves 77.53\% Top-1 accuracy on ImageNet dataset, surpassing the performance of original setting by 0.65\%.

cs.CV↗

arXiv2022

Relativistic origin of Hertz-form and extended Hertz-form equations for Maxwell theory of electromagnetism

We show explicitly that the Hertz-form Maxwell's equations and their extensions can be obtained from the non-relativistic expansion of Lorentz transformation of Maxwell's equations. The explicit expression for the parameter $α$ in the extended Hertz-form equations can be derived from such a non-relativistic expansion. The extended Hertz-form equations, which do not preserve Galilean invariance, origin from Lorentz transformation of Maxwell's equations and differ from the Galilean-transformed Maxwell equations (the original Hertz equations) by the relative sign differences between the two $α$ terms etc. Especially, the $α$ parameter is of relativistic origin. The superluminal behavior illustrated by the D'Alembert equation from the extended Hertz-form equations should be removed by including all subleading contributions in the $v/c$ expansion, although such a superluminal behavior will not occur in the vacuum because $α=0$. We should note that in the Hertz form and extended Hertz form equations, the electromagnetic fields should take the forms $ \vec{\mathcal{E}}(x)=\vec{E}(Λ^{-1}x)$ and $ \vec{\mathcal{B}}(x)=\vec{B}(Λ^{-1}x)$. Such a choice of description for the fields is different from the ordinary one with $\vec{E}(x)$ and $\vec{B}(x)$, which are well known to satisfy the ordinary Maxwell's equations. The descriptions of electromagnetic phenomena using the function set $\{\vec{\mathcal{E}}(x),\vec{\mathcal{B}}(x)\}$ and the function set $(\vec{E}(x),\vec{B}(x))$ are equivalent, with the $\{\vec{\mathcal{E}}(x),\vec{\mathcal{B}}(x)\}$ description satisfying the extended Hertz-form Maxwell's equations in the low speed approximation. The solution of (extended) Hertz-form Maxwell's equations describe the traveling wave form electromagnetic field.

physics.class-ph↗

arXiv2022

DyRep: Bootstrapping Training with Dynamic Re-parameterization

Structural re-parameterization (Rep) methods achieve noticeable improvements on simple VGG-style networks. Despite the prevalence, current Rep methods simply re-parameterize all operations into an augmented network, including those that rarely contribute to the model's performance. As such, the price to pay is an expensive computational overhead to manipulate these unnecessary behaviors. To eliminate the above caveats, we aim to bootstrap the training with minimal cost by devising a dynamic re-parameterization (DyRep) method, which encodes Rep technique into the training process that dynamically evolves the network structures. Concretely, our proposal adaptively finds the operations which contribute most to the loss in the network, and applies Rep to enhance their representational capacity. Besides, to suppress the noisy and redundant operations introduced by Rep, we devise a de-parameterization technique for a more compact re-parameterization. With this regard, DyRep is more efficient than Rep since it smoothly evolves the given network instead of constructing an over-parameterized network. Experimental results demonstrate our effectiveness, e.g., DyRep improves the accuracy of ResNet-18 by $2.04\%$ on ImageNet and reduces $22\%$ runtime over the baseline. Code is available at: https://github.com/hunto/DyRep.

cs.CV↗

arXiv2022

SimMatch: Semi-supervised Learning with Similarity Matching

Learning with few labeled data has been a longstanding problem in the computer vision and machine learning research community. In this paper, we introduced a new semi-supervised learning framework, SimMatch, which simultaneously considers semantic similarity and instance similarity. In SimMatch, the consistency regularization will be applied on both semantic-level and instance-level. The different augmented views of the same instance are encouraged to have the same class prediction and similar similarity relationship respected to other instances. Next, we instantiated a labeled memory buffer to fully leverage the ground truth labels on instance-level and bridge the gaps between the semantic and instance similarities. Finally, we proposed the \textit{unfolding} and \textit{aggregation} operation which allows these two similarities be isomorphically transformed with each other. In this way, the semantic and instance pseudo-labels can be mutually propagated to generate more high-quality and reliable matching targets. Extensive experimental results demonstrate that SimMatch improves the performance of semi-supervised learning tasks across different benchmark datasets and different settings. Notably, with 400 epochs of training, SimMatch achieves 67.2\%, and 74.4\% Top-1 Accuracy with 1\% and 10\% labeled examples on ImageNet, which significantly outperforms the baseline methods and is better than previous semi-supervised learning frameworks. Code and pre-trained models are available at https://github.com/KyleZheng1997/simmatch.

cs.CV↗

arXiv2022

Traveling Wave Form Description for Dirac Field and Its Deduction To Pauli Equation Type Forms in Quantum Mechanics

We derive an equivalent traveling wave form description for Dirac field. In the non-relativistic limit, such form can reduce to inverse-Galilean transformed Schrodinger-type equation. We find that, the resulting two-component Schrodinger-type equation from the reduction of traveling wave form description of Dirac field is different to the naive Galilean transformed Schrodinger equation. Taking into account the interactions of the system to electromagnetic field by adding proper forms of covariant derivative, the traveling wave form description for Pauli equation can be similarly obtained in the non-relativistic limit. Such descriptions allow one to choose arbitrary convenient reference frame for quantum system involving spins. Using Bargmann-Wigner formalism for field with arbitrary spin $s\geq 1/2$, which satisfy Dirac-type equations in all its indices, the traveling wave description for such a field can be similarly obtained from the traveling wave form description of Dirac field, for example, for the spin-3/2 Rarita-Schwinger field and spin-2 gravitational field.

quant-ph↗

arXiv2022

The growth mechanism of boundary layers for the 2D Navier-Stokes equations

We give a detailed description of formation of the boundary layers in the inviscid limit problem. To be more specific, we prove that the magnitude of the vorticity near the boundary is growing to the size of $1/\sqrtν$ and the width of the layer is spreading out to be proportional the $\sqrtν$ in a finite time period. In fact, the growth time scaling is almost $ν$

math.AP↗

arXiv2022

CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoising

Low-dose computed tomography (LDCT) denoising is an important problem in CT research. Compared to the normal dose CT (NDCT), LDCT images are subjected to severe noise and artifacts. Recently in many studies, vision transformers have shown superior feature representation ability over convolutional neural networks (CNNs). However, unlike CNNs, the potential of vision transformers in LDCT denoising was little explored so far. To fill this gap, we propose a Convolution-free Token2Token Dilated Vision Transformer for low-dose CT denoising. The CTformer uses a more powerful token rearrangement to encompass local contextual information and thus avoids convolution. It also dilates and shifts feature maps to capture longer-range interaction. We interpret the CTformer by statically inspecting patterns of its internal attention maps and dynamically tracing the hierarchical attention flow with an explanatory graph. Furthermore, an overlapped inference mechanism is introduced to effectively eliminate the boundary artifacts that are common for encoder-decoder-based denoising models. Experimental results on Mayo LDCT dataset suggest that the CTformer outperforms the state-of-the-art denoising methods with a low computation overhead.

eess.IV↗

arXiv2022

Relational Surrogate Loss Learning

Evaluation metrics in machine learning are often hardly taken as loss functions, as they could be non-differentiable and non-decomposable, e.g., average precision and F1 score. This paper aims to address this problem by revisiting the surrogate loss learning, where a deep neural network is employed to approximate the evaluation metrics. Instead of pursuing an exact recovery of the evaluation metric through a deep neural network, we are reminded of the purpose of the existence of these evaluation metrics, which is to distinguish whether one model is better or worse than another. In this paper, we show that directly maintaining the relation of models between surrogate losses and metrics suffices, and propose a rank correlation-based optimization method to maximize this relation and learn surrogate losses. Compared to previous works, our method is much easier to optimize and enjoys significant efficiency and performance gains. Extensive experiments show that our method achieves improvements on various tasks including image classification and neural machine translation, and even outperforms state-of-the-art methods on human pose estimation and machine reading comprehension tasks. Code is available at: https://github.com/hunto/ReLoss.

cs.LG↗

arXiv2022

Evolution of Dopant-Concentration-Induced Magnetic Exchange Interaction in Topological Insulator Thin Films

Two essential ingredients for the quantum anomalous Hall (QAH) effect, i.e. topological and magnetic orders, can be combined by doping magnetic ions into a topological insulator (TI) film. Through this approach, the QAH effect has been realized in chromium (Cr)- and/or vanadium (V)-doped TI (Bi,Sb)2Te3 thin films. In this work, we synthesize both V- and Cr-doped Bi2Te3 thin films with controlled dopant concentration using molecular beam epitaxy (MBE). By performing magneto-transport measurements, we find that both systems show an unusual but yet similar ferromagnetic response with respect to magnetic dopant concentration, specifically the Curie temperature does not increase monotonically but shows a local maximum at a critical dopant concentration. Our angle-resolved photoemission spectroscopy (ARPES) measurements show that the Cr/V doping introduces hole carriers into Bi2Te3, which consequently move the chemical potential toward the charge neutral point. In addition, the Cr/V doping also reduces the spin-orbit coupling of Bi2Te3 which drives it from a nontrivial TI to a trivial semiconductor. The unusual ferromagnetic response observed in Cr/V-doped Bi2Te3 thin films is attributed to the dopant-concentration-induced magnetic exchange interaction, which displays the evolution from the van Vleck-type ferromagnetism in a nontrivial magnetic TI to the Ruderman-Kittel-Kasuya-Yosida (RKKY)-type ferromagnetism in a trivial diluted magnetic semiconductor. Our work provides insights into the ferromagnetic properties of magnetically doped TI thin films and facilitates the pursuit of high-temperature QAH effect.

cond-mat.mes-hall↗

arXiv2022

Deep Petrov-Galerkin Method for Solving Partial Differential Equations

Deep neural networks are powerful tools for approximating functions, and they are applied to successfully solve various problems in many fields. In this paper, we propose a neural network-based numerical method to solve partial differential equations. In this new framework, the method is designed on weak formulations, and the unknown functions are approximated by deep neural networks and test functions can be chosen by different approaches, for instance, basis functions of finite element methods, neural networks, and so on. Because the spaces of trial function and test function are different, we name this new approach by Deep Petrov-Galerkin Method (DPGM). The resulted linear system is not necessarily to be symmetric and square, so the discretized problem is solved by a least-square method. Take the Poisson problem as an example, mixed DPGMs based on several mixed formulations are proposed and studied as well. In addition, we apply the DPGM to solve two classical time-dependent problems based on the space-time approach, that is, the unknown function is approximated by a neural network, in which temporal variable and spatial variables are treated equally, and the initial conditions are regarded as boundary conditions for the space-time domain. Finally, several numerical examples are presented to show the performance of the DPGMs, and we observe that this new method outperforms traditional numerical methods in several aspects.

math.NA↗

arXiv2022

A Privacy-Preserving Unsupervised Domain Adaptation Framework for Clinical Text Analysis

Unsupervised domain adaptation (UDA) generally aligns the unlabeled target domain data to the distribution of the source domain to mitigate the distribution shift problem. The standard UDA requires sharing the source data with the target, having potential data privacy leaking risks. To protect the source data's privacy, we first propose to share the source feature distribution instead of the source data. However, sharing only the source feature distribution may still suffer from the membership inference attack who can infer an individual's membership by the black-box access to the source model. To resolve this privacy issue, we further study the under-explored problem of privacy-preserving domain adaptation and propose a method with a novel differential privacy training strategy to protect the source data privacy. We model the source feature distribution by Gaussian Mixture Models (GMMs) under the differential privacy setting and send it to the target client for adaptation. The target client resamples differentially private source features from GMMs and adapts on target data with several state-of-art UDA backbones. With our proposed method, the source data provider could avoid leaking source data privacy during domain adaptation as well as reserve the utility. To evaluate our proposed method's utility and privacy loss, we apply our model on a medical report disease label classification task using two noisy challenging clinical text datasets. The results show that our proposed method can preserve source data's privacy with a minor performance influence on the text classification task.

cs.CL↗

arXiv2022

Forecast-based Multi-aspect Framework for Multivariate Time-series Anomaly Detection

Today's cyber-world is vastly multivariate. Metrics collected at extreme varieties demand multivariate algorithms to properly detect anomalies. However, forecast-based algorithms, as widely proven approaches, often perform sub-optimally or inconsistently across datasets. A key common issue is they strive to be one-size-fits-all but anomalies are distinctive in nature. We propose a method that tailors to such distinction. Presenting FMUAD - a Forecast-based, Multi-aspect, Unsupervised Anomaly Detection framework. FMUAD explicitly and separately captures the signature traits of anomaly types - spatial change, temporal change and correlation change - with independent modules. The modules then jointly learn an optimal feature representation, which is highly flexible and intuitive, unlike most other models in the category. Extensive experiments show our FMUAD framework consistently outperforms other state-of-the-art forecast-based anomaly detectors.

cs.LG↗

arXiv2022

Low energy supersymmetry confronted with current experiments: an overview

This study provides a brief overview of low-energy supersymmetry (SUSY) in light of current experimental constraints, such as collider searches, dark matter searches, and muon $g-2$ measurements. In addition, we survey a variety of low energy supersymmetric models: the phenomenological minimal supersymmetric model (MSSM); the supersymmetric models with cut-off-scale boundary conditions, i.e., the minimal supergravity (mSUGRA) or the constrained MSSM (CMSSM), the gauge mediation of SUSY breaking (GMSB), and the anomaly mediation of SUSY breaking (AMSB), as well as their extensions. The conclusion is that the low energy SUSY can survive all current experimental constraints and remains compelling, albeit suffering from a little fine-tuning problem. The fancy models like mSUGRA, GMSB, and AMSB need to be extended if the muon $g-2$ anomaly comes from new physics.

hep-ph↗