arXiv subjects

Yang Wang

Publications and source records attributed to Yang Wang.

At least 541 records · Page 30Linked to original sources

arXiv2022

Transformer-Empowered 6G Intelligent Networks: From Massive MIMO Processing to Semantic Communication

It is anticipated that 6G wireless networks will accelerate the convergence of the physical and cyber worlds and enable a paradigm-shift in the way we deploy and exploit communication networks. Machine learning, in particular deep learning (DL), is expected to be one of the key technological enablers of 6G by offering a new paradigm for the design and optimization of networks with a high level of intelligence. In this article, we introduce an emerging DL architecture, known as the transformer, and discuss its potential impact on 6G network design. We first discuss the differences between the transformer and classical DL architectures, and emphasize the transformer's self-attention mechanism and strong representation capabilities, which make it particularly appealing for tackling various challenges in wireless network design. Specifically, we propose transformer-based solutions for various massive multiple-input multiple-output (MIMO) and semantic communication problems, and show their superiority compared to other architectures. Finally, we discuss key challenges and open issues in transformer-based solutions, and identify future research directions for their deployment in intelligent 6G networks.

cs.IT↗

arXiv2022

A Joint Framework Towards Class-aware and Class-agnostic Alignment for Few-shot Segmentation

Few-shot segmentation (FSS) aims to segment objects of unseen classes given only a few annotated support images. Most existing methods simply stitch query features with independent support prototypes and segment the query image by feeding the mixed features to a decoder. Although significant improvements have been achieved, existing methods are still face class biases due to class variants and background confusion. In this paper, we propose a joint framework that combines more valuable class-aware and class-agnostic alignment guidance to facilitate the segmentation. Specifically, we design a hybrid alignment module which establishes multi-scale query-support correspondences to mine the most relevant class-aware information for each query image from the corresponding support features. In addition, we explore utilizing base-classes knowledge to generate class-agnostic prior mask which makes a distinction between real background and foreground by highlighting all object regions, especially those of unseen classes. By jointly aggregating class-aware and class-agnostic alignment guidance, better segmentation performances are obtained on query images. Extensive experiments on PASCAL-$5^i$ and COCO-$20^i$ datasets demonstrate that our proposed joint framework performs better, especially on the 1-shot setting.

cs.CV↗

arXiv2022

Joint Semantic Transfer Network for IoT Intrusion Detection

In this paper, we propose a Joint Semantic Transfer Network (JSTN) towards effective intrusion detection for large-scale scarcely labelled IoT domain. As a multi-source heterogeneous domain adaptation (MS-HDA) method, the JSTN integrates a knowledge rich network intrusion (NI) domain and another small-scale IoT intrusion (II) domain as source domains, and preserves intrinsic semantic properties to assist target II domain intrusion detection. The JSTN jointly transfers the following three semantics to learn a domain-invariant and discriminative feature representation. The scenario semantic endows source NI and II domain with characteristics from each other to ease the knowledge transfer process via a confused domain discriminator and categorical distribution knowledge preservation. It also reduces the source-target discrepancy to make the shared feature space domain-invariant. Meanwhile, the weighted implicit semantic transfer boosts discriminability via a fine-grained knowledge preservation, which transfers the source categorical distribution to the target domain. The source-target divergence guides the importance weighting during knowledge preservation to reflect the degree of knowledge learning. Additionally, the hierarchical explicit semantic alignment performs centroid-level and representative-level alignment with the help of a geometric similarity-aware pseudo-label refiner, which exploits the value of unlabelled target II domain and explicitly aligns feature representations from a global and local perspective in a concentrated manner. Comprehensive experiments on various tasks verify the superiority of the JSTN against state-of-the-art comparing methods, on average a 10.3% of accuracy boost is achieved. The statistical soundness of each constituting component and the computational efficiency are also verified.

cs.CR↗

arXiv2022

The dispersion measure of Fast Radio Bursts host galaxies: estimation from cosmological simulations

The dispersion measure(DM) of fast radio burst encodes important information such as its distance, properties of intervening medium. Based on simulations in the Illustris and IllustrisTNG projects, we analyze the DM of FRBs contributed by the interstellar medium and circumgalactic medium in the hosts, $\rm{DM_{host}}$. We explore two population models - tracing the star formation rate (SFR), and the stellar mass, i.e. young and old progenitors respectively. The distribution of $\rm{DM_{host}}$ shows significant differences at $z=0$ between two populations: the stellar mass model exhibits an excess at the low DM end with respect to the SFR model. The SFR (stellar mass) model has a median value of $\rm{DM_{host}}$=179 (63) $\rm{pc\, cm^{-3}}$ for galaxies with $M_*=10^{8-13}\,M_{\odot}$ in the TNG100-1. Galaxies in the Illustris-1 have a much smaller $\rm{DM_{host}}$. The distributions of $\rm{DM_{host}}$ deviate from log-normal function for both models. Furthermore, two populations differ moderately in the spatial offset from host galaxy's center, in the stellar mass function of hosts. $\rm{DM_{host}}$ increases with the stellar mass of hosts when $M_*<10^{10.5}\,M_{\odot}$, and fluctuate at higher mass. At $0<z<2$, $\rm{DM_{host}}$ increases with redshift. The differences in $\rm{DM_{host}}$ between two populations declines with increasing redshift. With more localized events available in the future, statistics such as $\rm{DM_{host}}$, the offset from galaxy center and the stellar mass function of hosts will be of great helpful to ascertain the origin of FRB. Meanwhile, statistics of $\rm{DM_{host}}$ of localized FRB events could help to constrain the baryon physics models in galaxy evolution.

astro-ph.HE↗

arXiv2022

Video Summarization Overview

With the broad growth of video capturing devices and applications on the web, it is more demanding to provide desired video content for users efficiently. Video summarization facilitates quickly grasping video content by creating a compact summary of videos. Much effort has been devoted to automatic video summarization, and various problem settings and approaches have been proposed. Our goal is to provide an overview of this field. This survey covers early studies as well as recent approaches which take advantage of deep learning techniques. We describe video summarization approaches and their underlying concepts. We also discuss benchmarks and evaluations. We overview how prior work addressed evaluation and detail the pros and cons of the evaluation protocols. Last but not least, we discuss open challenges in this field.

cs.CV↗

arXiv2022

Aging Channel Modeling and Transmission Block Size Optimization for Massive MIMO Vehicular Networks in Non-Isotropic Scattering Environment

We investigate the effect of channel aging on multi-cell massive multiple-input multiple-output (MIMO) vehicular networks in a generic non-isotropic scattering environment. Based on the single cluster scattering assumption and the von Mises distribution assumptions of the scatterers' angles, an aging channel model is established to capture the joint effect of spatial and temporal correlations resulting from different angular spread conditions in various application scenarios. Expressions of the user uplink transmission spectral efficiency (SE) are derived for maximum ratio (MR) and minimum mean square error (MMSE) combining. Through numerical studies, the area spectral efficiency (ASE) performance of the network is evaluated in freeway and urban Manhattan road grid scenarios, and easy-to-use empirical models for the optimal transmission block size for ASE maximization are obtained for the evaluated scenarios.

cs.IT↗

arXiv2022

Few-Shot Learning of Compact Models via Task-Specific Meta Distillation

We consider a new problem of few-shot learning of compact models. Meta-learning is a popular approach for few-shot learning. Previous work in meta-learning typically assumes that the model architecture during meta-training is the same as the model architecture used for final deployment. In this paper, we challenge this basic assumption. For final deployment, we often need the model to be small. But small models usually do not have enough capacity to effectively adapt to new tasks. In the mean time, we often have access to the large dataset and extensive computing power during meta-training since meta-training is typically performed on a server. In this paper, we propose task-specific meta distillation that simultaneously learns two models in meta-learning: a large teacher model and a small student model. These two models are jointly learned during meta-training. Given a new task during meta-testing, the teacher model is first adapted to this task, then the adapted teacher model is used to guide the adaptation of the student model. The adapted student model is used for final deployment. We demonstrate the effectiveness of our approach in few-shot image classification using model-agnostic meta-learning (MAML). Our proposed method outperforms other alternatives on several benchmark datasets.

cs.LG↗

arXiv2022

Analysis of ab-initio total energies obtained by different DFT implementations

Ab-initio crystal structure prediction depends on accurate calculation of the energies of competing structures. Many DFT codes are available that utilize different approaches to solve the Kohn-Sham equation. We evaluate the consistency of three software packages (WIEN2k, VASP and MuST) that utilize three different methods (FL-APW, plane-wave pseudopotential and the KKR-Green's Function methods) by comparing the relative total energies obtained for a set of BCC and FCC binary metallic alloys. We focus on the impact of choices such as muffin-tin $vs.$ full-potential, angular momentum cutoff and other important KKR parameters. Different alloy systems prove more or less sensitive to these choices, and we explain the differences through study of the angular variation of their potentials. Our results can provide guidance in the application of KKR as a total energy method for structure prediction.

cond-mat.mtrl-sci↗

arXiv2022

FAQS: Communication-efficient Federate DNN Architecture and Quantization Co-Search for personalized Hardware-aware Preferences

Due to user privacy and regulatory restrictions, federate learning (FL) is proposed as a distributed learning framework for training deep neural networks (DNN) on decentralized data clients. Recent advancements in FL have applied Neural Architecture Search (NAS) to replace the predefined one-size-fit-all DNN model, which is not optimal for all tasks of various data distributions, with searchable DNN architectures. However, previous methods suffer from expensive communication cost rasied by frequent large model parameters transmission between the server and clients. Such difficulty is further amplified when combining NAS algorithms, which commonly require prohibitive computation and enormous model storage. Towards this end, we propose FAQS, an efficient personalized FL-NAS-Quantization framework to reduce the communication cost with three features: weight-sharing super kernels, bit-sharing quantization and masked transmission. FAQS has an affordable search time and demands very limited size of transmitted messages at each round. By setting different personlized pareto function loss on local clients, FAQS can yield heterogeneous hardware-aware models for various user preferences. Experimental results show that FAQS achieves average reduction of 1.58x in communication bandwith per round compared with normal FL framework and 4.51x compared with FL+NAS framwork.

cs.LG↗

arXiv2022

HQNAS: Auto CNN deployment framework for joint quantization and architecture search

Deep learning applications are being transferred from the cloud to edge with the rapid development of embedded computing systems. In order to achieve higher energy efficiency with the limited resource budget, neural networks(NNs) must be carefully designed in two steps, the architecture design and the quantization policy choice. Neural Architecture Search(NAS) and Quantization have been proposed separately when deploying NNs onto embedded devices. However, taking the two steps individually is time-consuming and leads to a sub-optimal final deployment. To this end, we propose a novel neural network design framework called Hardware-aware Quantized Neural Architecture Search(HQNAS) framework which combines the NAS and Quantization together in a very efficient manner using weight-sharing and bit-sharing. It takes only 4 GPU hours to discover an outstanding NN policy on CIFAR10. It also takes only %10 GPU time to generate a comparable model on Imagenet compared to the traditional NAS method with 1.8x decrease of latency and a negligible accuracy loss of only 0.7%. Besides, our method can be adapted in a lifelong situation where the neural network needs to evolve occasionally due to changes of local data, environment and user preference.

cs.LG↗

arXiv2022

An efficient low-density grating setup for monochromatization of XUV ultrafast light sources

Ultrafast light sources have become an indispensable tool to access and understand transient phenomenon in material science. However, a simple and easy-to-implement method for harmonic selection, with high transmission efficiency and pulse duration conservation, is still a challenge. Here we showcase and compare two approaches for selecting the desired harmonic from a high harmonic generation source while achieving the above goals. The first approach is the combination of extreme ultraviolet spherical mirrors with transmission filters and the second approach uses a normal-incidence spherical grating. Both solutions target time- and angle-resolved photoemission spectroscopy with photon energies in the 10-20 eV range but are relevant for other experimental techniques as well. The two approaches for harmonic selection are characterized in terms of focusing quality, efficiency, and temporal broadening. It is demonstrated that a focusing grating is able to provide much higher transmission as compared to the mirror+filter approach (3.3 times higher for 10.8 eV and 12.9 times higher for 18.1 eV), with only a slight temporal broadening (6.8% increase) and a somewhat larger spot size (~30% increase). Overall, our study establishes an experimental perspective on the trade-off between a single grating normal incidence monochromator design and the use of filters. As such, it provides a basis for selecting the most appropriate approach in various fields where an easy-to-implement harmonic selection from high harmonic generation is needed.

physics.optics↗

arXiv2022

On the existence of cut points of connected generalized Sierpinski carpets

In a previous work joint with Dai and Luo, we show that a connected generalized Sierpiński carpet (or shortly a GSC) has cut points if and only if the associated $n$-th Hata graph has a long tail for all $n\geq 2$. In this paper, we extend the above result by showing that it suffices to check a finite number of those graphs to reach a conclusion. This criterion provides a truly "algorithmic" solution to the cut point problem of connected GSCs. We also construct for each $m\geq 1$ a connected GSC with exactly $m$ cut points and demonstrate that when $m\geq 2$, such a GSC must be of the so-called fragile type.

math.GN↗

arXiv2022

Hierarchical Few-Shot Object Detection: Problem, Benchmark and Method

Few-shot object detection (FSOD) is to detect objects with a few examples. However, existing FSOD methods do not consider hierarchical fine-grained category structures of objects that exist widely in real life. For example, animals are taxonomically classified into orders, families, genera and species etc. In this paper, we propose and solve a new problem called hierarchical few-shot object detection (Hi-FSOD), which aims to detect objects with hierarchical categories in the FSOD paradigm. To this end, on the one hand, we build the first large-scale and high-quality Hi-FSOD benchmark dataset HiFSOD-Bird, which contains 176,350 wild-bird images falling to 1,432 categories. All the categories are organized into a 4-level taxonomy, consisting of 32 orders, 132 families, 572 genera and 1,432 species. On the other hand, we propose the first Hi-FSOD method HiCLPL, where a hierarchical contrastive learning approach is developed to constrain the feature space so that the feature distribution of objects is consistent with the hierarchical taxonomy and the model's generalization power is strengthened. Meanwhile, a probabilistic loss is designed to enable the child nodes to correct the classification errors of their parent nodes in the taxonomy. Extensive experiments on the benchmark dataset HiFSOD-Bird show that our method HiCLPL outperforms the existing FSOD methods.

cs.CV↗

arXiv2022

Improving ProtoNet for Few-Shot Video Object Recognition: Winner of ORBIT Challenge 2022

In this work, we present the winning solution for ORBIT Few-Shot Video Object Recognition Challenge 2022. Built upon the ProtoNet baseline, the performance of our method is improved with three effective techniques. These techniques include the embedding adaptation, the uniform video clip sampler and the invalid frame detection. In addition, we re-factor and re-implement the official codebase to encourage modularity, compatibility and improved performance. Our implementation accelerates the data loading in both training and testing.

cs.CV↗

arXiv2022

Realization of high-dynamic-range broadband magnetic-field sensing with ensemble nitrogen-vacancy centers in diamond

We present a new magnetometry method integrating an ensemble of nitrogen-vacancy (NV) centers in a single-crystal diamond with an extended dynamic range for monitoring the fast changing magnetic-field. The NV-center spin resonance frequency is tracked using a closed-loop frequency locked technique with fast frequency hopping to achieve a 10 kHz measurement bandwidth, thus, allowing for the detection of fast changing magnetic signals up to 0.723 T/s.This technique exhibits an extended dynamic range subjected to the working bandwidth of the microwave source. This extended dynamic range can reach up to 4.3 mT, which is 86 times broader than the intrinsic dynamic range. The essential components for NV spin control and signal processing such as signal generation, microwave frequency control, data processing and readout are integrated in a board-level system. With this platform, we demonstrate broadband magnetometry with an optimized sensitivity of 4.2 nT-Hz-1/2. This magnetometry method has the potential to be implemented in a multichannel frequency locked vector magnetometer suitable for a wide range of practical applications such as magnetocardiography and high-precision current sensors.

quant-ph↗

arXiv2022

S-Rocket: Selective Random Convolution Kernels for Time Series Classification

Random convolution kernel transform (Rocket) is a fast, efficient, and novel approach for time series feature extraction using a large number of independent randomly initialized 1-D convolution kernels of different configurations. The output of the convolution operation on each time series is represented by a partial positive value (PPV). A concatenation of PPVs from all kernels is the input feature vector to a Ridge regression classifier. Unlike typical deep learning models, the kernels are not trained and there is no weighted/trainable connection between kernels or concatenated features and the classifier. Since these kernels are generated randomly, a portion of these kernels may not positively contribute in performance of the model. Hence, selection of the most important kernels and pruning the redundant and less important ones is necessary to reduce computational complexity and accelerate inference of Rocket for applications on the edge devices. Selection of these kernels is a combinatorial optimization problem. In this paper, we propose a scheme for selecting these kernels while maintaining the classification performance. First, the original model is pre-trained at full capacity. Then, a population of binary candidate state vectors is initialized where each element of a vector represents the active/inactive status of a kernel. A population-based optimization algorithm evolves the population in order to find a best state vector which minimizes the number of active kernels while maximizing the accuracy of the classifier. This activation function is a linear combination of the total number of active kernels and the classification accuracy of the pre-trained classifier with the active kernels. Finally, the selected kernels in the best state vector are utilized to train the Ridge regression classifier with the selected kernels.

cs.LG↗

arXiv2022

Delving Globally into Texture and Structure for Image Inpainting

Image inpainting has achieved remarkable progress and inspired abundant methods, where the critical bottleneck is identified as how to fulfill the high-frequency structure and low-frequency texture information on the masked regions with semantics. To this end, deep models exhibit powerful superiority to capture them, yet constrained on the local spatial regions. In this paper, we delve globally into texture and structure information to well capture the semantics for image inpainting. As opposed to the existing arts trapped on the independent local patches, the texture information of each patch is reconstructed from all other patches across the whole image, to match the coarsely filled information, specially the structure information over the masked regions. Unlike the current decoder-only transformer within the pixel level for image inpainting, our model adopts the transformer pipeline paired with both encoder and decoder. On one hand, the encoder captures the texture semantic correlations of all patches across image via self-attention module. On the other hand, an adaptive patch vocabulary is dynamically established in the decoder for the filled patches over the masked regions. Building on this, a structure-texture matching attention module anchored on the known regions comes up to marry the best of these two worlds for progressive inpainting via a probabilistic diffusion process. Our model is orthogonal to the fashionable arts, such as Convolutional Neural Networks (CNNs), Attention and Transformer model, from the perspective of texture and structure information for image inpainting. The extensive experiments over the benchmarks validate its superiority. Our code is available at https://github.com/htyjers/DGTS-Inpainting.

cs.CV↗

arXiv2022

Observation of nonlinear planar Hall effect in magnetic insulator/topological insulator heterostructures

Interfacing topological insulators (TIs) with magnetic insulators (MIs) has been widely used to study the interaction between topological surface states and magnetism. Previous transport studies typically interpret the suppression of weak antilocalization or appearance of the anomalous Hall effect as signatures of magnetic proximity effect (MPE) imposed to TIs. Here, we report the observation of nonlinear planar Hall effect (NPHE) in Bi2Se3 films grown on MI thulium and yttrium iron garnet (TmIG and YIG) substrates, which is an order of magnitude larger than that in Bi2Se3 grown on nonmagnetic gadolinium gallium garnet (GGG) substrate. The nonlinear Hall resistance in TmIG/Bi2Se3 depends linearly on the external magnetic field, while that in YIG/Bi2Se3 exhibits an extra hysteresis loop around zero field. The magnitude of the NPHE is found to scale inversely with carrier density. We speculate the observed NPHE is related to the MPE-induced exchange gap opening and out-of-plane spin textures in the TI surface states, which may be used as an alternative transport signature of the MPE in MI/TI heterostructures.

cond-mat.mes-hall↗