arXiv subjects

Kai Wang

Publications and source records attributed to Kai Wang.

At least 631 records · Page 35Linked to original sources

arXiv2022

MSDN: Mutually Semantic Distillation Network for Zero-Shot Learning

The key challenge of zero-shot learning (ZSL) is how to infer the latent semantic knowledge between visual and attribute features on seen classes, and thus achieving a desirable knowledge transfer to unseen classes. Prior works either simply align the global features of an image with its associated class semantic vector or utilize unidirectional attention to learn the limited latent semantic representations, which could not effectively discover the intrinsic semantic knowledge e.g., attribute semantics) between visual and attribute features. To solve the above dilemma, we propose a Mutually Semantic Distillation Network (MSDN), which progressively distills the intrinsic semantic representations between visual and attribute features for ZSL. MSDN incorporates an attribute$\rightarrow$visual attention sub-net that learns attribute-based visual features, and a visual$\rightarrow$attribute attention sub-net that learns visual-based attribute features. By further introducing a semantic distillation loss, the two mutual attention sub-nets are capable of learning collaboratively and teaching each other throughout the training process. The proposed MSDN yields significant improvements over the strong baselines, leading to new state-of-the-art performances on three popular challenging benchmarks, i.e., CUB, SUN, and AWA2. Our codes have been available at: \url{https://github.com/shiming-chen/MSDN}.

cs.CV↗

arXiv2022

Multi-label topic classification for COVID-19 literature with Bioformer

We describe Bioformer team's participation in the multi-label topic classification task for COVID-19 literature (track 5 of BioCreative VII). Topic classification is performed using different BERT models (BioBERT, PubMedBERT, and Bioformer). We formulate the topic classification task as a sentence pair classification problem, where the title is the first sentence, and the abstract is the second sentence. Our results show that Bioformer outperforms BioBERT and PubMedBERT in this task. Compared to the baseline results, our best model increased micro, macro, and instance-based F1 score by 8.8%, 15.5%, 7.4%, respectively. Bioformer achieved the highest micro F1 and macro F1 scores in this challenge. In post-challenge experiments, we found that pretraining of Bioformer on COVID-19 articles further improves the performance.

cs.CL↗

arXiv2022

Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation

Text-based video segmentation aims to segment the target object in a video based on a describing sentence. Incorporating motion information from optical flow maps with appearance and linguistic modalities is crucial yet has been largely ignored by previous work. In this paper, we design a method to fuse and align appearance, motion, and linguistic features to achieve accurate segmentation. Specifically, we propose a multi-modal video transformer, which can fuse and aggregate multi-modal and temporal features between frames. Furthermore, we design a language-guided feature fusion module to progressively fuse appearance and motion features in each feature level with guidance from linguistic features. Finally, a multi-modal alignment loss is proposed to alleviate the semantic gap between features from different modalities. Extensive experiments on A2D Sentences and J-HMDB Sentences verify the performance and the generalization ability of our method compared to the state-of-the-art methods.

cs.CV↗

arXiv2022

Deep Q-learning of global optimizer of multiply model parameters for viscoelastic imaging

Objective: Estimation of the global optima of multiple model parameters is valuable in imaging to form a reliable diagnostic image. Given non convexity of the objective function, it is challenging to avoid from different local minima. Methods: We first formulate the global searching of multiply parameters to be a k-D move in the parametric space, and convert parameters updating to be state-action decision-making problem. We proposed a novel Deep Q-learning of Model Parameters (DQMP) method for global optimization of model parameters by updating the parameter configurations through actions that maximize a Q-value, which employs a Deep Reward Network designed to learn global reward values from both visible curve fitting errors and hidden parameter errors. Results: The DQMP method was evaluated by viscoelastic imaging on soft matter by Kelvin-Voigt fractional derivative (KVFD) modeling. In comparison to other methods, imaging of parameters by DQMP yielded the smallest errors (< 2%) to the ground truth images. DQMP was applied to viscoelastic imaging on biological tissues, which indicated a great potential of imaging on physical parameters in diagnostic applications. Conclusions: DQMP method is able to achieve global optima, yielding accurate model parameter estimates in viscoelastic imaging. Assessment of DQMP by simulation imaging and ultrasound breast imaging demonstrated the consistency, reliability of the imaged parameters, and powerful global searching ability of DQMP. Significance: DQMP method is promising for imaging of multiple parameters, and can be generalized to global optimization for many other complex nonconvex functions and imaging of physical parameters.

cs.LG↗

arXiv2022

Crafting Better Contrastive Views for Siamese Representation Learning

Recent self-supervised contrastive learning methods greatly benefit from the Siamese structure that aims at minimizing distances between positive pairs. For high performance Siamese representation learning, one of the keys is to design good contrastive pairs. Most previous works simply apply random sampling to make different crops of the same image, which overlooks the semantic information that may degrade the quality of views. In this work, we propose ContrastiveCrop, which could effectively generate better crops for Siamese representation learning. Firstly, a semantic-aware object localization strategy is proposed within the training process in a fully unsupervised manner. This guides us to generate contrastive views which could avoid most false positives (i.e., object vs. background). Moreover, we empirically find that views with similar appearances are trivial for the Siamese model training. Thus, a center-suppressed sampling is further designed to enlarge the variance of crops. Remarkably, our method takes a careful consideration of positive pairs for contrastive learning with negligible extra training overhead. As a plug-and-play and framework-agnostic module, ContrastiveCrop consistently improves SimCLR, MoCo, BYOL, SimSiam by 0.4% ~ 2.0% classification accuracy on CIFAR-10, CIFAR-100, Tiny ImageNet and STL-10. Superior results are also achieved on downstream detection and segmentation tasks when pre-trained on ImageNet-1K.

cs.CV↗

arXiv2022

CAFE: Learning to Condense Dataset by Aligning Features

Dataset condensation aims at reducing the network training effort through condensing a cumbersome training set into a compact synthetic one. State-of-the-art approaches largely rely on learning the synthetic data by matching the gradients between the real and synthetic data batches. Despite the intuitive motivation and promising results, such gradient-based methods, by nature, easily overfit to a biased set of samples that produce dominant gradients, and thus lack global supervision of data distribution. In this paper, we propose a novel scheme to Condense dataset by Aligning FEatures (CAFE), which explicitly attempts to preserve the real-feature distribution as well as the discriminant power of the resulting synthetic set, lending itself to strong generalization capability to various architectures. At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales, while accounting for the classification of real samples. Our scheme is further backed up by a novel dynamic bi-level optimization, which adaptively adjusts parameter updates to prevent over-/under-fitting. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art: on the SVHN dataset, for example, the performance gain is up to 11%. Extensive experiments and analyses verify the effectiveness and necessity of proposed designs.

cs.CV↗

arXiv2022

Detection of Prompt Fast-Variable Thermal Component in Multi-Pulse Short Gamma-Ray Burst 170206A

We report the detection of a strong thermal component in the short Gamma-Ray Burst 170206A with three intense pulses in its light curves, throughout which the fluxes of this thermal component exhibit fast temporal variability same as that of the accompanying non-thermal component. The values of the time-resolved low-energy photon index in the non-thermal component are between about -0.79 and -0.16, most of which are harder than -2/3 excepted in the synchrotron emission process. In addition, we found a common evolution between the thermal component and the non-thermal component, $E_{\rm p,CPL} \propto kT_{\rm BB}^{0.95\pm0.28}$, and $F_{\rm CPL} \propto F_{\rm BB}^{0.67\pm0.18}$, where $E_{\rm p,CPL}$ and $F_{\rm CPL}$ are the peak photon energy and corresponding flux of the non-thermal component, and $ kT_{\rm BB}$ and $F_{BB}$ are the temperature and corresponding flux of the thermal component, respectively. Finally, we proposed that the photospheric thermal emission and the Comptonization of thermal photons may be responsible for the observational features of GRB 170206A.

astro-ph.HE↗

arXiv2022

Creating boundaries along a synthetic frequency dimension

Synthetic dimensions have garnered widespread interest for implementing high dimensional classical and quantum dynamics on lower dimensional geometries. Synthetic frequency dimensions, in particular, have been used to experimentally realize a plethora of bulk physics effects, such as effective gauge potentials, nontrivial Hermitian as well as non-Hermitian topology, spin-momentum locking, complex long-range coupling, unidirectional frequency conversion, and four-dimensional lattices. However, in synthetic frequency dimensions there has not been any demonstration of boundary effects which are of paramount importance in topological physics due to the bulk edge correspondence, since systems exhibiting synthetic frequency dimensions do not support well-defined sharp boundaries. Here we theoretically elucidate a method to construct boundaries in the synthetic frequency dimension of dynamically modulated ring resonators by strongly coupling it to an auxiliary ring, and provide an experimental demonstration of this method. We experimentally explore various physics effects associated with the creation of such boundaries in the synthetic frequency dimension, including confinement of the spectrum of light, the discretization of the band structure, and the interaction of such boundaries with the topologically protected one-way chiral modes in a quantum Hall ladder. The incorporation of boundaries allows us to observe topologically robust transport of light along the frequency axis, which shows that the frequency of light can be controlled through topological concepts. Our demonstration of such sharp boundaries fundamentally expands the capability of exploring topological physics, and is also of importance for other applications such as classical and quantum information processing in synthetic frequency dimensions.

quant-ph↗

arXiv2022

Ultra-High Lithium Storage Capacity of Al2C Monolayer under Restricted Multilayered Growth Mechanism

Designing anode materials with high lithium specific capacity is crucial to the development of high energy-density lithium ion batteries. Herein, a distinctive lithium growth mechanism, namely, the restricted multilayered growth for lithium, and a strategy for lithium storage are proposed to achieve the balance between the ultra-high specific capacity and the need to avert uncontrolled dendritic growth of lithium. In particular, based on first-principles computation, we show that the Al2C monolayer with planar tetracoordinate carbon structure can be an ideal platform for realizing the restricted multilayered growth mechanism as a 2D anode material. Furthermore, the Al2C monolayer exhibits ultra-high specific capacity of lithium of 4059 mAh/g, yet with a low dif-fusion barrier of 0.039-0.17 eV as well as low open circuit voltage in the range of 0.002-0.34 V. These novel properties endow the Al2C monolayer a promising anode material for future lithium ion batteries. Our study offers a new way to design promising 2D anode materials with high specific capacity, fast lithium-ion diffusion, and safe lithium storage mechanism.

cond-mat.mtrl-sci↗

arXiv2022

Anisotropic Electrene T'-Ca2P with Electron Gas Magnetic Coupling as Anode Material for Na/K Ion Batteries

There is an urgently need for the high-performance rechargeable electrical storage devices as supplement or substitutions of lithium ion batteries due to the shortage of lithium in nature. Herein we propose a stable 2D electrene T'-Ca2P as anode material for Na/K ion batteries by first-principle calculations. Our calculated results show that T'-Ca2P monolayer is an antiferromagnetic semiconducting electrene with spin-polarized electron gas. It exhibits suitable adsorption for both Na and K atoms, and its anisotropic migration energy barriers are 0.050/0.101 eV and 0.037/0.091 eV in b/a direction, respectively. The theoretical capacities for Na and K are both 482 mAh/g, while the average working voltage platforms are 0.171-0.226 V and 0.013-0.267 V, respectively. All the results reveal that the T'-Ca2P monolayer has promised application prospects as anode materials for Na/K ion batteries.

cond-mat.mtrl-sci↗

arXiv2022

An Efficient Training Approach for Very Large Scale Face Recognition

Face recognition has achieved significant progress in deep learning era due to the ultra-large-scale and welllabeled datasets. However, training on the outsize datasets is time-consuming and takes up a lot of hardware resource. Therefore, designing an efficient training approach is indispensable. The heavy computational and memory costs mainly result from the million-level dimensionality of thefully connected (FC) layer. To this end, we propose a novel training approach, termed Faster Face Classification (F2C), to alleviate time and cost without sacrificing the performance. This method adopts Dynamic Class Pool (DCP) for storing and updating the identities features dynamically, which could be regarded as a substitute for the FC layer. DCP is efficiently time-saving and cost-saving, as its smaller size with the independence from the whole face identities together. We further validate the proposed F2C method across several face benchmarks and private datasets, and display comparable results, meanwhile the speed is faster than state-of-the-art FC-based methods in terms of recognition accuracy and hardware costs. Moreover, our method is further improved by a well-designed dual data loader including indentity-based and instancebased loaders, which makes it more efficient for the updating DCP parameters.

cs.CV↗

arXiv2022

Magnetic field tuning of the valley population in the Weyl phase of Nd$_2$Ir$_2$O$_7$

The frustrated magnet Nd$_2$Ir$_2$O$_7$, where strong correlations together with spin-orbit coupling play a crucial role, is predicted to be a Weyl semimetal and to host topological pairs of bulk Dirac-like valleys. Here we use an external magnetic field to manipulate the localized rare earth 4f moments coupled to the 5d electronic bands. Low energy optical spectroscopy reveals that a field of only a few teslas suffices to create charge compensating pockets of holes and electrons in different regions of momentum space, thus introducing a valley population shift that can be tuned with the field.

cond-mat.str-el↗

arXiv2022

Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information

Deep learning-based pronunciation scoring models highly rely on the availability of the annotated non-native data, which is costly and has scalability issues. To deal with the data scarcity problem, data augmentation is commonly used for model pretraining. In this paper, we propose a phone-level mixup, a simple yet effective data augmentation method, to improve the performance of word-level pronunciation scoring. Specifically, given a phoneme sequence from lexicon, the artificial augmented word sample can be generated by randomly sampling from the corresponding phone-level features in training data, while the word score is the average of their GOP scores. Benefit from the arbitrary phone-level combination, the mixup is able to generate any word with various pronunciation scores. Moreover, we utilize multi-source information (e.g., MFCC and deep features) to further improve the scoring system performance. The experiments conducted on the Speechocean762 show that the proposed system outperforms the baseline by adding the mixup data for pretraining, with Pearson correlation coefficients (PCC) increasing from 0.567 to 0.61. The results also indicate that proposed method achieves similar performance by using 1/10 unlabeled data of baseline. In addition, the experimental results also demonstrate the efficiency of our proposed multi-source approach.

eess.AS↗

arXiv2022

The Large High Altitude Air Shower Observatory (LHAASO) Science Book (2021 Edition)

Since the science white paper of the Large High Altitude Air Shower Observatory (LHAASO) published on arXiv in 2019 [e-Print: 1905.02773 (astro-ph.HE)], LHAASO has completed the transition from a project to an operational gamma-ray astronomical observatory LHAASO is a new generation multi-component facility located in Daocheng, Sichuan province of China, at an altitude of 4410 meters. It aims at measuring with unprecedented sensitivity the spectrum, composition, and anisotropy of cosmic rays in the energy range between 10$^{12}$ and 10$^{18}$~eV, and acting simultaneously as a wide aperture (one stereoradiant) continuously operating gamma-ray telescope in the energy range between 10$^{11}$ and $10^{15}$~eV with the designed sensitivity of 1.3\% of the Crab Unit (CU) above 100 TeV. LHAASO's capability of measuring simultaneously different shower components (electrons, muons, and Cherenkov/fluorescence light), will allow it to investigate the origin, acceleration, and propagation of CR through measurement of the energy spectrum, elemental composition, and anisotropy with unprecedented resolution. The remarkable sensitivity of LHAASO will play a key role in CR physics and gamma-ray astronomy for a general and comprehensive exploration of the high energy universe and will allow important studies of fundamental physics (such as indirect dark matter search, Lorentz invariance violation, quantum gravity) and solar and heliospheric physics. The LHAASO Collaboration organized an editorial working group and finished all editorial work of this science book, to summarize the instrumental features and outline the prospects of scientific researches with the LHAASO experiment.

astro-ph.HE↗

arXiv2022

Tensor Theory for Higher Dimensional Chern Insulators with Large Chern Numbers

Recent advances in topological artificial systems open the door to realizing topological states in dimensions higher than the usual three-dimensional space. Here, we present a "tensor product" theory, which offers a method to construct Chern insulators with arbitrarily high dimensions and Chern numbers. Particularly, we show that the tensor product of a $d_A$D Chern insulator $\langle \mathcal{H}_A^{(κ_{A})}, C_A\rangle$ with a $d_B$D Chern insulator $\langle \mathcal{H}_B^{(κ_B)}, C_B\rangle$ leads to a $(d_A+d_B)$D Chern insulator $\langle \mathcal{H}_{A B}^{(κ_A\star κ_B)},-2C_AC_B\rangle $, where in the brackets, $\mathcal{H}^{(κ)}$ is the $d$D Hamiltonian with $d$ even, $C$ is the corresponding $(d/2)$th Chern number, and $κ$ labels the five non-chiral Altland-Zirnbauer symmetry classes A, AI, D, AII and C. The four real classes AI, D, AII and C form a Klein four-group under the multiplication `$\star$' with class AI the identity, and class A is the zero element. Our theory leads to novel higher-dimensional topological physics. (i) The construction can generate large higher-order Chern numbers, e.g., for some cases the resultant classification is $8\mathbb{Z}$. (ii) Fascinatingly, the boundary states feature flat nodal hypersurfaces with nontrivial Chern charges. For the constructed $(d_A+d_B)$D Chern insulator, a boundary perpendicular to a direction of $\mathcal{H}_A$ generically hosts $|C_A|$ $d_B$D nodal hypersurfaces, each of which has topological charge $\pm 2C_B$. Under perturbations, each nodal hypersurface bursts into stable unit nodal points, with the total Chern charge conserved. Examples are given to demonstrate our theory, which can be experimentally realized in artificial systems such as acoustic crystals, electric circuit arrays, ultracold atoms, or mechanical networks.

cond-mat.mes-hall↗

arXiv2022

Weighted $L_p$ Markov factors with doubling weights on the ball

Let $L_{p,w},\ 1 \le p<\infty,$ denote the weighted $L_p$ space of functions on the unit ball $\Bbb B^d$ with a doubling weight $w$ on $\Bbb B^d$. The Markov factor for $L_{p,w}$ on a polynomial $P$ is defined by $\frac{\|\, |\nabla P|\,\|_{p,w}}{\|P\|_{p,w}}$, where $\nabla P$ is the gradient of $P$. We investigate the worst case Markov factors for $L_{p,w}\ (1\le p<\infty)$ and obtain that the degree of these factors are at most $2$. In particular, for the Jacobi weight $w_μ(x)=(1-|x|^2)^{μ-1/2}, \ μ\ge0$, the exponent $2$ is sharp. We also study the average case Markov factor for $L_{2,w}$ on random polynomials with independent $N(0, σ^2)$ coefficients and obtain that the upper bound of the average (expected) Markov factor is order degree to the $3/2$, as compared to the degree squared worst case upper bound.

math.CA↗

arXiv2022

Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning

In this paper, we provide a theory of using graph neural networks (GNNs) for multi-node representation learning (where we are interested in learning a representation for a set of more than one node, such as link). We know that GNN is designed to learn single-node representations. When we want to learn a node set representation involving multiple nodes, a common practice in previous works is to directly aggregate the single-node representations obtained by a GNN into a joint node set representation. In this paper, we show a fundamental constraint of such an approach, namely the inability to capture the dependence between nodes in the node set, and argue that directly aggregating individual node representations does not lead to an effective joint representation for multiple nodes. Then, we notice that a few previous successful works for multi-node representation learning, including SEAL, Distance Encoding, and ID-GNN, all used node labeling. These methods first label nodes in the graph according to their relationships with the target node set before applying a GNN. Then, the node representations obtained in the labeled graph are aggregated into a node set representation. By investigating their inner mechanisms, we unify these node labeling techniques into a single and most general form -- labeling trick. We prove that with labeling trick a sufficiently expressive GNN learns the most expressive node set representations, thus in principle solves any joint learning tasks over node sets. Experiments on one important two-node representation learning task, link prediction, verified our theory. Our work explains the superior performance of previous node-labeling-based methods, and establishes a theoretical foundation of using GNNs for multi-node representation learning.

cs.LG↗

arXiv2022

Role of surface functional groups to superconductivity in Nb$_2$C-MXene: Experiments and density functional theory calculations

The recently discovered surface-group-dependent superconductivity in Nb$_2$C-MXene fabricated by the molten salts method is attracting wide attention. However, regarding the superconductivity of Nb$_2$C-MXene with functional F groups (Nb$_2$CF$_x$), there were some conflicting results in experimental and theoretical studies. Herein, we systematically carried out experimental and theoretical investigations on the superconductivity in Nb$_2$C-MXene with the Cl functional group (Nb$_2$CCl$_x$) and Nb$_2$CF$_x$. The experimental results of the Meissner effect and zero resistivity have proved that Nb$_2$CClx is superconducting with the transition temperature (Tc) ~ 5.2 K. We extract its superconducting parameters from the temperature dependence of resistivity and the field dependence of the magnetization. The Ginzburg-Landau parameter K$_G$$_L$ is estimated to be 2.41, indicating that Nb$_2$CClx is a typical type-II superconductor. Conversely, both magnetic and electrical transport measurements demonstrate that Nb$_2$CF$_x$ is not superconducting. The first-principles density functional theory (DFT) calculations show that the Tc of Nb$_2$Cl$_x$ is ~ 5.2 K, while Nb$_2$CF$_x$ is dynamically unstable with imaginary frequency in phonon spectrum, which is in good agreement with the experimental results. Our studies not only are useful for clarifying the present inconsistency but also offer referential significance for future investigations on the superconductivity of MXenes.

cond-mat.supr-con↗