arXiv subjects

Yu Wang

Publications and source records attributed to Yu Wang.

At least 1,207 records · Page 67Linked to original sources

arXiv2020

Learning a Unified Sample Weighting Network for Object Detection

Region sampling or weighting is significantly important to the success of modern region-based object detectors. Unlike some previous works, which only focus on "hard" samples when optimizing the objective function, we argue that sample weighting should be data-dependent and task-dependent. The importance of a sample for the objective function optimization is determined by its uncertainties to both object classification and bounding box regression tasks. To this end, we devise a general loss function to cover most region-based object detectors with various sampling strategies, and then based on it we propose a unified sample weighting network to predict a sample's task weights. Our framework is simple yet effective. It leverages the samples' uncertainty distributions on classification loss, regression loss, IoU, and probability score, to predict sample weights. Our approach has several advantages: (i). It jointly learns sample weights for both classification and regression tasks, which differentiates it from most previous work. (ii). It is a data-driven process, so it avoids some manual parameter tuning. (iii). It can be effortlessly plugged into most object detectors and achieves noticeable performance improvements without affecting their inference time. Our approach has been thoroughly evaluated with recent object detection frameworks and it can consistently boost the detection accuracy. Code has been made available at \url{https://github.com/caiqi/sample-weighting-network}.

cs.CV↗

arXiv2020

Learning Semantic Program Embeddings with Graph Interval Neural Network

Learning distributed representations of source code has been a challenging task for machine learning models. Earlier works treated programs as text so that natural language methods can be readily applied. Unfortunately, such approaches do not capitalize on the rich structural information possessed by source code. Of late, Graph Neural Network (GNN) was proposed to learn embeddings of programs from their graph representations. Due to the homogeneous and expensive message-passing procedure, GNN can suffer from precision issues, especially when dealing with programs rendered into large graphs. In this paper, we present a new graph neural architecture, called Graph Interval Neural Network (GINN), to tackle the weaknesses of the existing GNN. Unlike the standard GNN, GINN generalizes from a curated graph representation obtained through an abstraction method designed to aid models to learn. In particular, GINN focuses exclusively on intervals for mining the feature representation of a program, furthermore, GINN operates on a hierarchy of intervals for scaling the learning to large graphs. We evaluate GINN for two popular downstream applications: variable misuse prediction and method name prediction. Results show in both cases GINN outperforms the state-of-the-art models by a comfortable margin. We have also created a neural bug detector based on GINN to catch null pointer deference bugs in Java code. While learning from the same 9,000 methods extracted from 64 projects, GINN-based bug detector significantly outperforms GNN-based bug detector on 13 unseen test projects. Next, we deploy our trained GINN-based bug detector and Facebook Infer to scan the codebase of 20 highly starred projects on GitHub. Through our manual inspection, we confirm 38 bugs out of 102 warnings raised by GINN-based bug detector compared to 34 bugs out of 129 warnings for Facebook Infer.

cs.SE↗

arXiv2020

Precisely Predicting Acute Kidney Injury with Convolutional Neural Network Based on Electronic Health Record Data

The incidence of Acute Kidney Injury (AKI) commonly happens in the Intensive Care Unit (ICU) patients, especially in the adults, which is an independent risk factor affecting short-term and long-term mortality. Though researchers in recent years highlight the early prediction of AKI, the performance of existing models are not precise enough. The objective of this research is to precisely predict AKI by means of Convolutional Neural Network on Electronic Health Record (EHR) data. The data sets used in this research are two public Electronic Health Record (EHR) databases: MIMIC-III and eICU database. In this study, we take several Convolutional Neural Network models to train and test our AKI predictor, which can precisely predict whether a certain patient will suffer from AKI after admission in ICU according to the last measurements of the 16 blood gas and demographic features. The research is based on Kidney Disease Improving Global Outcomes (KDIGO) criteria for AKI definition. Our work greatly improves the AKI prediction precision, and the best AUROC is up to 0.988 on MIMIC-III data set and 0.936 on eICU data set, both of which outperform the state-of-art predictors. And the dimension of the input vector used in this predictor is much fewer than that used in other existing researches. Compared with the existing AKI predictors, the predictor in this work greatly improves the precision of early prediction of AKI by using the Convolutional Neural Network architecture and a more concise input vector. Early and precise prediction of AKI will bring much benefit to the decision of treatment, so it is believed that our work is a very helpful clinical application.

cs.LG↗

arXiv2020

A Complex KBQA System using Multiple Reasoning Paths

Multi-hop knowledge based question answering (KBQA) is a complex task for natural language understanding. Many KBQA approaches have been proposed in recent years, and most of them are trained based on labeled reasoning path. This hinders the system's performance as many correct reasoning paths are not labeled as ground truth, and thus they cannot be learned. In this paper, we introduce an end-to-end KBQA system which can leverage multiple reasoning paths' information and only requires labeled answer as supervision. We conduct experiments on several benchmark datasets containing both single-hop simple questions as well as muti-hop complex questions, including WebQuestionSP (WQSP), ComplexWebQuestion-1.1 (CWQ), and PathQuestion-Large (PQL), and demonstrate strong performance.

cs.LG↗

arXiv2020

The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding

We present MT-DNN, an open-source natural language understanding (NLU) toolkit that makes it easy for researchers and developers to train customized deep learning models. Built upon PyTorch and Transformers, MT-DNN is designed to facilitate rapid customization for a broad spectrum of NLU tasks, using a variety of objectives (classification, regression, structured prediction) and text encoders (e.g., RNNs, BERT, RoBERTa, UniLM). A unique feature of MT-DNN is its built-in support for robust and transferable learning using the adversarial multi-task learning paradigm. To enable efficient production deployment, MT-DNN supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. We demonstrate the effectiveness of MT-DNN on a wide range of NLU applications across general and biomedical domains. The software and pre-trained models will be publicly available at https://github.com/namisan/mt-dnn.

cs.CL↗

arXiv2020

A Comprehensive Survey of Grammar Error Correction

Grammar error correction (GEC) is an important application aspect of natural language processing techniques. The past decade has witnessed significant progress achieved in GEC for the sake of increasing popularity of machine learning and deep learning, especially in late 2010s when near human-level GEC systems are available. However, there is no prior work focusing on the whole recapitulation of the progress. We present the first survey in GEC for a comprehensive retrospect of the literature in this area. We first give the introduction of five public datasets, data annotation schema, two important shared tasks and four standard evaluation metrics. More importantly, we discuss four kinds of basic approaches, including statistical machine translation based approach, neural machine translation based approach, classification based approach and language model based approach, six commonly applied performance boosting techniques for GEC systems and two data augmentation methods. Since GEC is typically viewed as a sister task of machine translation, many GEC systems are based on neural machine translation (NMT) approaches, where the neural sequence-to-sequence model is applied. Similarly, some performance boosting techniques are adapted from machine translation and are successfully combined with GEC systems for enhancement on the final performance. Furthermore, we conduct an analysis in level of basic approaches, performance boosting techniques and integrated GEC systems based on their experiment results respectively for more clear patterns and conclusions. Finally, we discuss five prospective directions for future GEC researches.

cs.CL↗

arXiv2020

Adversarial Training for Large Neural Language Models

Generalization and robustness are both key desiderata for designing machine learning methods. Adversarial training can enhance robustness, but past work often finds it hurts generalization. In natural language processing (NLP), pre-training large neural language models such as BERT have demonstrated impressive gain in generalization for a variety of tasks, with further improvement from adversarial fine-tuning. However, these models are still vulnerable to adversarial attacks. In this paper, we show that adversarial pre-training can improve both generalization and robustness. We propose a general algorithm ALUM (Adversarial training for large neural LangUage Models), which regularizes the training objective by applying perturbations in the embedding space that maximizes the adversarial loss. We present the first comprehensive study of adversarial training in all stages, including pre-training from scratch, continual pre-training on a well-trained model, and task-specific fine-tuning. ALUM obtains substantial gains over BERT on a wide range of NLP tasks, in both regular and adversarial scenarios. Even for models that have been well trained on extremely large text corpora, such as RoBERTa, ALUM can still produce significant gains from continual pre-training, whereas conventional non-adversarial methods can not. ALUM can be further combined with task-specific fine-tuning to attain additional gains. The ALUM code is publicly available at https://github.com/namisan/mt-dnn.

cs.CL↗

arXiv2020

Hyperproperties for Robotics: Planning via HyperLTL

There is a growing interest on formal methods-based robotic planning for temporal logic objectives. In this work, we extend the scope of existing synthesis methods to hyper-temporal logics. We are motivated by the fact that important planning objectives, such as optimality, robustness, and privacy, (maybe implicitly) involve the interrelation between multiple paths. Such objectives are thus hyperproperties, and cannot be expressed with usual temporal logics like the linear temporal logic (LTL). We show that such hyperproperties can be expressed by HyperLTL, an extension of LTL to multiple paths. To handle the complexity of planning with HyperLTL specifications, we introduce a symbolic approach for synthesizing planning strategies on discrete transition systems. Our planning method is evaluated on several case studies.

cs.RO↗

arXiv2020

Sense and Sensibility: Characterizing Social Media Users Regarding the Use of Controversial Terms for COVID-19

With the world-wide development of 2019 novel coronavirus, although WHO has officially announced the disease as COVID-19, one controversial term - "Chinese Virus" is still being used by a great number of people. In the meantime, global online media coverage about COVID-19-related racial attacks increases steadily, most of which are anti-Chinese or anti-Asian. As this pandemic becomes increasingly severe, more people start to talk about it on social media platforms such as Twitter. When they refer to COVID-19, there are mainly two ways: using controversial terms like "Chinese Virus" or "Wuhan Virus", or using non-controversial terms like "Coronavirus". In this study, we attempt to characterize the Twitter users who use controversial terms and those who use non-controversial terms. We use the Tweepy API to retrieve 17 million related tweets and the information of their authors. We find significant differences between these two groups of Twitter users across their demographics, user-level features like the number of followers, political following status, as well as their geo-locations. Moreover, we apply classification models to predict Twitter users who are more likely to use controversial terms. To our best knowledge, this is the first large-scale social media-based study to characterize users with respect to their usage of controversial terms during a major crisis.

cs.SI↗

arXiv2020

Angular dependence and absorption properties of the anapolemode of Si nano-disks

The polarization- and angle- resolved optical response of the anapole mode in silicon nano-disks array have been experimentally and theoretically investigated. The good agreement betweenmeasured data and simulations yields to a consistent description of the anapole mode behaviorthat exhibits different features for TE or TM polarization excitation. Scattering matrix calculationallows us to disentangle scattered and diffused light contributions and to provide a quantitativeestimation of the absorbance enhancement associated to 2D excitation of the anapole mode. Weperformed the multipolar decomposition of the far-field scattered radiation for both TE and TMpolarizations and unambiguously identified the anapole resonant condition in excellent agreementwith the experimental results over a large range of incident angles. Our findings demonstrate thecontrolled excitation of electromagnetic anapole modes in engineered arrays of silicon nano-disksfor the development of optical nanostructures with enhanced light-matter interaction

cond-mat.mes-hall↗

arXiv2020

Statistically Model Checking PCTL Specifications on Markov Decision Processes via Reinforcement Learning

Probabilistic Computation Tree Logic (PCTL) is frequently used to formally specify control objectives such as probabilistic reachability and safety. In this work, we focus on model checking PCTL specifications statistically on Markov Decision Processes (MDPs) by sampling, e.g., checking whether there exists a feasible policy such that the probability of reaching certain goal states is greater than a threshold. We use reinforcement learning to search for such a feasible policy for PCTL specifications, and then develop a statistical model checking (SMC) method with provable guarantees on its error. Specifically, we first use upper-confidence-bound (UCB) based Q-learning to design an SMC algorithm for bounded-time PCTL specifications, and then extend this algorithm to unbounded-time specifications by identifying a proper truncation time by checking the PCTL specification and its negation at the same time. Finally, we evaluate the proposed method on case studies.

cs.LG↗

arXiv2020

The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic

Recently, the pandemic of the novel Coronavirus Disease-2019 (COVID-19) has presented governments with ultimate challenges. In the United States, the country with the highest confirmed COVID-19 infection cases, a nationwide social distancing protocol has been implemented by the President. For the first time in a hundred years since the 1918 flu pandemic, the US population is mandated to stay in their households and avoid public contact. As a result, the majority of public venues and services have ceased their operations. Following the closure of the University of Washington on March 7th, more than a thousand colleges and universities in the United States have cancelled in-person classes and campus activities, impacting millions of students. This paper aims to discover the social implications of this unprecedented disruption in our interactive society regarding both the general public and higher education populations by mining people's opinions on social media. We discover several topics embedded in a large number of COVID-19 tweets that represent the most central issues related to the pandemic, which are of great concerns for both college students and the general public. Moreover, we find significant differences between these two groups of Twitter users with respect to the sentiments they expressed towards the COVID-19 issues. To our best knowledge, this is the first social media-based study which focuses on the college student community's demographics and responses to prevalent social issues during a major crisis.

cs.SI↗

arXiv2020

Design of a novel monolithic parabolic-mirror ion-trap to precisely align the RF null point with the optical focus

We propose a novel ion trap design with the high collection efficiency parabolic-mirror integrated with the ion trap electrodes. This design has three radio frequency (RF) electrodes and eight direct current(DC) compensation electrodes. By carefully adjusting three RF voltages, the parabolic mirror focus can be made precisely coincident with the RF null point. Thus, the aberration and the ion micromotion can be minimized at the same time. This monolithic design can significantly improve the ion-ion entanglement generation speed by extending the photon collecting solid angle beyond $90\%\cdot4π$. Further analysis of the trapping setup shows that the RF voltage variation method relexes machining accuracy to a broad range. This design is expected to be a robust scheme for trapping ion to speed entanglement network node.

quant-ph↗

arXiv2020

Exchange coupling in a linear chain of three quantum-dot spin qubits in silicon

Quantum gates between spin qubits can be implemented leveraging the natural Heisenberg exchange interaction between two electrons in contact with each other. This interaction is controllable by electrically tailoring the overlap between electronic wavefunctions in quantum dot systems, as long as they occupy neighbouring dots. An alternative route is the exploration of superexchange - the coupling between remote spins mediated by a third idle electron that bridges the distance between quantum dots. We experimentally demonstrate direct exchange coupling and provide evidence for second neighbour mediated superexchange in a linear array of three single-electron spin qubits in silicon, inferred from the electron spin resonance frequency spectra. We confirm theoretically through atomistic modeling that the device geometry only allows for sizeable direct exchange coupling for neighbouring dots, while next nearest neighbour coupling cannot stem from the vanishingly small tail of the electronic wavefunction of the remote dots, and is only possible if mediated.

quant-ph↗

arXiv2020

Learning a Static Bug Finder from Data

We present an alternative approach to creating static bug finders. Instead of relying on human expertise, we utilize deep neural networks to train static analyzers directly from data. In particular, we frame the problem of bug finding as a classification task and train a classifier to differentiate the buggy from non-buggy programs using Graph Neural Network (GNN). Crucially, we propose a novel interval-based propagation mechanism that leads to a significantly more efficient, accurate and scalable generalization of GNN. We have realized our approach into a framework, NeurSA, and extensively evaluated it. In a cross-project prediction task, three neural bug detectors we instantiate from NeurSA are effective in catching null pointer dereference, array index out of bound and class cast bugs in unseen code. We compare NeurSA against several static analyzers (e.g. Facebook Infer and Pinpoint) on a set of null pointer dereference bugs. Results show that NeurSA is more precise in catching the real bugs and suppressing the spurious warnings. We also apply NeurSA to several popular Java projects on GitHub and discover 50 new bugs, among which 9 have been fixed, and 3 have been confirmed.

cs.SE↗

arXiv2020

Enabling Efficient and Flexible FPGA Virtualization for Deep Learning in the Cloud

FPGAs have shown great potential in providing low-latency and energy-efficient solutions for deep neural network (DNN) inference applications. Currently, the majority of FPGA-based DNN accelerators in the cloud run in a time-division multiplexing way for multiple users sharing a single FPGA, and require re-compilation with $\sim$100 s overhead. Such designs lead to poor isolation and heavy performance loss for multiple users, which are far away from providing efficient and flexible FPGA virtualization for neither public nor private cloud scenarios. To solve these problems, we introduce a novel virtualization framework for instruction architecture set (ISA) based on DNN accelerators by sharing a single FPGA. We enable the isolation by introducing a two-level instruction dispatch module and a multi-core based hardware resources pool. Such designs provide isolated and runtime-programmable hardware resources, further leading to performance isolation for multiple users. On the other hand, to overcome the heavy re-compilation overheads, we propose a tiling-based instruction frame package design and two-stage static-dynamic compilation. Only the light-weight runtime information is re-compiled with $\sim$1 ms overhead, thus the performance is guaranteed for the private cloud. Our extensive experimental results show that the proposed virtualization design achieves 1.07-1.69x and 1.88-3.12x throughput improvement over previous static designs using the single-core and the multi-core architectures, respectively.

cs.DC↗

arXiv2020

Stochasticity in radiative polarization of ultrarelativistic electrons in an ultrastrong laser pulse

Stochasticity effects in the spin (de)polarization of an ultrarelativistic electron beam during photon emissions in a counterpropoagating ultrastrong focused laser pulse in the quantum radiation reaction regime are investigated. We employ a Monte Carlo method to describe the electron dynamics semiclassically, and photon emissions as well as the electron radiative polarization quantum mechanically. While in the latter the photon emission is inherently stochastic, we were able to identify its imprints in comparison with the new developed semiclassical stochasticity-free method of radiative polarization applicable in the quantum regime. With an initially spin-polarized electron beam, the stochastic spin effects are seen in the dependence of the depolarization degree on the electron scattering angle and the electron final energy (spin stochastic diffusion). With an initially unpolarized electron beam, the spin stochasticity is exhibited in enhancing the known effect of splitting of the electron beam along the propagation direction into two oppositely polarized parts by an elliptically polarized laser pulse. The considered stochasticity effects for the spin are observable with currently achievable laser and electron beam parameters.

physics.plasm-ph↗

arXiv2020

On fibrations of Lie groupoids

As groupoids generalize groups, motivated by group extensions we consider a kind of fibrations of Lie groupoids, called locally topological product Lie groupoid fibrations with fiber $\sf A$, i.e., \[ 1\rightarrow {\sf A} \rightarrow {\sf G} \rightarrow {\sf K}\rightarrow 1 \] where $\sf A,\sf G$ and $\sf K$ are Lie groupoids. Similar to the theory of group extensions, we show that the existence of locally topological product Lie groupoid fibrations with fiber $\sf A$ over $\sf K$ is obstructed by a groupoid cohomology of $H^3_{\bar Λ}({\sf K},Z_{\sf A})$, and these locally topological product Lie groupoid fibrations are classified by $H^2_{\bar Λ}({\sf K},Z_{\sf A})$ once exists. Here $Z_{\sf A}$ is the center of $\sf A$. This generalizes the theory of group extensions, of gerbes over manifolds/groupoids and etc.

math.DG↗