arXiv subjects

Qiang Xu

Publications and source records attributed to Qiang Xu.

At least 181 records · Page 10Linked to original sources

arXiv2021

DeepFIB: Self-Imputation for Time Series Anomaly Detection

Time series (TS) anomaly detection (AD) plays an essential role in various applications, e.g., fraud detection in finance and healthcare monitoring. Due to the inherently unpredictable and highly varied nature of anomalies and the lack of anomaly labels in historical data, the AD problem is typically formulated as an unsupervised learning problem. The performance of existing solutions is often not satisfactory, especially in data-scarce scenarios. To tackle this problem, we propose a novel self-supervised learning technique for AD in time series, namely \emph{DeepFIB}. We model the problem as a \emph{Fill In the Blank} game by masking some elements in the TS and imputing them with the rest. Considering the two common anomaly shapes (point- or sequence-outliers) in TS data, we implement two masking strategies with many self-generated training samples. The corresponding self-imputation networks can extract more robust temporal relations than existing AD solutions and effectively facilitate identifying the two types of anomalies. For continuous outliers, we also propose an anomaly localization algorithm that dramatically reduces AD errors. Experiments on various real-world TS datasets demonstrate that DeepFIB outperforms state-of-the-art methods by a large margin, achieving up to $65.2\%$ relative improvement in F1-score.

cs.LG↗

arXiv2021

Testability-Aware Low Power Controller Design with Evolutionary Learning

XORNet-based low power controller is a popular technique to reduce circuit transitions in scan-based testing. However, existing solutions construct the XORNet evenly for scan chain control, and it may result in sub-optimal solutions without any design guidance. In this paper, we propose a novel testability-aware low power controller with evolutionary learning. The XORNet generated from the proposed genetic algorithm (GA) enables adaptive control for scan chains according to their usages, thereby significantly improving XORNet encoding capacity, reducing the number of failure cases with ATPG and decreasing test data volume. Experimental results indicate that under the same control bits, our GA-guided XORNet design can improve the fault coverage by up to 2.11%. The proposed GA-guided XORNets also allows reducing the number of control bits, and the total testing time decreases by 20.78% on average and up to 47.09% compared to the existing design without sacrificing test coverage.

cs.LG↗

arXiv2021

AppealNet: An Efficient and Highly-Accurate Edge/Cloud Collaborative Architecture for DNN Inference

This paper presents AppealNet, a novel edge/cloud collaborative architecture that runs deep learning (DL) tasks more efficiently than state-of-the-art solutions. For a given input, AppealNet accurately predicts on-the-fly whether it can be successfully processed by the DL model deployed on the resource-constrained edge device, and if not, appeals to the more powerful DL model deployed at the cloud. This is achieved by employing a two-head neural network architecture that explicitly takes inference difficulty into consideration and optimizes the tradeoff between accuracy and computation/communication cost of the edge/cloud collaborative architecture. Experimental results on several image classification datasets show up to more than 40% energy savings compared to existing techniques without sacrificing accuracy.

cs.LG↗

arXiv2021

Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation

Various deep learning techniques have been proposed to solve the single-view 2D-to-3D pose estimation problem. While the average prediction accuracy has been improved significantly over the years, the performance on hard poses with depth ambiguity, self-occlusion, and complex or rare poses is still far from satisfactory. In this work, we target these hard poses and present a novel skeletal GNN learning solution. To be specific, we propose a hop-aware hierarchical channel-squeezing fusion layer to effectively extract relevant information from neighboring nodes while suppressing undesired noises in GNN learning. In addition, we propose a temporal-aware dynamic graph construction procedure that is robust and effective for 3D pose estimation. Experimental results on the Human3.6M dataset show that our solution achieves 10.3\% average prediction accuracy improvement and greatly improves on hard poses over state-of-the-art techniques. We further apply the proposed technique on the skeleton-based action recognition task and also achieve state-of-the-art performance. Our code is available at https://github.com/ailingzengzzz/Skeletal-GNN.

cs.CV↗

arXiv2021

TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks

Deep learning (DL) has achieved unprecedented success in a variety of tasks. However, DL systems are notoriously difficult to test and debug due to the lack of explainability of DL models and the huge test input space to cover. Generally speaking, it is relatively easy to collect a massive amount of test data, but the labeling cost can be quite high. Consequently, it is essential to conduct test selection and label only those selected "high quality" bug-revealing test inputs for test cost reduction. In this paper, we propose a novel test prioritization technique that brings order into the unlabeled test instances according to their bug-revealing capabilities, namely TestRank. Different from existing solutions, TestRank leverages both intrinsic attributes and contextual attributes of test instances when prioritizing them. To be specific, we first build a similarity graph on test instances and training samples, and we conduct graph-based semi-supervised learning to extract contextual features. Then, for a particular test instance, the contextual features extracted from the graph neural network (GNN) and the intrinsic features obtained with the DL model itself are combined to predict its bug-revealing probability. Finally, TestRank prioritizes unlabeled test instances in descending order of the above probability value. We evaluate the performance of TestRank on a variety of image classification datasets. Experimental results show that the debugging efficiency of our method significantly outperforms existing test prioritization techniques.

cs.LG↗

arXiv2021

Skimming and Scanning for Untrimmed Video Action Recognition

Video action recognition (VAR) is a primary task of video understanding, and untrimmed videos are more common in real-life scenes. Untrimmed videos have redundant and diverse clips containing contextual information, so sampling dense clips is essential. Recently, some works attempt to train a generic model to select the N most representative clips. However, it is difficult to model the complex relations from intra-class clips and inter-class videos within a single model and fixed selected number, and the entanglement of multiple relations is also hard to explain. Thus, instead of "only look once", we argue "divide and conquer" strategy will be more suitable in untrimmed VAR. Inspired by the speed reading mechanism, we propose a simple yet effective clip-level solution based on skim-scan techniques. Specifically, the proposed Skim-Scan framework first skims the entire video and drops those uninformative and misleading clips. For the remaining clips, it scans clips with diverse features gradually to drop redundant clips but cover essential content. The above strategies can adaptively select the necessary clips according to the difficulty of the different videos. To trade off the computational complexity and performance, we observe the similar statistical expression between lightweight and heavy networks, thus it supports us to explore the combination of them. Comprehensive experiments are performed on ActivityNet and mini-FCVID datasets, and results demonstrate that our solution surpasses the state-of-the-art performance in terms of both accuracy and efficiency.

cs.CV↗

arXiv2020

T-WaveNet: Tree-Structured Wavelet Neural Network for Sensor-Based Time Series Analysis

Sensor-based time series analysis is an essential task for applications such as activity recognition and brain-computer interface. Recently, features extracted with deep neural networks (DNNs) are shown to be more effective than conventional hand-crafted ones. However, most of these solutions rely solely on the network to extract application-specific information carried in the sensor data. Motivated by the fact that usually a small subset of the frequency components carries the primary information for sensor data, we propose a novel tree-structured wavelet neural network for sensor data analysis, namely \emph{T-WaveNet}. To be specific, with T-WaveNet, we first conduct a power spectrum analysis for the sensor data and decompose the input signal into various frequency subbands accordingly. Then, we construct a tree-structured network, and each node on the tree (corresponding to a frequency subband) is built with an invertible neural network (INN) based wavelet transform. By doing so, T-WaveNet provides more effective representation for sensor information than existing DNN-based techniques, and it achieves state-of-the-art performance on various sensor datasets, including UCI-HAR for activity recognition, OPPORTUNITY for gesture recognition, BCICIV2a for intention recognition, and NinaPro DB1 for muscular movement recognition.

eess.SP↗

arXiv2020

DeepDyve: Dynamic Verification for Deep Neural Networks

Deep neural networks (DNNs) have become one of the enabling technologies in many safety-critical applications, e.g., autonomous driving and medical image analysis. DNN systems, however, suffer from various kinds of threats, such as adversarial example attacks and fault injection attacks. While there are many defense methods proposed against maliciously crafted inputs, solutions against faults presented in the DNN system itself (e.g., parameters and calculations) are far less explored. In this paper, we develop a novel lightweight fault-tolerant solution for DNN-based systems, namely DeepDyve, which employs pre-trained neural networks that are far simpler and smaller than the original DNN for dynamic verification. The key to enabling such lightweight checking is that the smaller neural network only needs to produce approximate results for the initial task without sacrificing fault coverage much. We develop efficient and effective architecture and task exploration techniques to achieve optimized risk/overhead trade-off in DeepDyve. Experimental results show that DeepDyve can reduce 90% of the risks at around 10% overhead.

cs.LG↗

arXiv2020

Region-Wise Attack: On Efficient Generation of Robust Physical Adversarial Examples

Deep neural networks (DNNs) are shown to be susceptible to adversarial example attacks. Most existing works achieve this malicious objective by crafting subtle pixel-wise perturbations, and they are difficult to launch in the physical world due to inevitable transformations (e.g., different photographic distances and angles). Recently, there are a few research works on generating physical adversarial examples, but they generally require the details of the model a priori, which is often impractical. In this work, we propose a novel physical adversarial attack for arbitrary black-box DNN models, namely Region-Wise Attack. To be specific, we present how to efficiently search for regionwise perturbations to the inputs and determine their shapes, locations and colors via both top-down and bottom-up techniques. In addition, we introduce two fine-tuning techniques to further improve the robustness of our attack. Experimental results demonstrate the efficacy and robustness of the proposed Region-Wise Attack in real world.

cs.LG↗

arXiv2020

Quantitative estimates for homogenization of nonlinear elliptic operators in perforated domains

This paper was devoted to study the quantitative homogenization problems for nonlinear elliptic operators in perforated domains. We obtained a sharp error estimate $O(\varepsilon)$ when the problem was anchored in the reference domain $\varepsilonω$. If concerning a bounded perforated domain, one will see a bad influence from the boundary layers, which leads to the loss of the convergence rate by $O(\varepsilon^{1/2})$. Equipped with the error estimates, we developed both interior and boundary Lipschitz estimates at large-scales. As an application, we received the so-called quenched Calderón-Zygumund estimates by Shen's real arguments. To overcome some difficulties, we improved the extension theory from (\cite[Theorem 4.3]{OSY}) to $L^p$-versions with $\frac{2d}{d+1}-ε<p<\frac{2d}{d-1}+ε$ and $0<ε\ll1$. Appealing to this, we established Poincaré-Sobolev inequalities of local type on perforated domains. Some of results in the present literature are new even for related linear elliptic models.

math.AP↗

arXiv2020

SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approach

Human poses that are rare or unseen in a training set are challenging for a network to predict. Similar to the long-tailed distribution problem in visual recognition, the small number of examples for such poses limits the ability of networks to model them. Interestingly, local pose distributions suffer less from the long-tail problem, i.e., local joint configurations within a rare pose may appear within other poses in the training set, making them less rare. We propose to take advantage of this fact for better generalization to rare and unseen poses. To be specific, our method splits the body into local regions and processes them in separate network branches, utilizing the property that a joint position depends mainly on the joints within its local body region. Global coherence is maintained by recombining the global context from the rest of the body into each branch as a low-dimensional vector. With the reduced dimensionality of less relevant body areas, the training set distribution within network branches more closely reflects the statistics of local poses instead of global body poses, without sacrificing information important for joint inference. The proposed split-and-recombine approach, called SRNet, can be easily adapted to both single-image and temporal models, and it leads to appreciable improvements in the prediction of rare and unseen poses.

cs.CV↗

arXiv2020

High-Voltage Honeycomb Layered Oxide Positive Electrodes for Rechargeable Sodium Batteries

Natural abundance, impressive chemical characteristics and economic feasibility have rekindled the appeal for rechargeable sodium (Na) batteries as a practical solution for the growing energy demand, environmental sustainability and energy independence. However, the scarcity of viable positive electrode materials remains a huge impediment to the actualization of this technology. In this paper, we explore honeycomb layered oxides adopting the composition Na$_2$Ni$_{2-x}$Co$_x$TeO$_6$ ($x = 0, 0.25$ and $0.50$) as feasible positive electrode (cathode) materials for rechargeable sodium batteries at both room- and elevated temperatures using ionic liquids. Through standard galvanostatic assessments and analyses we demonstrate that substitution of nickel with cobalt in Na$_2$Ni$_2$TeO$_6$ leads to an increase in the discharge voltage to nearly $4$ V (versus Na$^+$ / Na) for the Na$_2$Ni$_{2-x}$Co$_x$TeO$_6$ family of honeycomb layered oxide materials, which surpasses the attained average voltages for most layered oxide positive electrode materials that facilitate Na-ion desertion. We also verify the increased kinetics within the Na$_2$Ni$_{2-x}$Co$_x$TeO$_6$ honeycomb layered oxides during operations at elevated temperatures which lead to an increase in reversible capacity of the rechargeable Na battery. This study underpins the doping of congener transition metal atoms to the honeycomb structure of Na$_2$Ni$_2$TeO$_6$ in addition to elevated-temperature operation as a judicious route to enhance the electrochemical performance of analogous layered oxides.

cond-mat.mtrl-sci↗

arXiv2020

suboptimal error estimates for homogenization of linear elasticity systems on perforated domains

In the present work, we established almost-sharp error estimates for linear elasticity systems in periodically perforated domains. The first result was $L^{\frac{2d}{d-1-τ}}$-error estimates $O\big(\varepsilon^{1-\fracτ{2}}\big)$ with $0<τ<1$ for a bounded smooth domain. It followed from weighted Hardy-Sobolev's inequalities and a suboptimal error estimate for the square function of the first-order approximating corrector (which was earliest investigated by C. Kenig, F. Lin, Z. Shen \cite{KLS} under additional regularity assumption on coefficients). The new approach relied on the weighted quenched Calderón-Zygmund estimate (initially appeared in A. Gloria, S. Neukamm, F. Otto's work \cite{Gloria_Neukamm_Otto_2015} for a quantitative stochastic homogenization theory). The second effort was $L^2$-error estimates $O\big(\varepsilon^{\frac{5}{6}}\ln^{\frac{2}{3}}(1/\varepsilon)\big)$ for a Lipschitz domain, followed from a new duality scheme coupled with interpolation inequalities. Also, we developed a new weighted extension theorem for perforated domains, and a real method imposed by Z. Shen \cite{S3} played a fundamental role in the whole project.

math.AP↗

arXiv2020

Nonlocal Kinetic Energy Density Functionals for Isolated Systems via Local Density Approximation Kernels

Despite a large number of nonlocal kinetic energy density functionals (KEDFs) available for large-scale calculations, most of those nonlocal KEDFs designed for the extended systems cannot be directly applied to isolated systems. In this manuscript, we proposed a generalized scheme to construct nonlocal KEDFs via the local density approximation kernels and construct a family of KEDFs for simulations of isolated systems within orbital-free density functional theory. The performance of KEDFs has been demonstrated by several clusters encompassing Mg, Si and GaAs. The results show that our constructed KEDFs can achieve high numerical accuracy and stability for random clusters, therefore, making orbital-free density functional theory accessible for practical simulations of isolated systems.

cond-mat.mtrl-sci↗

arXiv2020

nuScenes: A multimodal dataset for autonomous driving

Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We define novel 3D detection and tracking metrics. We also provide careful dataset analysis as well as baselines for lidar and image based detection and tracking. Data, development kit and more information are available online.

cs.LG↗

arXiv2020

The Effect of the Multi-Layer Text Summarization Model on the Efficiency and Relevancy of the Vector Space-based Information Retrieval

The massive upload of text on the internet creates a huge inverted index in information retrieval systems, which hurts their efficiency. The purpose of this research is to measure the effect of the Multi-Layer Similarity model of the automatic text summarization on building an informative and condensed invert index in the IR systems. To achieve this purpose, we summarized a considerable number of documents using the Multi-Layer Similarity model, and we built the inverted index from the automatic summaries that were generated from this model. A series of experiments were held to test the performance in terms of efficiency and relevancy. The experiments include comparisons with three existing text summarization models; the Jaccard Coefficient Model, the Vector Space Model, and the Latent Semantic Analysis model. The experiments examined three groups of queries with manual and automatic relevancy assessment. The positive effect of the Multi-Layer Similarity in the efficiency of the IR system was clear without noticeable loss in the relevancy results. However, the evaluation showed that the traditional statistical models without semantic investigation failed to improve the information retrieval efficiency. Comparing with the previous publications that addressed the use of summaries as a source of the index, the relevancy assessment of our work was higher, and the Multi-Layer Similarity retrieval constructed an inverted index that was 58% smaller than the main corpus inverted index.

cs.IR↗

arXiv2019

ICSTrace: A Malicious IP Traceback Model for Attacking Data of Industrial Control System

Considering the attacks against industrial control system are mostly organized and premeditated actions, IP traceback is significant for the security of industrial control system. Based on the infrastructure of the Internet, we have developed a novel malicious IP traceback model-ICSTrace, without deploying any new services. The model extracts the function codes and their parameters from the attack data according to the format of industrial control protocol, and employs a short sequence probability method to transform the function codes and their parameter into a vector, which characterizes the attack pattern of malicious IP addresses. Furthermore, a Partial Seeded K-Means algorithm is proposed for the pattern's clustering, which helps in tracing the attacks back to an organization. ICSTrace is evaluated basing on the attack data captured by the large-scale deployed honeypots for industrial control system, and the results demonstrate that ICSTrace is effective on malicious IP traceback in industrial control system.

cs.CR↗

arXiv2019

DeepFuse: An IMU-Aware Network for Real-Time 3D Human Pose Estimation from Multi-View Image

In this paper, we propose a two-stage fully 3D network, namely \textbf{DeepFuse}, to estimate human pose in 3D space by fusing body-worn Inertial Measurement Unit (IMU) data and multi-view images deeply. The first stage is designed for pure vision estimation. To preserve data primitiveness of multi-view inputs, the vision stage uses multi-channel volume as data representation and 3D soft-argmax as activation layer. The second one is the IMU refinement stage which introduces an IMU-bone layer to fuse the IMU and vision data earlier at data level. without requiring a given skeleton model a priori, we can achieve a mean joint error of $28.9$mm on TotalCapture dataset and $13.4$mm on Human3.6M dataset under protocol 1, improving the SOTA result by a large margin. Finally, we discuss the effectiveness of a fully 3D network for 3D pose estimation experimentally which may benefit future research.

cs.CV↗