arXiv subjects

Di Wu

Publications and source records attributed to Di Wu.

At least 613 records · Page 34Linked to original sources

arXiv2021

Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understanding

Spoken language understanding (SLU) system usually consists of various pipeline components, where each component heavily relies on the results of its upstream ones. For example, Intent detection (ID), and slot filling (SF) require its upstream automatic speech recognition (ASR) to transform the voice into text. In this case, the upstream perturbations, e.g. ASR errors, environmental noise and careless user speaking, will propagate to the ID and SF models, thus deteriorating the system performance. Therefore, the well-performing SF and ID models are expected to be noise resistant to some extent. However, existing models are trained on clean data, which causes a \textit{gap between clean data training and real-world inference.} To bridge the gap, we propose a method from the perspective of domain adaptation, by which both high- and low-quality samples are embedding into similar vector space. Meanwhile, we design a denoising generation model to reduce the impact of the low-quality samples. Experiments on the widely-used dataset, i.e. Snips, and large scale in-house dataset (10 million training examples) demonstrate that this method not only outperforms the baseline models on real-world (noisy) corpus but also enhances the robustness, that is, it produces high-quality results under a noisy environment. The source code will be released.

cs.CL↗

arXiv2021

Representation range needs for 16-bit neural network training

Deep learning has grown rapidly thanks to its state-of-the-art performance across a wide range of real-world applications. While neural networks have been trained using IEEE-754 binary32 arithmetic, the rapid growth of computational demands in deep learning has boosted interest in faster, low precision training. Mixed-precision training that combines IEEE-754 binary16 with IEEE-754 binary32 has been tried, and other $16$-bit formats, for example Google's bfloat16, have become popular. In floating-point arithmetic there is a tradeoff between precision and representation range as the number of exponent bits changes; denormal numbers extend the representation range. This raises questions of how much exponent range is needed, of whether there is a format between binary16 (5 exponent bits) and bfloat16 (8 exponent bits) that works better than either of them, and whether or not denormals are necessary. In the current paper we study the need for denormal numbers for mixed-precision training, and we propose a 1/6/9 format, i.e., 6-bit exponent and 9-bit explicit mantissa, that offers a better range-precision tradeoff. We show that 1/6/9 mixed-precision training is able to speed up training on hardware that incurs a performance slowdown on denormal operations or eliminates the need for denormal numbers altogether. And, for a number of fully connected and convolutional neural networks in computer vision and natural language processing, 1/6/9 achieves numerical parity to standard mixed-precision.

cs.AI↗

arXiv2021

Software Defined Edge Computing for Distributed Management and Scalable Control in IoT Multinetworks

Recently, edge computing has emerged as a promising paradigm to support mobile access in IoT multinetworks. However, coexistence of heterogeneous wireless communication schemes brings about new challenges to the mobility management and access control in such networks. To maintain its computing continuum, Software-Defined Networking (SDN) architecture can be applied to the distributed edge networks, which provides consistent frameworks for mobility management and creates secure mechanisms for access control. In this paper, we first review the state-of-the-art SDN technology and its applications in edge computing. In particular, we highlight the prominent issues related to mobility management and access control for SDN in the edge computing environment. We also introduce a number of effective solutions to these issues by presenting real-world testbed experiments, prototypes and numerical analysis of adopting SDN based edge computing paradigms for IoT multinetworks.

cs.NI↗

arXiv2021

End-to-End Learning for Simultaneously Generating Decision Map and Multi-Focus Image Fusion Result

The general aim of multi-focus image fusion is to gather focused regions of different images to generate a unique all-in-focus fused image. Deep learning based methods become the mainstream of image fusion by virtue of its powerful feature representation ability. However, most of the existing deep learning structures failed to balance fusion quality and end-to-end implementation convenience. End-to-end decoder design often leads to unrealistic result because of its non-linear mapping mechanism. On the other hand, generating an intermediate decision map achieves better quality for the fused image, but relies on the rectification with empirical post-processing parameter choices. In this work, to handle the requirements of both output image quality and comprehensive simplicity of structure implementation, we propose a cascade network to simultaneously generate decision map and fused result with an end-to-end training procedure. It avoids the dependence on empirical post-processing methods in the inference stage. To improve the fusion quality, we introduce a gradient aware loss function to preserve gradient information in output fused image. In addition, we design a decision calibration strategy to decrease the time consumption in the application of multiple images fusion. Extensive experiments are conducted to compare with 19 different state-of-the-art multi-focus image fusion structures with 6 assessment metrics. The results prove that our designed structure can generally ameliorate the output fused image quality, while implementation efficiency increases over 30\% for multiple images fusion.

cs.CV↗

arXiv2021

Virtual Reality: A Survey of Enabling Technologies and its Applications in IoT

Virtual Reality (VR) has shown great potential to revolutionize the market by providing users immersive experiences with freedom of movement. Compared to traditional video streaming, VR is with ultra high-definition and dynamically changes with users' head and eye movements, which poses significant challenges for the realization of such potential. In this paper, we provide a detailed and systematic survey of enabling technologies of virtual reality and its applications in Internet of Things (IoT). We identify major challenges of virtual reality on system design, view prediction, computation, streaming, and quality of experience evaluation. We discuss each of them by extensively surveying and reviewing related papers in the recent years. We also introduce several use cases of VR for IoT. Last, issues and future research directions are also identified and discussed.

cs.NI↗

arXiv2021

Measuring Discrimination to Boost Comparative Testing for Multiple Deep Learning Models

The boom of DL technology leads to massive DL models built and shared, which facilitates the acquisition and reuse of DL models. For a given task, we encounter multiple DL models available with the same functionality, which are considered as candidates to achieve this task. Testers are expected to compare multiple DL models and select the more suitable ones w.r.t. the whole testing context. Due to the limitation of labeling effort, testers aim to select an efficient subset of samples to make an as precise rank estimation as possible for these models. To tackle this problem, we propose Sample Discrimination based Selection (SDS) to select efficient samples that could discriminate multiple models, i.e., the prediction behaviors (right/wrong) of these samples would be helpful to indicate the trend of model performance. To evaluate SDS, we conduct an extensive empirical study with three widely-used image datasets and 80 real world DL models. The experimental results show that, compared with state-of-the-art baseline methods, SDS is an effective and efficient sample selection method to rank multiple DL models.

cs.SE↗

arXiv2021

Environment-Aware and Training-Free Beam Alignment for mmWave Massive MIMO via Channel Knowledge Map

Millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) communication system is expected to achieve enormous transmission rate, provided that the transmit and receive beams are properly aligned with the MIMO channel. However, existing beam alignment techniques rely on either channel estimation or beam sweeping, which incur prohibitively high training overhead, especially for future wireless systems with further increased antenna dimensions and more stringent requirement on cost-effective hardware architectures. In this paper, we propose a new beam alignment technique, which is environment-aware and training-free, by utilizing the emerging concept of channel knowledge map (CKM), together with the user location information that is readily available in contemporary wireless systems. CKM is a site-specific database, tagged with the transmitter/receiver locations, which contains useful channel information to facilitate or even obviate real-time channel state information (CSI) acquistion. Two instances of CKM are proposed for beam alignment in mmWave massive MIMO systems, namely channel path map (CPM) and beam index map (BIM). It is shown that compared with existing training-based beam alignment schemes, the proposed CKM-enabled environment-aware beam alignment is able to drastically improve the effective communication rate, even with moderate user location errors, thanks to its significant saving of the prohibitive training overhead.

eess.SP↗

arXiv2021

A Thermodynamic Core using Voltage-Controlled Spin-Orbit-Torque Magnetic Tunnel Junctions

We present a magnetic implementation of a thermodynamic computing fabric. Magnetic devices within computing cores harness thermodynamics through its voltage-controlled thermal stability; while the evolution of network states is guided by the spin-orbit-torque effect. We theoretically derive the dynamics of the cores and show that the computing fabric can successfully compute ground states of a Boltzmann Machine. Subsequently, we demonstrate the physical realization of these devices based on a CoFeB-MgO magnetic tunnel junction structure. The results of this work pave the path towards the realization of highly efficient, high-performance thermodynamic computing hardware. Finally, this paper will also give a perspective of computing beyond thermodynamic computing.

cs.ET↗

arXiv2021

Optimizing Video Caching at the Edge: A Hybrid Multi-Point Process Approach

It is always a challenging problem to deliver a huge volume of videos over the Internet. To meet the high bandwidth and stringent playback demand, one feasible solution is to cache video contents on edge servers based on predicted video popularity. Traditional caching algorithms (e.g., LRU, LFU) are too simple to capture the dynamics of video popularity, especially long-tailed videos. Recent learning-driven caching algorithms (e.g., DeepCache) show promising performance, however, such black-box approaches are lack of explainability and interpretability. Moreover, the parameter tuning requires a large number of historical records, which are difficult to obtain for videos with low popularity. In this paper, we optimize video caching at the edge using a white-box approach, which is highly efficient and also completely explainable. To accurately capture the evolution of video popularity, we develop a mathematical model called \emph{HRS} model, which is the combination of multiple point processes, including Hawkes' self-exciting, reactive and self-correcting processes. The key advantage of the HRS model is its explainability, and much less number of model parameters. In addition, all its model parameters can be learned automatically through maximizing the Log-likelihood function constructed by past video request events. Next, we further design an online HRS-based video caching algorithm. To verify its effectiveness, we conduct a series of experiments using real video traces collected from Tencent Video, one of the largest online video providers in China. Experiment results demonstrate that our proposed algorithm outperforms the state-of-the-art algorithms, with 12.3\% improvement on average in terms of cache hit rate under realistic settings.

cs.MM↗

arXiv2021

Gain without Pain: Offsetting DP-injected Nosies Stealthily in Cross-device Federated Learning

Federated Learning (FL) is an emerging paradigm through which decentralized devices can collaboratively train a common model. However, a serious concern is the leakage of privacy from exchanged gradient information between clients and the parameter server (PS) in FL. To protect gradient information, clients can adopt differential privacy (DP) to add additional noises and distort original gradients before they are uploaded to the PS. Nevertheless, the model accuracy will be significantly impaired by DP noises, making DP impracticable in real systems. In this work, we propose a novel Noise Information Secretly Sharing (NISS) algorithm to alleviate the disturbance of DP noises by sharing negated noises among clients. We theoretically prove that: 1) If clients are trustworthy, DP noises can be perfectly offset on the PS; 2) Clients can easily distort negated DP noises to protect themselves in case that other clients are not totally trustworthy, though the cost lowers model accuracy. NISS is particularly applicable for FL across multiple IoT (Internet of Things) systems, in which all IoT devices need to collaboratively train a model. To verify the effectiveness and the superiority of the NISS algorithm, we conduct experiments with the MNIST and CIFAR-10 datasets. The experiment results verify our analysis and demonstrate that NISS can improve model accuracy by 21% on average and obtain better privacy protection if clients are trustworthy.

cs.DC↗

arXiv2021

The design of the Ali CMB Polarization Telescope receiver

Ali CMB Polarization Telescope (AliCPT-1) is the first CMB degree-scale polarimeter to be deployed on the Tibetan plateau at 5,250m above sea level. AliCPT-1 is a 90/150 GHz 72 cm aperture, two-lens refracting telescope cooled down to 4 K. Alumina lenses, 800mm in diameter, image the CMB in a 33.4° field of view on a 636mm wide focal plane. The modularized focal plane consists of dichroic polarization-sensitive Transition-Edge Sensors (TESes). Each module includes 1,704 optically active TESes fabricated on a 150mm diameter silicon wafer. Each TES array is read out with a microwave multiplexing readout system capable of a multiplexing factor up to 2,048. Such a large multiplexing factor has allowed the practical deployment of tens of thousands of detectors, enabling the design of a receiver that can operate up to 19 TES arrays for a total of 32,376 TESes. AliCPT-1 leverages the technological advancements in the detector design from multiple generations of previously successful feedhorn-coupled polarimeters, and in the instrument design from BICEP-3, but applied on a larger scale. The cryostat receiver is currently under integration and testing. During the first deployment year, the focal plane will be populated with up to 4 TES arrays. Further TES arrays will be deployed in the following years, fully populating the focal plane with 19 arrays on the fourth deployment year. Here we present the AliCPT-1 receiver design, and how the design has been optimized to meet the experimental requirements.

astro-ph.IM↗

arXiv2021

On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times

In spite that Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively, recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks. In the meanwhile, Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks. However, the adoption of DP by clients in FL may significantly jeopardize the model accuracy. It is still an open problem to understand the practicality of DP from a theoretic perspective. In this paper, we make the first attempt to understand the practicality of DP in FL through tuning the number of conducted iterations. Based on the FedAvg algorithm, we formally derive the convergence rate with DP noises in FL. Then, we theoretically derive: 1) the conditions for the DP based FedAvg to converge as the number of global iterations (GI) approaches infinity; 2) the method to set the number of local iterations (LI) to minimize the negative influence of DP noises. By further substituting the Laplace and Gaussian mechanisms into the derived convergence rate respectively, we show that: 3) The DP based FedAvg with the Laplace mechanism cannot converge, but the divergence rate can be effectively prohibited by setting the number of LIs with our method; 4) The learning error of the DP based FedAvg with the Gaussian mechanism can converge to a constant number finally if we use a fixed number of LIs per GI. To verify our theoretical findings, we conduct extensive experiments using two real-world datasets. The results not only validate our analysis results, but also provide useful guidelines on how to optimize model accuracy when incorporating DP into FL

cs.CR↗

arXiv2021

On the $L^\infty$ stability of Prandtl expansions in Gevrey class

In this paper, we prove the $L^\infty\cap L^2$ stability of Prandtl expansions of shear flow type as $\big(U(y/\sqrtν),0\big)$ for the initial perturbation in the Gevrey class, where $U(y)$ is a monotone and concave function and $ν$ is the viscosity coefficient. To this end, we develop the direct resolvent estimate method for the linearized Orr-Sommerfeld operator instead of the Rayleigh-Airy iteration method introduced by Grenier, Guo and Nguyen.

math.AP↗

arXiv2020

Aspects of the dyonic Kerr-Sen-AdS$_4$ black hole and its ultraspinning version

We explore some (especially, thermodynamical) properties of the dyonic Kerr-Sen-AdS$_4$ black hole and its ultraspinning counterpart, and check whether or not both black holes satisfy the first law and Bekenstein-Smarr mass formulas. To this end, new Christodoulou-Ruffini-like squared-mass formulae for the usual dyonic Kerr-Sen-AdS$_4$ solution and its ultraspinning cousin are deduced. Similar to the ultraspinning Kerr-Sen-AdS$_4$ black hole case, we demonstrate that the ultraspinning dyonic Kerr-Sen-AdS$_4$ black hole does not always violate the reverse isoperimetric inequality (RII) since the value of the isoperimetric ratio can either be larger/smaller than, or equal to unity, depending upon the range of the solution parameters, as is the case only with an electric charge. This property is apparently distinct from that of the superentropic dyonic Kerr-Newman-AdS$_4$ black hole, which always strictly violates the RII, although both of them have some similar properties in other aspects, such as the horizon geometry and conformal boundary.

gr-qc↗

arXiv2020

Smooth solutions to the Gauss image problem

In this paper we study the the Gauss image problem, which is a generalization of the Aleksandrov problem in convex geometry. By considering a geometric flow involving Gauss curvature and functions of normal vectors and radial vectors, we obtain the existence of smooth solutions to this problem.

math.AP↗

arXiv2020

Generalized Short Circuit Ratio for Grid Strength Assessment in Inhomogeneous Multi-infeed LCC-HVDC Systems

Generalized short circuit ratio (gSCR) for gird strength assessment of multi-infeed high voltage direct current (MIDC) systems is a rigorous theoretical extension of traditional short circuit ratio (SCR), which allows the considerable experience of using SCR to be extended to MIDC systems. However, gSCR was originally derived based on the assumption of homogeneous MIDC systems, where all HVDC converters have an identical control configuration, which poses challenges to the applications of gSCR to inhomogeneous MIDC systems. To weaken this assumption, this letter applies modal perturbation theory to explore the possibility of applying gSCR in inhomogeneous MIDC systems. Results of numerical experiments show that, in inhomogeneous MIDC systems, the previously proposed gSCR can still be used without modification, but critical gSCR (CgSCR) needs to be redefined by considering the characteristics of HVDC converter control configurations. Accordingly, the difference between gSCR and redefined CgSCR can effectively quantify the pertinent ac grid strength in terms of static voltage stability margin. The performance of our proposed method is demonstrated in a triple-infeed inhomogeneous LCC-HVDC system.

eess.SY↗

arXiv2020

Impact of Temporary Fork on the Evolution of Mining Pools in Blockchain Networks: An Evolutionary Game Analysis

Temporary fork is a fundamental phenomenon in many blockchains with proof of work, and the analysis of temporary fork has recently drawn great attention. Different from existing efforts that focus on the blockchain system factors such as block size, network propagation delay or block generation speed, in this paper we explore a new key dimension of computing power from the miners' perspective. Specifically, we first propose a detailed mathematical model to characterize the impact of computing power competition of the mining pools on the temporary fork. We also derive closed-form formula of the probability of temporary fork and the expected mining reward of a mining pool. To reveal the long-term trends on the computing power distributions over the competing mining pools, we then develop an evolutionary game framework based on the temporary fork modeling and accordingly characterize the set of stable evolution equilibriums. Both extensive numerical simulations and realistic blockchain data based evaluation provide evidence to support our theoretical models and discoveries.

cs.GT↗

arXiv2020

Context-Aware Cross-Attention for Non-Autoregressive Translation

Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence. However, due to the lack of target dependency modelling in the decoder, the conditional generation process heavily depends on the cross-attention. In this paper, we reveal a localness perception problem in NAT cross-attention, for which it is difficult to adequately capture source context. To alleviate this problem, we propose to enhance signals of neighbour source tokens into conventional cross-attention. Experimental results on several representative datasets show that our approach can consistently improve translation quality over strong NAT baselines. Extensive analyses demonstrate that the enhanced cross-attention achieves better exploitation of source contexts by leveraging both local and global information.

cs.CL↗