arXiv subjects

Shuai Wang

Publications and source records attributed to Shuai Wang.

At least 667 records · Page 37Linked to original sources

arXiv2021

Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimization

Wireless federated learning (FL) is an emerging machine learning paradigm that trains a global parametric model from distributed datasets via wireless communications. This paper proposes a unit-modulus wireless FL (UMWFL) framework, which simultaneously uploads local model parameters and computes global model parameters via optimized phase shifting. The proposed framework avoids sophisticated baseband signal processing, leading to both low communication delays and implementation costs. A training loss bound is derived and a penalty alternating minimization (PAM) algorithm is proposed to minimize the nonconvex nonsmooth loss bound. Experimental results in the Car Learning to Act (CARLA) platform show that the proposed UMWFL framework with PAM algorithm achieves smaller training losses and testing errors than those of the benchmark scheme.

eess.SP↗

arXiv2021

Self-Supervised Learning Based Domain Adaptation for Robust Speaker Verification

Large performance degradation is often observed for speaker ver-ification systems when applied to a new domain dataset. Givenan unlabeled target-domain dataset, unsupervised domain adaptation(UDA) methods, which usually leverage adversarial training strate-gies, are commonly used to bridge the performance gap caused bythe domain mismatch. However, such adversarial training strategyonly uses the distribution information of target domain data and cannot ensure the performance improvement on the target domain. Inthis paper, we incorporate self-supervised learning strategy to the un-supervised domain adaptation system and proposed a self-supervisedlearning based domain adaptation approach (SSDA). Compared tothe traditional UDA method, the new SSDA training strategy canfully leverage the potential label information from target domainand adapt the speaker discrimination ability from source domainsimultaneously. We evaluated the proposed approach on the Vox-Celeb (labeled source domain) and CnCeleb (unlabeled target do-main) datasets, and the best SSDA system obtains 10.2% Equal ErrorRate (EER) on the CnCeleb dataset without using any speaker labelson CnCeleb, which also can achieve the state-of-the-art results onthis corpus.

cs.SD↗

arXiv2021

Temperature Measurement Based on Electron Spin Resonance of Magnetic Nanoparticles

Magnetic nanoparticles (MNPs) have excellent magnetic-temperature characteristic. However, current temperature measurement based on MNPs is interfered by concentration. Utilizing the electron spin resonance (ESR), we propose a highly sensitive temperature measurement method without concentration coupling. The anisotropic field is affected by temperature, thus affecting the g-value. The influence of the MNP concentration, size, and the data analysis method on temperature estimation are studied. The optimal temperature sensitivity is achieved with 15-nm MNPs while Gaussian smoothing method allows an optimal accuracy at Fe concentration of 5 mg/ml with a root mean squared error of 0.07 K.

cond-mat.mes-hall↗

arXiv2021

Clustering by Orthogonal NMF Model and Non-Convex Penalty Optimization

The non-negative matrix factorization (NMF) model with an additional orthogonality constraint on one of the factor matrices, called the orthogonal NMF (ONMF), has been found a promising clustering model and can outperform the classical K-means. However, solving the ONMF model is a challenging optimization problem because the coupling of the orthogonality and non-negativity constraints introduces a mixed combinatorial aspect into the problem due to the determination of the correct status of the variables (positive or zero). Most of the existing methods directly deal with the orthogonality constraint in its original form via various optimization techniques, but are not scalable for large-scale problems. In this paper, we propose a new ONMF based clustering formulation that equivalently transforms the orthogonality constraint into a set of norm-based non-convex equality constraints. We then apply a non-convex penalty (NCP) approach to add them to the objective as penalty terms, leading to a problem that is efficiently solvable. One smooth penalty formulation and one non-smooth penalty formulation are respectively studied. We build theoretical conditions for the penalized problems to provide feasible stationary solutions to the ONMF based clustering problem, as well as proposing efficient algorithms for solving the penalized problems of the two NCP methods. Experimental results based on both synthetic and real datasets are presented to show that the proposed NCP methods are computationally time efficient, and either match or outperform the existing K-means and ONMF based methods in terms of the clustering performance.

cs.LG↗

arXiv2021

Accelerating Federated Edge Learning via Optimized Probabilistic Device Scheduling

The popular federated edge learning (FEEL) framework allows privacy-preserving collaborative model training via frequent learning-updates exchange between edge devices and server. Due to the constrained bandwidth, only a subset of devices can upload their updates at each communication round. This has led to an active research area in FEEL studying the optimal device scheduling policy for minimizing communication time. However, owing to the difficulty in quantifying the exact communication time, prior work in this area can only tackle the problem partially by considering either the communication rounds or per-round latency, while the total communication time is determined by both metrics. To close this gap, we make the first attempt in this paper to formulate and solve the communication time minimization problem. We first derive a tight bound to approximate the communication time through cross-disciplinary effort involving both learning theory for convergence analysis and communication theory for per-round latency analysis. Building on the analytical result, an optimized probabilistic scheduling policy is derived in closed-form by solving the approximate communication time minimization problem. It is found that the optimized policy gradually turns its priority from suppressing the remaining communication rounds to reducing per-round latency as the training process evolves. The effectiveness of the proposed scheme is demonstrated via a use case on collaborative 3D objective detection in autonomous driving.

cs.NI↗

arXiv2021

Data Partition and Rate Control for Learning and Energy Efficient Edge Intelligence

The rapid development of artificial intelligence together with the powerful computation capabilities of the advanced edge servers make it possible to deploy learning tasks at the wireless network edge, which is dubbed as edge intelligence (EI). The communication bottleneck between the data resource and the server results in deteriorated learning performance as well as tremendous energy consumption. To tackle this challenge, we explore a new paradigm called learning-and-energy-efficient (LEE) EI, which simultaneously maximizes the learning accuracies and energy efficiencies of multiple tasks via data partition and rate control. Mathematically, this results in a multi-objective optimization problem. Moreover, the continuous varying rates over the whole transmission duration introduce infinite variables. To solve this complex problem, we consider the case with infinite server buffer capacity and one-shot data arrival at sensor. First, the number of variables are reduced to a finite level by exploiting the optimality of constant-rate transmission in each epoch. Second, the optimal solution is found by applying stratified sequencing or objectives merging. By assuming higher priority of learning efficiency in stratified sequencing, the closed form of optimal data partition is derived by the Lagrange method, while the optimal rate control is proved to have the structure of directional water filling (DWF), based on which a string-pulling (SP) algorithm is proposed to obtain the numerical values. The DWF structure of rate control is also proved to be optimal in objectives merging via weighted summation. By exploiting the optimal rate changing properties, the SP algorithm is further extended to account for the cases with limited server buffer capacity or bursty data arrival at sensor. The performance of the proposed design is examined by extensive experiments based on public datasets.

cs.IT↗

arXiv2021

A Knowledge-Driven Approach to Classifying Object and Attribute Coreferences in Opinion Mining

Classifying and resolving coreferences of objects (e.g., product names) and attributes (e.g., product aspects) in opinionated reviews is crucial for improving the opinion mining performance. However, the task is challenging as one often needs to consider domain-specific knowledge (e.g., iPad is a tablet and has aspect resolution) to identify coreferences in opinionated reviews. Also, compiling a handcrafted and curated domain-specific knowledge base for each domain is very time consuming and arduous. This paper proposes an approach to automatically mine and leverage domain-specific knowledge for classifying objects and attribute coreferences. The approach extracts domain-specific knowledge from unlabeled review data and trains a knowledgeaware neural coreference classification model to leverage (useful) domain knowledge together with general commonsense knowledge for the task. Experimental evaluation on realworld datasets involving five domains (product types) shows the effectiveness of the approach.

cs.CL↗

arXiv2021

The DoF Region of Order-(K-1) Messages for the K-user MIMO Broadcast Channel with Delayed CSIT

This paper theoretically characterizes the degrees-of-freedom (DoF) region of order-$(K-1)$ messages for the $K$-user multiple-input multiple-output (MIMO) broadcast channel with delayed channel state information at the transmitter (CSIT) and arbitrary antenna configurations, where the transmitter has $M$ antennas and the receiver $i=1,2,\cdots,K$ has $N_i$ antennas. For the converse, we first derive the DoF region of order-$(K-1)$ messages for the $K$-user MIMO broadcast channel with no CSIT and arbitrary antenna configurations with the aid of the proposed Genie-bound, and then establish the DoF outer region. For the achievability, we first design a 2-phase transmission scheme, and then propose a backward/forward cancellation algorithm for decoding. Specifically, we efficiently derive the achievable DoF region from the designed transmission scheme by transformation approach. The main implication of this paper is that for the order-$(K-1)$ messages of $K$-user MIMO broadcast channel, the DoF region with delayed CSIT is larger than the DoF region with no CSIT when $N_2<M$, where $N_1 \le N_2 \le \cdots \le N_K$.

cs.IT↗

arXiv2021

Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition

Few-shot Named Entity Recognition (NER) exploits only a handful of annotations to identify and classify named entity mentions. Prototypical network shows superior performance on few-shot NER. However, existing prototypical methods fail to differentiate rich semantics in other-class words, which will aggravate overfitting under few shot scenario. To address the issue, we propose a novel model, Mining Undefined Classes from Other-class (MUCO), that can automatically induce different undefined classes from the other class to improve few-shot NER. With these extra-labeled undefined classes, our method will improve the discriminative ability of NER classifier and enhance the understanding of predefined classes with stand-by semantic knowledge. Experimental results demonstrate that our model outperforms five state-of-the-art models in both 1-shot and 5-shots settings on four NER benchmarks. We will release the code upon acceptance. The source code is released on https: //github.com/shuaiwa16/OtherClassNER.git.

cs.CL↗

arXiv2021

Detecting Changed-Hands Online Review Accounts

A reputable social media or review account can be a good cover for spamming activities. It has become prevalent that spammers buy/sell such accounts openly on the Web. We call these sold/bought accounts the changed-hands (CH) accounts. They are hard to detect by existing spam detection algorithms as their spamming activities are under the disguise of clean histories. In this paper, we first propose the problem of detecting CH accounts, and then design an effective detection algorithm which exploits changes in content and writing styles of individual accounts, and a proposed novel feature selection method that works at a fine-grained level within each individual account. The proposed method not only determines if an account has changed hands, but also pinpoints the change point. Experimental results with online review accounts demonstrate the high effectiveness of our approach.

cs.SI↗

arXiv2021

Multi-Task Learning and Adapted Knowledge Models for Emotion-Cause Extraction

Detecting what emotions are expressed in text is a well-studied problem in natural language processing. However, research on finer grained emotion analysis such as what causes an emotion is still in its infancy. We present solutions that tackle both emotion recognition and emotion cause detection in a joint fashion. Considering that common-sense knowledge plays an important role in understanding implicitly expressed emotions and the reasons for those emotions, we propose novel methods that combine common-sense knowledge via adapted knowledge models with multi-task learning to perform joint emotion classification and emotion cause tagging. We show performance improvement on both tasks when including common-sense reasoning and a multitask framework. We provide a thorough analysis to gain insights into model performance.

cs.CL↗

arXiv2021

Privileged Graph Distillation for Cold Start Recommendation

The cold start problem in recommender systems is a long-standing challenge, which requires recommending to new users (items) based on attributes without any historical interaction records. In these recommendation systems, warm users (items) have privileged collaborative signals of interaction records compared to cold start users (items), and these Collaborative Filtering (CF) signals are shown to have competing performance for recommendation. Many researchers proposed to learn the correlation between collaborative signal embedding space and the attribute embedding space to improve the cold start recommendation, in which user and item categorical attributes are available in many online platforms. However, the cold start recommendation is still limited by two embedding spaces modeling and simple assumptions of space transformation. As user-item interaction behaviors and user (item) attributes naturally form a heterogeneous graph structure, in this paper, we propose a privileged graph distillation model~(PGD). The teacher model is composed of a heterogeneous graph structure for warm users and items with privileged CF links. The student model is composed of an entity-attribute graph without CF links. Specifically, the teacher model can learn better embeddings of each entity by injecting complex higher-order relationships from the constructed heterogeneous graph. The student model can learn the distilled output with privileged CF embeddings from the teacher embeddings. Our proposed model is generally applicable to different cold start scenarios with new user, new item, or new user-new item. Finally, extensive experimental results on the real-world datasets clearly show the effectiveness of our proposed model on different types of cold start problems, with average $6.6\%, 5.6\%, $ and $17.1\%$ improvement over state-of-the-art baselines on three datasets, respectively.

cs.IR↗

arXiv2021

Towards Bidirectional Protection in Federated Learning

Prior efforts in enhancing federated learning (FL) security fall into two categories. At one end of the spectrum, some work uses secure aggregation techniques to hide the individual client's updates and only reveal the aggregated global update to a malicious server that strives to infer the clients' privacy from their updates. At the other end of the spectrum, some work uses Byzantine-robust FL protocols to suppress the influence of malicious clients' updates. We present a federated learning protocol F2ED-LEARNING, which, for the first time, offers bidirectional defense to simultaneously combat against the malicious centralized server and Byzantine malicious clients. To defend against Byzantine malicious clients, F2ED-LEARNING provides dimension-free estimation error by employing and calibrating a well-studied robust mean estimator FilterL2. F2ED-LEARNING also leverages secure aggregation to protect clients from a malicious server. One key challenge of F2ED-LEARNING is to address the incompatibility between FilterL2 and secure aggregation schemes. Concretely, FilterL2 has to check the individual updates from clients whereas secure aggregation hides those updates from the malicious server. To this end, we propose a practical and highly effective solution to split the clients into shards, where F2ED-LEARNING securely aggregates each shard's update and launches FilterL2 on updates from different shards. The evaluation shows that F2ED-LEARNING consistently achieves optimal or close-to-optimal performance and outperforms five secure FL protocols under five popular attacks.

cs.DC↗

arXiv2021

FED-$χ^2$: Privacy Preserving Federated Correlation Test

In this paper, we propose the first secure federated $χ^2$-test protocol Fed-$χ^2$. To minimize both the privacy leakage and the communication cost, we recast $χ^2$-test to the second moment estimation problem and thus can take advantage of stable projection to encode the local information in a short vector. As such encodings can be aggregated with only summation, secure aggregation can be naturally applied to hide the individual updates. We formally prove the security guarantee of Fed-$χ^2$ that the joint distribution is hidden in a subspace with exponential possible distributions. Our evaluation results show that Fed-$χ^2$ achieves negligible accuracy drops with small client-side computation overhead. In several real-world case studies, the performance of Fed-$χ^2$ is comparable to the centralized $χ^2$-test.

cs.CR↗

arXiv2021

High-Resolution Segmentation of Tooth Root Fuzzy Edge Based on Polynomial Curve Fitting with Landmark Detection

As the most economical and routine auxiliary examination in the diagnosis of root canal treatment, oral X-ray has been widely used by stomatologists. It is still challenging to segment the tooth root with a blurry boundary for the traditional image segmentation method. To this end, we propose a model for high-resolution segmentation based on polynomial curve fitting with landmark detection (HS-PCL). It is based on detecting multiple landmarks evenly distributed on the edge of the tooth root to fit a smooth polynomial curve as the segmentation of the tooth root, thereby solving the problem of fuzzy edge. In our model, a maximum number of the shortest distances algorithm (MNSDA) is proposed to automatically reduce the negative influence of the wrong landmarks which are detected incorrectly and deviate from the tooth root on the fitting result. Our numerical experiments demonstrate that the proposed approach not only reduces Hausdorff95 (HD95) by 33.9% and Average Surface Distance (ASD) by 42.1% compared with the state-of-the-art method, but it also achieves excellent results on the minute quantity of datasets, which greatly improves the feasibility of automatic root canal therapy evaluation by medical image computing.

cs.CV↗

arXiv2021

Secure Massive RIS aided Multicast with Uncertain CSI: Energy-Efficiency Maximization via Accelerated First-Order Algorithms

Reconfigurable intelligent surface (RIS) has the potential to significantly enhance the network secure transmission performance by reconfiguring the wireless propagation environment. However, due to the passive nature of eavesdroppers and the cascaded channel brought by the RIS, the eavesdroppers' channel state information is imperfectly obtained at the base station. Under the channel uncertainty, the optimal phase-shift, power allocation, and transmission rate design for secure transmission is currently unknown due to the difficulty of handling the probabilistic constraint with coupled variables. To fill this gap, this paper formulates a problem of energy-efficient secure transmission design while incorporating the probabilistic constraint. By transforming the probabilistic constraint and decoupling the variables, the secure energy efficiency maximization problem can be solved via alternatively executing concave-convex procedure and semidefinite relaxation technique. To scale the solution to massive antennas and reflecting elements scenario, an accelerated first-order algorithm with low complexity is further proposed.Simulation results show that the proposed accelerated first-order algorithm achieves identical performance to the conventional method but saves at least two orders of magnitude in computation time. Moreover, the resultant RIS aided secure transmission significantly improves the energy efficiency compared to baseline schemes of random phase-shift, fixed phase-shift, and RIS ignoring CSI uncertainty.

cs.IT↗

arXiv2021

Federated Meta Learning Enhanced Acoustic Radio Cooperative Framework for Ocean of Things Underwater Acoustic Communications

Sixth-generation wireless communication (6G) will be an integrated architecture of "space, air, ground and sea". One of the most difficult part of this architecture is the underwater information acquisition which need to transmitt information cross the interface between water and air.In this senario, ocean of things (OoT) will play an important role, because it can serve as a hub connecting Internet of things (IoT) and Internet of underwater things (IoUT). OoT device not only can collect data through underwater methods, but also can utilize radio frequence over the air. For underwater communications, underwater acoustic communications (UWA COMMs) is the most effective way for OoT devices to exchange information, but it is always tormented by doppler shift and synchronization errors. In this paper, in order to overcome UWA tough conditions, a deep neural networks based receiver for underwater acoustic chirp communication, called C-DNN, is proposed. Moreover, to improve the performance of DL-model and solve the problem of model generalization, we also proposed a novel federated meta learning (FML) enhanced acoustic radio cooperative (ARC) framework, dubbed ARC/FML, to do transfer. Particularly, tractable expressions are derived for the convergence rate of FML in a wireless setting, accounting for effects from both scheduling ratio, local epoch and the data amount on a single node.From our analysis and simulation results, it is shown that, the proposed C-DNN can provide a better BER performance and lower complexity than classical matched filter (MF) in underwater acoustic communications scenario. The ARC/FML framework has good convergence under a variety of channels than federated learning (FL). In summary, the proposed ARC/FML for OoT is a promising scheme for information exchange across water and air.

eess.SP↗

arXiv2021

Voice activity detection in the wild: A data-driven approach using teacher-student training

Voice activity detection is an essential pre-processing component for speech-related tasks such as automatic speech recognition (ASR). Traditional supervised VAD systems obtain frame-level labels from an ASR pipeline by using, e.g., a Hidden Markov model. These ASR models are commonly trained on clean and fully transcribed data, limiting VAD systems to be trained on clean or synthetically noised datasets. Therefore, a major challenge for supervised VAD systems is their generalization towards noisy, real-world data. This work proposes a data-driven teacher-student approach for VAD, which utilizes vast and unconstrained audio data for training. Unlike previous approaches, only weak labels during teacher training are required, enabling the utilization of any real-world, potentially noisy dataset. Our approach firstly trains a teacher model on a source dataset (Audioset) using clip-level supervision. After training, the teacher provides frame-level guidance to a student model on an unlabeled, target dataset. A multitude of student models trained on mid- to large-sized datasets are investigated (Audioset, Voxceleb, NIST SRE). Our approach is then respectively evaluated on clean, artificially noised, and real-world data. We observe significant performance gains in artificially noised and real-world scenarios. Lastly, we compare our approach against other unsupervised and supervised VAD methods, demonstrating our method's superiority.

cs.SD↗