arXiv subjects

Liang Wang

Publications and source records attributed to Liang Wang.

At least 487 records · Page 27Linked to original sources

arXiv2022

UKD: Debiasing Conversion Rate Estimation via Uncertainty-regularized Knowledge Distillation

In online advertising, conventional post-click conversion rate (CVR) estimation models are trained using clicked samples. However, during online serving the models need to estimate for all impression ads, leading to the sample selection bias (SSB) issue. Intuitively, providing reliable supervision signals for unclicked ads is a feasible way to alleviate the SSB issue. This paper proposes an uncertainty-regularized knowledge distillation (UKD) framework to debias CVR estimation via distilling knowledge from unclicked ads. A teacher model learns click-adaptive representations and produces pseudo-conversion labels on unclicked ads as supervision signals. Then a student model is trained on both clicked and unclicked ads with knowledge distillation, performing uncertainty modeling to alleviate the inherent noise in pseudo-labels. Experiments on billion-scale datasets show that UKD outperforms previous debiasing methods. Online results verify that UKD achieves significant improvements.

cs.IR↗

arXiv2022

Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework

With the fast-growing number of classification models being produced every day, numerous model interpretation and comparison solutions have also been introduced. For example, LIME and SHAP can interpret what input features contribute more to a classifier's output predictions. Different numerical metrics (e.g., accuracy) can be used to easily compare two classifiers. However, few works can interpret the contribution of a data feature to a classifier in comparison with its contribution to another classifier. This comparative interpretation can help to disclose the fundamental difference between two classifiers, select classifiers in different feature conditions, and better ensemble two classifiers. To accomplish it, we propose a learning-from-disagreement (LFD) framework to visually compare two classification models. Specifically, LFD identifies data instances with disagreed predictions from two compared classifiers and trains a discriminator to learn from the disagreed instances. As the two classifiers' training features may not be available, we train the discriminator through a set of meta-features proposed based on certain hypotheses of the classifiers to probe their behaviors. Interpreting the trained discriminator with the SHAP values of different meta-features, we provide actionable insights into the compared classifiers. Also, we introduce multiple metrics to profile the importance of meta-features from different perspectives. With these metrics, one can easily identify meta-features with the most complementary behaviors in two classifiers, and use them to better ensemble the classifiers. We focus on binary classification models in the financial services and advertising industry to demonstrate the efficacy of our proposed framework and visualizations.

cs.LG↗

arXiv2022

A type of oscillatory integral operator and its applications

In this paper, we consider $L^p$- estimate for a class of oscillatory integral operators satisfying the Carleson-Sjölin conditions with further convex and straight assumptions. As applications, the multiplier problem related to a general class of hypersurfaces with nonvanishing Gaussian curvature, local smoothing estimates for the fractional Schrödinger equation and the sharp resolvent estimates outside of the uniform boundedness range are discussed.

math.AP↗

arXiv2021

Knowledge Graph Embedding in E-commerce Applications: Attentive Reasoning, Explanations, and Transferable Rules

Knowledge Graphs (KGs), representing facts as triples, have been widely adopted in many applications. Reasoning tasks such as link prediction and rule induction are important for the development of KGs. Knowledge Graph Embeddings (KGEs) embedding entities and relations of a KG into continuous vector spaces, have been proposed for these reasoning tasks and proven to be efficient and robust. But the plausibility and feasibility of applying and deploying KGEs in real-work applications has not been well-explored. In this paper, we discuss and report our experiences of deploying KGEs in a real domain application: e-commerce. We first identity three important desiderata for e-commerce KG systems: 1) attentive reasoning, reasoning over a few target relations of more concerns instead of all; 2) explanation, providing explanations for a prediction to help both users and business operators understand why the prediction is made; 3) transferable rules, generating reusable rules to accelerate the deployment of a KG to new systems. While non existing KGE could meet all these desiderata, we propose a novel one, an explainable knowledge graph attention network that make prediction through modeling correlations between triples rather than purely relying on its head entity, relation and tail entity embeddings. It could automatically selects attentive triples for prediction and records the contribution of them at the same time, from which explanations could be easily provided and transferable rules could be efficiently produced. We empirically show that our method is capable of meeting all three desiderata in our e-commerce application and outperform typical baselines on datasets from real domain applications.

cs.AI↗

arXiv2021

Power-type derivatives for rough volatility with jumps

In this paper we propose a novel pricing-hedging framework for volatility derivatives which simultaneously takes into account rough volatility and volatility jumps. Our model directly targets the instantaneous variance of a risky asset and consists of a generalized fractional Ornstein-Uhlenbeck process driven by a Lévy subordinator and an independent sinusoidal-composite Lévy process. The former component captures short-term dependence in the instantaneous volatility, while the latter is introduced expressly for rectifying the activity level of the average forward variance. Such a framework ensures that the characteristic function of average forward variance is obtainable in semi-closed form, without having to invoke any geometric-mean approximations. To analyze swaps and European-style options on average forward volatility, we introduce a general class of power-type derivatives on the average forward variance, which also provide flexible nonlinear leverage exposure. Pricing-hedging formulae are based on a modified numerical Fourier transform technique. A comparative empirical study is conducted on two independent recent data sets on VIX options, before and during the COVID-19 pandemic, to demonstrate that the proposed framework is highly amenable to efficient model calibration under various choices of kernels.

q-fin.PR↗

arXiv2021

Long-Term CSI-based Design for RIS-Aided Multiuser MISO Systems Exploiting Deep Reinforcement Learning

In this paper, we study the transmission design for reconfigurable intelligent surface (RIS)-aided multiuser communication networks. Different from most of the existing contributions, we consider long-term CSI-based transmission design, where both the beamforming vectors at the base station (BS) and the phase shifts at the RIS are designed based on long-term CSI, which can significantly reduce the channel estimation overhead. Due to the lack of explicit ergodic data rate expression, we propose a novel deep deterministic policy gradient (DDPG) based algorithm to solve the optimization problem, which was trained by using the channel vectors generated in an offline manner. Simulation results demonstrate that the achievable net throughput is higher than that achieved by the conventional instantaneous-CSI based scheme when taking the channel estimation overhead into account.

cs.IT↗

arXiv2021

Animal Behavior Classification via Accelerometry Data and Recurrent Neural Networks

We study the classification of animal behavior using accelerometry data through various recurrent neural network (RNN) models. We evaluate the classification performance and complexity of the considered models, which feature long short-time memory (LSTM) or gated recurrent unit (GRU) architectures with varying depths and widths, using four datasets acquired from cattle via collar or ear tags. We also include two state-of-the-art convolutional neural network (CNN)-based time-series classification models in the evaluations. The results show that the RNN-based models can achieve similar or higher classification accuracy compared with the CNN-based models while having less computational and memory requirements. We also observe that the models with GRU architecture generally outperform the ones with LSTM architecture in terms of classification accuracy despite being less complex. A single-layer uni-directional GRU model with 64 hidden units appears to offer a good balance between accuracy and complexity making it suitable for implementation on edge/embedded devices.

cs.LG↗

arXiv2021

FakeTransformer: Exposing Face Forgery From Spatial-Temporal Representation Modeled By Facial Pixel Variations

With the rapid development of generation model, AI-based face manipulation technology, which called DeepFakes, has become more and more realistic. This means of face forgery can attack any target, which poses a new threat to personal privacy and property security. Moreover, the misuse of synthetic video shows potential dangers in many areas, such as identity harassment, pornography and news rumors. Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish between real videos and synthetic videos from the variations of facial pixels, which are highly related to physiological information. Our approach first applies Eulerian Video Magnification (EVM) at multiple Gaussian scales to the original video to enlarge the physiological variations caused by the change of facial blood volume, and then transform the original video and magnified videos into a Multi-Scale Eulerian Magnified Spatial-Temporal map (MEMSTmap), which can represent time-varying physiological enhancement sequences on different octaves. Then, these maps are reshaped into frame patches in column units and sent to the vision Transformer to learn the spatio-time descriptors of frame levels. Finally, we sort out the feature embedding and output the probability of judging whether the video is real or fake. We validate our method on the FaceForensics++ and DeepFake Detection datasets. The results show that our model achieves excellent performance in forgery detection, and also show outstanding generalization capability in cross-data domain.

cs.CV↗

arXiv2021

Representation and Correlation Enhanced Encoder-Decoder Framework for Scene Text Recognition

Attention-based encoder-decoder framework is widely used in the scene text recognition task. However, for the current state-of-the-art(SOTA) methods, there is room for improvement in terms of the efficient usage of local visual and global context information of the input text image, as well as the robust correlation between the scene processing module(encoder) and the text processing module(decoder). In this paper, we propose a Representation and Correlation Enhanced Encoder-Decoder Framework(RCEED) to address these deficiencies and break performance bottleneck. In the encoder module, local visual feature, global context feature, and position information are aligned and fused to generate a small-size comprehensive feature map. In the decoder module, two methods are utilized to enhance the correlation between scene and text feature space. 1) The decoder initialization is guided by the holistic feature and global glimpse vector exported from the encoder. 2) The feature enriched glimpse vector produced by the Multi-Head General Attention is used to assist the RNN iteration and the character prediction at each time step. Meanwhile, we also design a Layernorm-Dropout LSTM cell to improve model's generalization towards changeable texts. Extensive experiments on the benchmarks demonstrate the advantageous performance of RCEED in scene text recognition tasks, especially the irregular ones.

cs.CV↗

arXiv2021

Data-Plane Security Applications in Adversarial Settings

High-speed programmable switches have emerged as a promising building block for developing performant data-plane applications. In this paper, we argue that the resource constraints and programming model in hardware switches has led to developers adopting problematic design patterns, whose security implications are not widely understood. We bridge the gap by identifying the major challenges and common design pitfalls in switch-based applications in adversarial settings. Examining six recently-proposed switch-based security applications, we find that adversaries can exploit these design pitfalls to completely bypass the protection these applications were designed to provide, or disrupt system operations by introducing collateral damage.

cs.CR↗

arXiv2021

Learning Diverse Policies in MOBA Games via Macro-Goals

Recently, many researchers have made successful progress in building the AI systems for MOBA-game-playing with deep reinforcement learning, such as on Dota 2 and Honor of Kings. Even though these AI systems have achieved or even exceeded human-level performance, they still suffer from the lack of policy diversity. In this paper, we propose a novel Macro-Goals Guided framework, called MGG, to learn diverse policies in MOBA games. MGG abstracts strategies as macro-goals from human demonstrations and trains a Meta-Controller to predict these macro-goals. To enhance policy diversity, MGG samples macro-goals from the Meta-Controller prediction and guides the training process towards these goals. Experimental results on the typical MOBA game Honor of Kings demonstrate that MGG can execute diverse policies in different matches and lineups, and also outperform the state-of-the-art methods over 102 heroes.

cs.LG↗

arXiv2021

Anomaly Detection in Dynamic Graphs via Transformer

Detecting anomalies for dynamic graphs has drawn increasing attention due to their wide applications in social networks, e-commerce, and cybersecurity. Recent deep learning-based approaches have shown promising results over shallow methods. However, they fail to address two core challenges of anomaly detection in dynamic graphs: the lack of informative encoding for unattributed nodes and the difficulty of learning discriminate knowledge from coupled spatial-temporal dynamic graphs. To overcome these challenges, in this paper, we present a novel Transformer-based Anomaly Detection framework for DYnamic graphs (TADDY). Our framework constructs a comprehensive node encoding strategy to better represent each node's structural and temporal roles in an evolving graphs stream. Meanwhile, TADDY captures informative representation from dynamic graphs with coupled spatial-temporal patterns via a dynamic graph transformer model. The extensive experimental results demonstrate that our proposed TADDY framework outperforms the state-of-the-art methods by a large margin on six real-world datasets.

cs.LG↗

arXiv2021

Relation-aware Heterogeneous Graph for User Profiling

User profiling has long been an important problem that investigates user interests in many real applications. Some recent works regard users and their interacted objects as entities of a graph and turn the problem into a node classification task. However, they neglect the difference of distinct interaction types, e.g. user clicks an item v.s.user purchases an item, and thus cannot incorporate such information well. To solve these issues, we propose to leverage the relation-aware heterogeneous graph method for user profiling, which also allows capturing significant meta relations. We adopt the query, key, and value mechanism in a transformer fashion for heterogeneous message passing so that entities can effectively interact with each other. Via such interactions on different relation types, our model can generate representations with rich information for the user profile prediction. We conduct experiments on two real-world e-commerce datasets and observe a significant performance boost of our approach.

cs.IR↗

arXiv2021

Learning the dynamics of coupled oscillators from transients

Whereas the importance of transient dynamics to the functionality and management of complex systems has been increasingly recognized, most of the studies are based on models. Yet in realistic situations the models are often unknown and what available are only measured time series. Meanwhile, many real-world systems are dynamically stable, in the sense that the systems return to their equilibria in a short time after perturbations. This increases further the difficulty of dynamics analysis, as many information of the system dynamics are lost once the system is settled onto the equilibrium states. The question we ask is: given the transient time series of a complex dynamical system measured in the stable regime, can we infer from the data some properties of the system dynamics and make predictions, e.g., predicting the critical point where the equilibrium state becomes unstable? We show that for the typical transitions in system of coupled oscillators, including quorum sensing, amplitude death and complete synchronization, this question can be addressed by the technique of reservoir computing in machine learning. More specifically, by the transient series acquired at several states in the stable regime, we demonstrate that the trained machine is able to predict accurately not only the transient behaviors of the system in the stable regime, but also the critical point where the stable state becomes unstable. Considering the ubiquitous existence of transient activities in natural and man-made systems, the findings may have broad applications.

nlin.AO↗

arXiv2021

Online Multi-horizon Transaction Metric Estimation with Multi-modal Learning in Payment Networks

Predicting metrics associated with entities' transnational behavior within payment processing networks is essential for system monitoring. Multivariate time series, aggregated from the past transaction history, can provide valuable insights for such prediction. The general multivariate time series prediction problem has been well studied and applied across several domains, including manufacturing, medical, and entomology. However, new domain-related challenges associated with the data such as concept drift and multi-modality have surfaced in addition to the real-time requirements of handling the payment transaction data at scale. In this work, we study the problem of multivariate time series prediction for estimating transaction metrics associated with entities in the payment transaction database. We propose a model with five unique components to estimate the transaction metrics from multi-modality data. Four of these components capture interaction, temporal, scale, and shape perspectives, and the fifth component fuses these perspectives together. We also propose a hybrid offline/online training scheme to address concept drift in the data and fulfill the real-time requirements. Combining the estimation model with a graphical user interface, the prototype transaction metric estimation system has demonstrated its potential benefit as a tool for improving a payment processing company's system monitoring capability.

cs.LG↗

arXiv2021

Aligning Cross-lingual Sentence Representations with Dual Momentum Contrast

In this paper, we propose to align sentence representations from different languages into a unified embedding space, where semantic similarities (both cross-lingual and monolingual) can be computed with a simple dot product. Pre-trained language models are fine-tuned with the translation ranking task. Existing work (Feng et al., 2020) uses sentences within the same batch as negatives, which can suffer from the issue of easy negatives. We adapt MoCo (He et al., 2020) to further improve the quality of alignment. As the experimental results show, the sentence representations produced by our model achieve the new state-of-the-art on several tasks, including Tatoeba en-zh similarity search (Artetxe and Schwenk, 2019b), BUCC en-zh bitext mining, and semantic textual similarity on 7 datasets.

cs.CL↗

arXiv2021

Chemical abundances of three new Ba stars from the Keck/HIRES spectra

Based on high resolution, high signal-to-noise (S/N) ratio spectra from Keck/HIRES, we have determined abundances of 20 elements for 18 Ba candidates. The parameter space of these stars are in the range of 4880 $\leq$ $\rm{T_{eff}}$ $\leq$ 6050 K, 2.56 $\leq$ log $g$ $\leq$ 4.53 dex and -0.27 $\leq$ [Fe/H] $\leq$ 0.09 dex. It is found that four of them can be identified as Ba stars with [s/Fe] $>$ 0.25 dex (s: Sr, Y, Zr, Ba, La, Ce and Nd), and three of them are newly discovered, which includes two Ba giants (HD 16178 and HD 22233) and one Ba subgiant (HD 2946). Our results show that the abundances of $α$, odd and iron-peak elements (O, Na, Mg, Al, Si, Ca, Sc, Ti, Mn, Ni and Cu) for our program stars are similar to those of the thin disk, while the distribution of [hs/ls] (hs: Ba, La, Ce and Nd, ls: Sr, Y and Zr) ratios of our Ba stars is similar to those of the known Ba objects. None of the four Ba stars show clear enhancement in carbon including the known CH subgiant HD 4395. It is found that three of the Ba stars present clear evidences of hosting stellar or sub-stellar companions from the radial velocity data.

astro-ph.SR↗

arXiv2021

Bifurcated backbone strategy for RGB-D salient object detection

Multi-level feature fusion is a fundamental topic in computer vision. It has been exploited to detect, segment and classify objects at various scales. When multi-level features meet multi-modal cues, the optimal feature aggregation and multi-modal learning strategy become a hot potato. In this paper, we leverage the inherent multi-modal and multi-level nature of RGB-D salient object detection to devise a novel cascaded refinement network. In particular, first, we propose to regroup the multi-level features into teacher and student features using a bifurcated backbone strategy (BBS). Second, we introduce a depth-enhanced module (DEM) to excavate informative depth cues from the channel and spatial views. Then, RGB and depth modalities are fused in a complementary way. Our architecture, named Bifurcated Backbone Strategy Network (BBS-Net), is simple, efficient, and backbone-independent. Extensive experiments show that BBS-Net significantly outperforms eighteen SOTA models on eight challenging datasets under five evaluation measures, demonstrating the superiority of our approach ($\sim 4 \%$ improvement in S-measure $vs.$ the top-ranked model: DMRA-iccv2019). In addition, we provide a comprehensive analysis on the generalization ability of different RGB-D datasets and provide a powerful training set for future research.

cs.CV↗