arXiv subjects

Xin Guo

Publications and source records attributed to Xin Guo.

At least 163 records · Page 9Linked to original sources

arXiv2021

Connecting GANs, MFGs, and OT

Generative adversarial networks (GANs) have enjoyed tremendous success in image generation and processing, and have recently attracted growing interests in financial modelings. This paper analyzes GANs from the perspectives of mean-field games (MFGs) and optimal transport. More specifically, from the game theoretical perspective, GANs are interpreted as MFGs under Pareto Optimality criterion or mean-field controls; from the optimal transport perspective, GANs are to minimize the optimal transport cost indexed by the generator from the known latent distribution to the unknown true distribution of data. The MFGs perspective of GANs leads to a GAN-based computational method (MFGANs) to solve MFGs: one neural network for the backward Hamilton-Jacobi-Bellman equation and one neural network for the forward Fokker-Planck equation, with the two neural networks trained in an adversarial way. Numerical experiments demonstrate superior performance of this proposed algorithm, especially in the higher dimensional case, when compared with existing neural network approaches.

cs.GT↗

arXiv2021

FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update

Federated learning aims to protect users' privacy while performing data analysis from different participants. However, it is challenging to guarantee the training efficiency on heterogeneous systems due to the various computational capabilities and communication bottlenecks. In this work, we propose FedSkel to enable computation-efficient and communication-efficient federated learning on edge devices by only updating the model's essential parts, named skeleton networks. FedSkel is evaluated on real edge devices with imbalanced datasets. Experimental results show that it could achieve up to 5.52$\times$ speedups for CONV layers' back-propagation, 1.82$\times$ speedups for the whole training process, and reduce 64.8% communication cost, with negligible accuracy loss.

cs.LG↗

arXiv2021

DanceIt: Music-inspired Dancing Video Synthesis

Close your eyes and listen to music, one can easily imagine an actor dancing rhythmically along with the music. These dance movements are usually made up of dance movements you have seen before. In this paper, we propose to reproduce such an inherent capability of the human-being within a computer vision system. The proposed system consists of three modules. To explore the relationship between music and dance movements, we propose a cross-modal alignment module that focuses on dancing video clips, accompanied on pre-designed music, to learn a system that can judge the consistency between the visual features of pose sequences and the acoustic features of music. The learned model is then used in the imagination module to select a pose sequence for the given music. Such pose sequence selected from the music, however, is usually discontinuous. To solve this problem, in the spatial-temporal alignment module we develop a spatial alignment algorithm based on the tendency and periodicity of dance movements to predict dance movements between discontinuous fragments. In addition, the selected pose sequence is often misaligned with the music beat. To solve this problem, we further develop a temporal alignment algorithm to align the rhythm of music and dance. Finally, the processed pose sequence is used to synthesize realistic dancing videos in the imagination module. The generated dancing videos match the content and rhythm of the music. Experimental results and subjective evaluations show that the proposed approach can perform the function of generating promising dancing videos by inputting music.

cs.CV↗

arXiv2021

Relaxed Wasserstein with Applications to GANs

Wasserstein Generative Adversarial Networks (WGANs) provide a versatile class of models, which have attracted great attention in various applications. However, this framework has two main drawbacks: (i) Wasserstein-1 (or Earth-Mover) distance is restrictive such that WGANs cannot always fit data geometry well; (ii) It is difficult to achieve fast training of WGANs. In this paper, we propose a new class of \textit{Relaxed Wasserstein} (RW) distances by generalizing Wasserstein-1 distance with Bregman cost functions. We show that RW distances achieve nice statistical properties while not sacrificing the computational tractability. Combined with the GANs framework, we develop Relaxed WGANs (RWGANs) which are not only statistically flexible but can be approximated efficiently using heuristic approaches. Experiments on real images demonstrate that the RWGAN with Kullback-Leibler (KL) cost function outperforms other competing approaches, e.g., WGANs, even with gradient penalty.

stat.ML↗

arXiv2021

Plasmon-Exciton Coupling Effect on Plasmon Damping

Plasmon decay via the surface or interface is a critical process for practical energy conversion and plasmonic catalysis. However, the relationship between plasmon damping and the coupling between the plasmon and 2D materials is still unclear. The spectral splitting due to plasmon-exciton interaction impedes the conventional single-particle method to evaluate the plasmon damping rate by the spectral linewidth directly. Here, we investigated the interaction between a single gold nanorod (GNR) and 2D materials using the single-particle spectroscopy method assisted with in situ nanomanipulation technique by comparing scattering intensity and linewidth together. Our approach allows us to indisputably identify that the plasmon-exciton coupling in the GNR-WSe2 hybrid would induce plasmon damping. We can also isolate the contribution between the charge transfer channel and resonant energy transfer channel for the plasmon decay in the GNR-graphene hybrid by comparing that with thin hBN layers as an intermediate medium to block the charge transfer. We find out that the contact layer between the GNR and 2D materials contributes most of the interfacial plasmon damping. These findings contribute to a deep understanding of interfacial excitonic effects on the plasmon and 2D materials hybrid.

physics.optics↗

arXiv2021

Interbank lending with benchmark rates: Pareto optima for a class of singular control games

We analyze a class of stochastic differential games of singular control, motivated by the study of a dynamic model of interbank lending with benchmark rates. We describe Pareto optima for this game and show how they may be achieved through the intervention of a regulator, whose policy is a solution to a singular stochastic control problem. Pareto optima are characterized in terms of the solutions to a new class of Skorokhod problems with piecewise-continuous free boundary. Pareto optimal policies are shown to correspond to the enforcement of endogenous bounds on interbank lending rates. Analytical comparison between Pareto optima and Nash equilibria provides insight into the impact of regulatory intervention on the stability of interbank rates.

math.OC↗

arXiv2021

The application of adaptive perturbation theory in strongly coupled double harmonic oscillator system

The idea of adaptive perturbation theory is to divide a Hamiltonian into a solvable part and a perturbation part. The solvable part contains the non-interacting sector and the diagonal elements of Fock space from the interacting terms. The perturbed term is the non-diagonal sector of Fock space. Therefore, the perturbation parameter is not coupling constant. This is different from the standard procedure of previous perturbation method. In this letter, we use the adaptive perturbation theory to extract the solvable elements in the strongly coupled double harmonic oscillator system and obtain the energy spectrum of the solvable part. Then, we diagonalize the Hamiltonian to obtain the numerical solution. In order to study the accuracy of adaptive perturbation theory in the strongly coupled double harmonic oscillator system, we demonstrate the analytical study from the leading order and second-order perturbation. The deviations between the leading-order and the numerical solution show that as the gap between the quasiparticle number n1 and n2 decreases, or the coupling constant $λ$ increases, the gap between the exact solutions in the strongly coupling region of double harmonic oscillator system and the numerical solutions becomes smaller. Overall, the numerical gap is about 1% to 3%, which is not a bad result. The value of the second-order is quite close to the numerical solution when n1 doesn't equal n2. In most cases, the deviation is less than 1%, which means that the adaptive perturbation theory is effective for the double harmonic oscillator system in strong coupling field.

quant-ph↗

arXiv2021

RoSearch: Search for Robust Student Architectures When Distilling Pre-trained Language Models

Pre-trained language models achieve outstanding performance in NLP tasks. Various knowledge distillation methods have been proposed to reduce the heavy computation and storage requirements of pre-trained language models. However, from our observations, student models acquired by knowledge distillation suffer from adversarial attacks, which limits their usage in security sensitive scenarios. In order to overcome these security problems, RoSearch is proposed as a comprehensive framework to search the student models with better adversarial robustness when performing knowledge distillation. A directed acyclic graph based search space is built and an evolutionary search strategy is utilized to guide the searching approach. Each searched architecture is trained by knowledge distillation on pre-trained language model and then evaluated under a robustness-, accuracy- and efficiency-aware metric as environmental fitness. Experimental results show that RoSearch can improve robustness of student models from 7%~18% up to 45.8%~47.8% on different datasets with comparable weight compression ratio to existing distillation methods (4.6$\times$~6.5$\times$ improvement from teacher model BERT_BASE) and low accuracy drop. In addition, we summarize the relationship between student architecture and robustness through statistics of searched models.

cs.CL↗

arXiv2021

Discriminative-Generative Dual Memory Video Anomaly Detection

Recently, people tried to use a few anomalies for video anomaly detection (VAD) instead of only normal data during the training process. A side effect of data imbalance occurs when a few abnormal data face a vast number of normal data. The latest VAD works use triplet loss or data re-sampling strategy to lessen this problem. However, there is still no elaborately designed structure for discriminative VAD with a few anomalies. In this paper, we propose a DiscRiminative-gEnerative duAl Memory (DREAM) anomaly detection model to take advantage of a few anomalies and solve data imbalance. We use two shallow discriminators to tighten the normal feature distribution boundary along with a generator for the next frame prediction. Further, we propose a dual memory module to obtain a sparse feature representation in both normality and abnormality space. As a result, DREAM not only solves the data imbalance problem but also learn a reasonable feature space. Further theoretical analysis shows that our DREAM also works for the unknown anomalies. Comparing with the previous methods on UCSD Ped1, UCSD Ped2, CUHK Avenue, and ShanghaiTech, our model outperforms all the baselines with no extra parameters. The ablation study demonstrates the effectiveness of our dual memory module and discriminative-generative network.

cs.CV↗

arXiv2021

Delay Analysis of Wireless Federated Learning Based on Saddle Point Approximation and Large Deviation Theory

Federated learning (FL) is a collaborative machine learning paradigm, which enables deep learning model training over a large volume of decentralized data residing in mobile devices without accessing clients' private data. Driven by the ever increasing demand for model training of mobile applications or devices, a vast majority of FL tasks are implemented over wireless fading channels. Due to the time-varying nature of wireless channels, however, random delay occurs in both the uplink and downlink transmissions of FL. How to analyze the overall time consumption of a wireless FL task, or more specifically, a FL's delay distribution, becomes a challenging but important open problem, especially for delay-sensitive model training. In this paper, we present a unified framework to calculate the approximate delay distributions of FL over arbitrary fading channels. Specifically, saddle point approximation, extreme value theory (EVT), and large deviation theory (LDT) are jointly exploited to find the approximate delay distribution along with its tail distribution, which characterizes the quality-of-service of a wireless FL system. Simulation results will demonstrate that our approximation method achieves a small approximation error, which vanishes with the increase of training accuracy.

eess.SP↗

arXiv2021

Edge-Labeling based Directed Gated Graph Network for Few-shot Learning

Existing graph-network-based few-shot learning methods obtain similarity between nodes through a convolution neural network (CNN). However, the CNN is designed for image data with spatial information rather than vector form node feature. In this paper, we proposed an edge-labeling-based directed gated graph network (DGGN) for few-shot learning, which utilizes gated recurrent units to implicitly update the similarity between nodes. DGGN is composed of a gated node aggregation module and an improved gated recurrent unit (GRU) based edge update module. Specifically, the node update module adopts a gate mechanism using activation of edge feature, making a learnable node aggregation process. Besides, improved GRU cells are employed in the edge update procedure to compute the similarity between nodes. Further, this mechanism is beneficial to gradient backpropagation through the GRU sequence across layers. Experiment results conducted on two benchmark datasets show that our DGGN achieves a comparable performance to the-state-of-art methods.

cs.CV↗

arXiv2020

Universal Silicon Microwave Photonic Spectral Shaper

Optical modulation plays arguably the utmost important role in microwave photonic (MWP) systems. Precise synthesis of modulated optical spectra dictates virtually all aspects of MWP system quality including loss, noise figure, linearity, and the types of functionality that can be executed. But for such a critical function, the versatility to generate and transform analog optical modulation is severely lacking, blocking the pathways to truly unique MWP functions including ultra-linear links and low-loss high rejection filters. Here we demonstrate versatile RF photonic spectrum synthesis in an all-integrated silicon photonic circuit, enabling electrically-tailorable universal analog modulation transformation. We show a series of unprecedented RF filtering experiments through monolithic integration of the spectrum-synthesis circuit with a network of reconfigurable ring resonators.

eess.SP↗

arXiv2020

The Buckley-Osthus model and the block preferential attachment model: statistical analysis and application

This paper is concerned with statistical estimation of two preferential attachment models: the Buckley-Osthus model and the hierarchical preferential attachment model. We prove that the maximum likelihood estimates for both models are consistent. We perform simulation studies to corroborate our theoretical findings. We also apply both models to study the evolution of a real-world network. A list of open problems are presented.

math.ST↗

arXiv2020

All-Dielectric Nanolaser with Subnanometer Field Confinement

The ability to generate a laser field with ultratight spatial confinement is important for pushing the limit of optical science and technology. Although plasmon lasers can offer sub-diffraction-confined laser fields, it is restricted by a trade-off between optical confinement and loss of free-electron oscillation in metals. We show here an all-dielectric nanolaser that can circumvent the confinement-loss trade-off, and offer a field confinement down to subnanometer level. The ultra-confined lasing field, supported by low-loss oscillation of polarized bound electrons surrounding a 1-nm-width slit in a coupled CdS nanowire pair, is manifested as the dominant peak of a TE0-like lasing mode around 520-nm wavelength, with a field confinement down to 0.23 nm and a peak-to-background ratio of ~32.7 dB. This laser may pave a way towards new regions for nanolasers and light-matter interaction.

physics.optics↗

arXiv2020

Nonzero-sum stochastic games and mean-field games with impulse controls

We consider a general class of nonzero-sum $N$-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limit situation of $N \to \infty$, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, the existence of unique NE solution to the MFG, which is an $ε$-NE approximation to the $N$-player game, with $ε=O\left(\frac{1}{\sqrt{N}}\right)$. As an example, we analyze in details a class of two-player stochastic games which extends the classical cash management problem to the game setting. In particular, we present numerical analysis for the cases of the single player, the two-player game, and the MFG, showing the impact of competition on the player's optimal strategy, with sensitivity analysis of the model parameters.

math.OC↗

arXiv2020

Kernel-based L_2-Boosting with Structure Constraints

Developing efficient kernel methods for regression is very popular in the past decade. In this paper, utilizing boosting on kernel-based weaker learners, we propose a novel kernel-based learning algorithm called kernel-based re-scaled boosting with truncation, dubbed as KReBooT. The proposed KReBooT benefits in controlling the structure of estimators and producing sparse estimate, and is near overfitting resistant. We conduct both theoretical analysis and numerical simulations to illustrate the power of KReBooT. Theoretically, we prove that KReBooT can achieve the almost optimal numerical convergence rate for nonlinear approximation. Furthermore, using the recently developed integral operator approach and a variant of Talagrand's concentration inequality, we provide fast learning rates for KReBooT, which is a new record of boosting-type algorithms. Numerically, we carry out a series of simulations to show the promising performance of KReBooT in terms of its good generalization, near over-fitting resistance and structure constraints.

cs.LG↗

arXiv2020

MFGs for partially reversible investment

This paper analyzes a class of infinite-time-horizon stochastic games with singular controls motivated from the partially reversible problem. It provides an explicit solution for the mean-field game (MFG) and presents sensitivity analysis to compare the solution for the MFG with that for the single-agent control problem. It shows that in the MFG, model parameters not only affect the optimal strategies as in the single-agent case, but also influence the equilibrium price. It then establishes that the solution to the MFG is an $ε$-Nash Equilibrium to the corresponding $N$-player game, with $ε=O\left(\frac{1}{\sqrt N}\right)$.

math.OC↗

arXiv2020

Accelerating CNN Training by Pruning Activation Gradients

Sparsification is an efficient approach to accelerate CNN inference, but it is challenging to take advantage of sparsity in training procedure because the involved gradients are dynamically changed. Actually, an important observation shows that most of the activation gradients in back-propagation are very close to zero and only have a tiny impact on weight-updating. Hence, we consider pruning these very small gradients randomly to accelerate CNN training according to the statistical distribution of activation gradients. Meanwhile, we theoretically analyze the impact of pruning algorithm on the convergence. The proposed approach is evaluated on AlexNet and ResNet-\{18, 34, 50, 101, 152\} with CIFAR-\{10, 100\} and ImageNet datasets. Experimental results show that our training approach could substantially achieve up to $5.92 \times$ speedups at back-propagation stage with negligible accuracy loss.

cs.LG↗