arXiv subjects

Peng Zhao

Publications and source records attributed to Peng Zhao.

At least 199 records · Page 11Linked to original sources

arXiv2021

Improved Analysis for Dynamic Regret of Strongly Convex and Smooth Functions

In this paper, we present an improved analysis for dynamic regret of strongly convex and smooth functions. Specifically, we investigate the Online Multiple Gradient Descent (OMGD) algorithm proposed by Zhang et al. (2017). The original analysis shows that the dynamic regret of OMGD is at most $\mathcal{O}(\min\{\mathcal{P}_T,\mathcal{S}_T\})$, where $\mathcal{P}_T$ and $\mathcal{S}_T$ are path-length and squared path-length that measures the cumulative movement of minimizers of the online functions. We demonstrate that by an improved analysis, the dynamic regret of OMGD can be improved to $\mathcal{O}(\min\{\mathcal{P}_T,\mathcal{S}_T,\mathcal{V}_T\})$, where $\mathcal{V}_T$ is the function variation of the online functions. Note that the quantities of $\mathcal{P}_T, \mathcal{S}_T, \mathcal{V}_T$ essentially reflect different aspects of environmental non-stationarity -- they are not comparable in general and are favored in different scenarios. Therefore, the dynamic regret presented in this paper actually achieves a \emph{best-of-three-worlds} guarantee and is strictly tighter than previous results.

cs.LG↗

arXiv2021

Pinpointing the Memory Behaviors of DNN Training

The training of deep neural networks (DNNs) is usually memory-hungry due to the limited device memory capacity of DNN accelerators. Characterizing the memory behaviors of DNN training is critical to optimize the device memory pressures. In this work, we pinpoint the memory behaviors of each device memory block of GPU during training by instrumenting the memory allocators of the runtime system. Our results show that the memory access patterns of device memory blocks are stable and follow an iterative fashion. These observations are useful for the future optimization of memory-efficient training from the perspective of raw memory access patterns.

cs.PF↗

arXiv2021

Large Motion Video Super-Resolution with Dual Subnet and Multi-Stage Communicated Upsampling

Video super-resolution (VSR) aims at restoring a video in low-resolution (LR) and improving it to higher-resolution (HR). Due to the characteristics of video tasks, it is very important that motion information among frames should be well concerned, summarized and utilized for guidance in a VSR algorithm. Especially, when a video contains large motion, conventional methods easily bring incoherent results or artifacts. In this paper, we propose a novel deep neural network with Dual Subnet and Multi-stage Communicated Upsampling (DSMC) for super-resolution of videos with large motion. We design a new module named U-shaped residual dense network with 3D convolution (U3D-RDN) for fine implicit motion estimation and motion compensation (MEMC) as well as coarse spatial feature extraction. And we present a new Multi-Stage Communicated Upsampling (MSCU) module to make full use of the intermediate results of upsampling for guiding the VSR. Moreover, a novel dual subnet is devised to aid the training of our DSMC, whose dual loss helps to reduce the solution space as well as enhance the generalization ability. Our experimental results confirm that our method achieves superior performance on videos with large motion compared to state-of-the-art methods.

cs.CV↗

arXiv2021

Modeling Multivariate Cyber Risks: Deep Learning Dating Extreme Value Theory

Modeling cyber risks has been an important but challenging task in the domain of cyber security. It is mainly because of the high dimensionality and heavy tails of risk patterns. Those obstacles have hindered the development of statistical modeling of the multivariate cyber risks. In this work, we propose a novel approach for modeling the multivariate cyber risks which relies on the deep learning and extreme value theory. The proposed model not only enjoys the high accurate point predictions via deep learning but also can provide the satisfactory high quantile prediction via extreme value theory. The simulation study shows that the proposed model can model the multivariate cyber risks very well and provide satisfactory prediction performances. The empirical evidence based on real honeypot attack data also shows that the proposed model has very satisfactory prediction performances.

stat.AP↗

arXiv2021

Storage Fit Learning with Feature Evolvable Streams

Feature evolvable learning has been widely studied in recent years where old features will vanish and new features will emerge when learning with streams. Conventional methods usually assume that a label will be revealed after prediction at each time step. However, in practice, this assumption may not hold whereas no label will be given at most time steps. A good solution is to leverage the technique of manifold regularization to utilize the previous similar data to assist the refinement of the online model. Nevertheless, this approach needs to store all previous data which is impossible in learning with streams that arrive sequentially in large volume. Thus we need a buffer to store part of them. Considering that different devices may have different storage budgets, the learning approaches should be flexible subject to the storage budget limit. In this paper, we propose a new setting: Storage-Fit Feature-Evolvable streaming Learning (SF$^2$EL) which incorporates the issue of rarely-provided labels into feature evolution. Our framework is able to fit its behavior to different storage budgets when learning with feature evolvable streams with unlabeled data. Besides, both theoretical and empirical results validate that our approach can preserve the merit of the original feature evolvable learning i.e., can always track the best baseline and thus perform well at any time step.

cs.LG↗

arXiv2020

Dynamic Regret of Convex and Smooth Functions

We investigate online convex optimization in non-stationary environments and choose the dynamic regret as the performance measure, defined as the difference between cumulative loss incurred by the online algorithm and that of any feasible comparator sequence. Let $T$ be the time horizon and $P_T$ be the path-length that essentially reflects the non-stationarity of environments, the state-of-the-art dynamic regret is $\mathcal{O}(\sqrt{T(1+P_T)})$. Although this bound is proved to be minimax optimal for convex functions, in this paper, we demonstrate that it is possible to further enhance the dynamic regret by exploiting the smoothness condition. Specifically, we propose novel online algorithms that are capable of leveraging smoothness and replace the dependence on $T$ in the dynamic regret by problem-dependent quantities: the variation in gradients of loss functions, the cumulative loss of the comparator sequence, and the minimum of the previous two terms. These quantities are at most $\mathcal{O}(T)$ while could be much smaller in benign environments. Therefore, our results are adaptive to the intrinsic difficulty of the problem, since the bounds are tighter than existing results for easy problems and meanwhile guarantee the same rate in the worst case.

cs.LG↗

arXiv2020

An Unbiased Risk Estimator for Learning with Augmented Classes

This paper studies the problem of learning with augmented classes (LAC), where augmented classes unobserved in the training data might emerge in the testing phase. Previous studies generally attempt to discover augmented classes by exploiting geometric properties, achieving inspiring empirical performance yet lacking theoretical understandings particularly on the generalization ability. In this paper we show that, by using unlabeled training data to approximate the potential distribution of augmented classes, an unbiased risk estimator of the testing distribution can be established for the LAC problem under mild assumptions, which paves a way to develop a sound approach with theoretical guarantees. Moreover, the proposed approach can adapt to complex changing environments where augmented classes may appear and the prior of known classes may change simultaneously. Extensive experiments confirm the effectiveness of our proposed approach.

cs.LG↗

arXiv2020

Impact of the Central Frequency of Environment on Non-Markovian Dynamics in Piezoelectric Optomechanical Devices

The piezoelectric optomechanical devices supply a promising experimental platform to realize the coherent and effective control and measurement of optical circuits working in Terahertz (THz) frequencies via superconducting electron devices typically working in Radio (MHz) frequencies. However, quantum fluctuations are unavoidable when the size of mechanical oscillators enters into the nanoscale. The consequences of the noisy environment are still challenging due to the lack of analytical tools. In this paper, a semi-classical and full-quantum model of piezoelectric optomechanical systems coupled to a noisy bosonic quantum environment are introduced and solved in terms of quantum-state diffusion (QSD) trajectories in the non-Markovian regime. We show that the noisy environment, particularly the central frequency of the environment, can enhance the entanglement generation between optical cavities and LC circuits in some parameter regimes. Moreover, we observe the critical points in the coefficient functions, which can lead the different behaviors in the system. Besides, we also witness the entanglement transfers between macroscopic objects due to the memory effect of the environment. Our work can be applied in the fields of electric/ optical switches, and long-distance distribution in a large-scale quantum network.

quant-ph↗

arXiv2020

Partition Functions of $\mathcal{N}=1$ Gauge Theories on $S^2 \times \mathbb{R}^2_\varepsilon$ and Duality

We compute the partition functions of $\mathcal{N} = 1$ gauge theories on $S^2 \times \mathbb{R}^2_\varepsilon$ using supersymmetric localization. The path integral reduces to a sum over vortices at the poles of $S^2$ and at the origin of $\mathbb{R}^2_\varepsilon$. The exact partition functions allow us to test Seiberg duality beyond the supersymmetric index. We propose the $\mathcal{N} = 1$ partition functions on the $Ω$-background, and show that the Nekrasov partition functions can be recovered from these building blocks.

hep-th↗

arXiv2020

Switchable next-nearest-neighbor coupling for controlled two-qubit operations

In a superconducting quantum processor with nearest neighbor coupling, the dispersive interaction between adjacent qubits can result in an effective next-nearest-neighbor coupling whose strength depends on the state of the intermediary qubit. Here, we theoretically explore the possibility of engineering this next-nearest-neighbor coupling to implement controlled two-qubit operations where the intermediary qubit controls the operation on the next-nearest neighbor pair of qubits. In particular, in a system comprising two types of superconducting qubits with anharmonicities of opposite-sign arranged in an -A-B-A- pattern, where the unwanted static ZZ coupling between adjacent qubits could be heavily suppressed, a switchable coupling between the next-nearest-neighbor qubits can be achieved via the intermediary qubit, the qubit state of which functions as an on/off switch for this coupling. Therefore, depending on the adopted activating scheme, various controlled two-qubit operations such as controlled-iSWAP gate can be realized, potentially enabling circuit depth reductions as to a standard decomposition approach for implementing generic quantum algorithms.

quant-ph↗

arXiv2020

Should the Ransomware be Paid?

Ransomware has emerged as one of the most concerned cyber risks in recent years, which has caused millions of dollars monetary loss over the world. It typically demands a certain amount of ransom payment within a limited timeframe to decrypt the encrypted victim's files. This paper explores whether the ransomware should be paid in a novel game-theoretic model from the perspective of Bayesian game. In particular, the new model analyzes the ransom payment strategies within the framework of incomplete information for both hacker and victim. Our results show that there exist pure and randomized Bayesian Nash equilibria under some mild conditions for the hacker and victim. The sufficient conditions that when the ransom should be paid are presented when an organization is compromised by the ransomware attack. We further study how the costs and probabilities of cracking or recovering affect the expected payoffs of the hacker and the victim in the equilibria. In particular, it is found that the backup option for computer files is not always beneficial, which actually depends on the related cost. Moreover, it is discovered that fake ransomware may be more than expected because of the potential high payoffs. Numerical examples are also presented for illustration.

econ.GN↗

arXiv2020

Latent Dirichlet Allocation Model Training with Differential Privacy

Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for hidden semantic discovery of text data and serves as a fundamental tool for text analysis in various applications. However, the LDA model as well as the training process of LDA may expose the text information in the training data, thus bringing significant privacy concerns. To address the privacy issue in LDA, we systematically investigate the privacy protection of the main-stream LDA training algorithm based on Collapsed Gibbs Sampling (CGS) and propose several differentially private LDA algorithms for typical training scenarios. In particular, we present the first theoretical analysis on the inherent differential privacy guarantee of CGS based LDA training and further propose a centralized privacy-preserving algorithm (HDP-LDA) that can prevent data inference from the intermediate statistics in the CGS training. Also, we propose a locally private LDA training algorithm (LP-LDA) on crowdsourced data to provide local differential privacy for individual data contributors. Furthermore, we extend LP-LDA to an online version as OLP-LDA to achieve LDA training on locally private mini-batches in a streaming setting. Extensive analysis and experiment results validate both the effectiveness and efficiency of our proposed privacy-preserving LDA training algorithms.

cs.LG↗

arXiv2020

A Single Frame and Multi-Frame Joint Network for 360-degree Panorama Video Super-Resolution

Spherical videos, also known as \ang{360} (panorama) videos, can be viewed with various virtual reality devices such as computers and head-mounted displays. They attract large amount of interest since awesome immersion can be experienced when watching spherical videos. However, capturing, storing and transmitting high-resolution spherical videos are extremely expensive. In this paper, we propose a novel single frame and multi-frame joint network (SMFN) for recovering high-resolution spherical videos from low-resolution inputs. To take advantage of pixel-level inter-frame consistency, deformable convolutions are used to eliminate the motion difference between feature maps of the target frame and its neighboring frames. A mixed attention mechanism is devised to enhance the feature representation capability. The dual learning strategy is exerted to constrain the space of solution so that a better solution can be found. A novel loss function based on the weighted mean square error is proposed to emphasize on the super-resolution of the equatorial regions. This is the first attempt to settle the super-resolution of spherical videos, and we collect a novel dataset from the Internet, MiG Panorama Video, which includes 204 videos. Experimental results on 4 representative video clips demonstrate the efficacy of the proposed method. The dataset and code are available at https://github.com/lovepiano/SMFN_For_360VSR.

cs.CV↗

arXiv2020

Joint Cyber Risk Assessment of Network Systems with Heterogeneous Components

Cyber risks are the most common risks encountered by a modern network system. However, it is significantly difficult to assess the joint cyber risk owing to the network topology, risk propagation, and heterogeneities of components. In this paper, we propose a novel backward elimination approach for computing the joint cyber risk encountered by different types of components in a network system; moreover, explicit formulas are also presented. Certain specific network topologies including complete, star, and complete bi-partite topologies are studied. The effects of propagation depth and compromise probabilities on the joint cyber risk are analyzed using stochastic comparisons. The variances and correlations of cyber risks are examined by a simulation experiment. It was discovered that both variances and correlations change rapidly when the propagation depth increases from its initial value. Further, numerical examples are also presented.

cs.SI↗

arXiv2020

Improved existence for the characteristic initial value problem with the conformal Einstein field equations

We adapt Luk's analysis of the characteristic initial value problem in General Relativity to the asymptotic characteristic problem for the conformal Einstein field equations to demonstrate the local existence of solutions in a neighbourhood of the set on which the data are given. In particular, we obtain existence of solutions along a narrow rectangle along null infinity which, in turn, corresponds to an infinite domain in the asymptotic region of the physical spacetime. This result generalises work by Kánnár on the local existence of solutions to the characteristic initial value problem by means of Rendall's reduction strategy. In analysing the conformal Einstein equations we make use of the Newman-Penrose formalism and a gauge due to J. Stewart.

gr-qc↗

arXiv2020

CDC: Classification Driven Compression for Bandwidth Efficient Edge-Cloud Collaborative Deep Learning

The emerging edge-cloud collaborative Deep Learning (DL) paradigm aims at improving the performance of practical DL implementations in terms of cloud bandwidth consumption, response latency, and data privacy preservation. Focusing on bandwidth efficient edge-cloud collaborative training of DNN-based classifiers, we present CDC, a Classification Driven Compression framework that reduces bandwidth consumption while preserving classification accuracy of edge-cloud collaborative DL. Specifically, to reduce bandwidth consumption, for resource-limited edge servers, we develop a lightweight autoencoder with a classification guidance for compression with classification driven feature preservation, which allows edges to only upload the latent code of raw data for accurate global training on the Cloud. Additionally, we design an adjustable quantization scheme adaptively pursuing the tradeoff between bandwidth consumption and classification accuracy under different network conditions, where only fine-tuning is required for rapid compression ratio adjustment. Results of extensive experiments demonstrate that, compared with DNN training with raw data, CDC consumes 14.9 times less bandwidth with an accuracy loss no more than 1.06%, and compared with DNN training with data compressed by AE without guidance, CDC introduces at least 100% lower accuracy loss.

cs.LG↗

arXiv2020

Simultaneously exciting two atoms with photon-mediated Raman interaction

We propose an approach to simultaneously excite two atoms by using cavity-assisted Raman process in combination with cavity photon-mediated interaction. The system consists of a two-level atom and a $Λ$-type or V-type three-level atom, which are coupled together with a cavity mode. Having derived the effective Hamiltonian, we find that under certain circumstances a single photon can simultaneously excite two atoms. In addition, multiple photons and even a classical field can also simultaneously excite two atoms. As an example, we show a scheme to realize our proposal in a circuit QED setup, which is artificial atoms coupled with a cavity. The dynamics and the quantum statistical properties of the process are investigated with experimentally feasible parameters.

quant-ph↗

arXiv2020

Bandit Convex Optimization in Non-stationary Environments

Bandit Convex Optimization (BCO) is a fundamental framework for modeling sequential decision-making with partial information, where the only feedback available to the player is the one-point or two-point function values. In this paper, we investigate BCO in non-stationary environments and choose the \emph{dynamic regret} as the performance measure, which is defined as the difference between the cumulative loss incurred by the algorithm and that of any feasible comparator sequence. Let $T$ be the time horizon and $P_T$ be the path-length of the comparator sequence that reflects the non-stationarity of environments. We propose a novel algorithm that achieves $O(T^{3/4}(1+P_T)^{1/2})$ and $O(T^{1/2}(1+P_T)^{1/2})$ dynamic regret respectively for the one-point and two-point feedback models. The latter result is optimal, matching the $Ω(T^{1/2}(1+P_T)^{1/2})$ lower bound established in this paper. Notably, our algorithm is more adaptive to non-stationary environments since it does not require prior knowledge of the path-length $P_T$ ahead of time, which is generally unknown.

cs.LG↗