arXiv subjects

Bo Liu

Publications and source records attributed to Bo Liu.

At least 577 records · Page 32Linked to original sources

arXiv2020

RDP-GAN: A Rényi-Differential Privacy based Generative Adversarial Network

Generative adversarial network (GAN) has attracted increasing attention recently owing to its impressive ability to generate realistic samples with high privacy protection. Without directly interactive with training examples, the generative model can be fully used to estimate the underlying distribution of an original dataset while the discriminative model can examine the quality of the generated samples by comparing the label values with the training examples. However, when GANs are applied on sensitive or private training examples, such as medical or financial records, it is still probable to divulge individuals' sensitive and private information. To mitigate this information leakage and construct a private GAN, in this work we propose a Rényi-differentially private-GAN (RDP-GAN), which achieves differential privacy (DP) in a GAN by carefully adding random noises on the value of the loss function during training. Moreover, we derive the analytical results of the total privacy loss under the subsampling method and cumulated iterations, which show its effectiveness on the privacy budget allocation. In addition, in order to mitigate the negative impact brought by the injecting noise, we enhance the proposed algorithm by adding an adaptive noise tuning step, which will change the volume of added noise according to the testing accuracy. Through extensive experimental results, we verify that the proposed algorithm can achieve a better privacy level while producing high-quality samples compared with a benchmark DP-GAN scheme based on noise perturbation on training gradients.

cs.LG↗

arXiv2020

Finite-Sample Analysis of Proximal Gradient TD Algorithms

In this paper, we analyze the convergence rate of the gradient temporal difference learning (GTD) family of algorithms. Previous analyses of this class of algorithms use ODE techniques to prove asymptotic convergence, and to the best of our knowledge, no finite-sample analysis has been done. Moreover, there has been not much work on finite-sample analysis for convergent off-policy reinforcement learning algorithms. In this paper, we formulate GTD methods as stochastic gradient algorithms w.r.t.~a primal-dual saddle-point objective function, and then conduct a saddle-point error analysis to obtain finite-sample bounds on their performance. Two revised algorithms are also proposed, namely projected GTD2 and GTD2-MP, which offer improved convergence guarantees and acceleration, respectively. The results of our theoretical analysis show that the GTD family of algorithms are indeed comparable to the existing LSTD methods in off-policy learning scenarios.

cs.LG↗

arXiv2020

A Light-Weighted Convolutional Neural Network for Bitemporal SAR Image Change Detection

Recently, many Convolution Neural Networks (CNN) have been successfully employed in bitemporal SAR image change detection. However, most of the existing networks are too heavy and occupy a large volume of memory for storage and calculation. Motivated by this, in this paper, we propose a lightweight neural network to reduce the computational and spatial complexity and facilitate the change detection on an edge device. In the proposed network, we replace normal convolutional layers with bottleneck layers that keep the same number of channels between input and output. Next, we employ dilated convolutional kernels with a few non-zero entries that reduce the running time in convolutional operators. Comparing with the conventional convolutional neural network, our light-weighted neural network will be more efficient with fewer parameters. We verify our light-weighted neural network on four sets of bitemporal SAR images. The experimental results show that the proposed network can obtain better performance than the conventional CNN and has better model generalization, especially on the challenging datasets with complex scenes.

eess.IV↗

arXiv2020

Understanding Global Loss Landscape of One-hidden-layer ReLU Networks, Part 1: Theory

For one-hidden-layer ReLU networks, we prove that all differentiable local minima are global inside differentiable regions. We give the locations and losses of differentiable local minima, and show that these local minima can be isolated points or continuous hyperplanes, depending on an interplay between data, activation pattern of hidden neurons and network size. Furthermore, we give necessary and sufficient conditions for the existence of saddle points as well as non-differentiable local minima, and their locations if they exist.

cs.LG↗

arXiv2020

Understanding Global Loss Landscape of One-hidden-layer ReLU Networks, Part 2: Experiments and Analysis

The existence of local minima for one-hidden-layer ReLU networks has been investigated theoretically in [8]. Based on the theory, in this paper, we first analyze how big the probability of existing local minima is for 1D Gaussian data and how it varies in the whole weight space. We show that this probability is very low in most regions. We then design and implement a linear programming based approach to judge the existence of genuine local minima, and use it to predict whether bad local minima exist for the MNIST and CIFAR-10 datasets, and find that there are no bad differentiable local minima almost everywhere in weight space once some hidden neurons are activated by samples. These theoretical predictions are verified experimentally by showing that gradient descent is not trapped in the cells from which it starts. We also perform experiments to explore the count and size of differentiable cells in the weight space.

cs.LG↗

arXiv2020

Operational definition of a quantum speed limit

The quantum speed limit is a fundamental concept in quantum mechanics, which aims at finding the minimum time scale or the maximum dynamical speed for some fixed targets. In a large number of studies in this field, the construction of valid bounds for the evolution time is always the core mission, yet the physics behind it and some fundamental questions like which states can really fulfill the target, are ignored. Understanding the physics behind the bounds is at least as important as constructing attainable bounds. Here we provide an operational approach for the definition of the quantum speed limit, which utilizes the set of states that can fulfill the target to define the speed limit. Its performances in various scenarios have been investigated. For time-independent Hamiltonians, it is inverse-proportional to the difference between the highest and lowest energies. The fact that its attainability does not require a zero ground-state energy suggests it can be used as an indicator of quantum phase transitions. For time-dependent Hamiltonians, it is shown that contrary to the results given by existing bounds, the true speed limit should be independent of the time. Moreover, in the case of spontaneous emission, we find a counterintuitive phenomenon that a lousy purity can benefit the reduction of the quantum speed limit.

quant-ph↗

arXiv2020

Few-Shot Open-Set Recognition using Meta-Learning

The problem of open-set recognition is considered. While previous approaches only consider this problem in the context of large-scale classifier training, we seek a unified solution for this and the low-shot classification setting. It is argued that the classic softmax classifier is a poor solution for open-set recognition, since it tends to overfit on the training classes. Randomization is then proposed as a solution to this problem. This suggests the use of meta-learning techniques, commonly used for few-shot classification, for the solution of open-set recognition. A new oPen sEt mEta LEaRning (PEELER) algorithm is then introduced. This combines the random selection of a set of novel classes per episode, a loss that maximizes the posterior entropy for examples of those classes, and a new metric learning formulation based on the Mahalanobis distance. Experimental results show that PEELER achieves state of the art open set recognition performance for both few-shot and large-scale recognition. On CIFAR and miniImageNet, it achieves substantial gains in seen/unseen class detection AUROC for a given seen-class classification accuracy.

cs.CV↗

arXiv2020

Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity

In this paper, we introduce proximal gradient temporal difference learning, which provides a principled way of designing and analyzing true stochastic gradient temporal difference learning algorithms. We show how gradient TD (GTD) reinforcement learning methods can be formally derived, not by starting from their original objective functions, as previously attempted, but rather from a primal-dual saddle-point objective function. We also conduct a saddle-point error analysis to obtain finite-sample bounds on their performance. Previous analyses of this class of algorithms use stochastic approximation techniques to prove asymptotic convergence, and do not provide any finite-sample analysis. We also propose an accelerated algorithm, called GTD2-MP, that uses proximal ``mirror maps'' to yield an improved convergence rate. The results of our theoretical analysis imply that the GTD family of algorithms are comparable and may indeed be preferred over existing least squares TD methods for off-policy learning, due to their linear complexity. We provide experimental results showing the improved performance of our accelerated gradient TD methods.

cs.LG↗

arXiv2020

Regularized Off-Policy TD-Learning

We present a novel $l_1$ regularized off-policy convergent TD-learning method (termed RO-TD), which is able to learn sparse representations of value functions with low computational complexity. The algorithmic framework underlying RO-TD integrates two key ideas: off-policy convergent gradient TD methods, such as TDC, and a convex-concave saddle-point formulation of non-smooth convex optimization, which enables first-order solvers and feature selection using online convex regularization. A detailed theoretical and experimental analysis of RO-TD is presented. A variety of experiments are presented to illustrate the off-policy convergence, sparse feature selection capability and low computational cost of the RO-TD algorithm.

cs.LG↗

arXiv2020

Defect-Correlated Skyrmions and Controllable Generation in Perpendicularly Magnetized CoFeB Ultrathin Films

Skyrmions have attracted significant interest due to their topological spin structures and fascinating physical features. The skyrmion phase arises in materials with Dzyaloshinskii-Moriya (DM) interaction at interfaces or in volume of non-centrosymmetric materials. However, although skyrmions were generated experimentally, one critical intrinsic relationship between fabrication, microstructures, magnetization and the existence of skyrmions remains to be established. Here, two series of CoFeB ultrathin films with controlled atomic scale structures are employed to reveal this relationship. By inverting the growth order, the amount of defects can be artificially tuned, and skyrmions are shown to be preferentially formed at defect sites. The stable region and the density of the skyrmions can be efficiently controlled in the return magnetization loops by utilizing first-order reversal curves to reach various metastable states. These findings establish the general and intrinsic relationship from sample preparation to skyrmion generation, offering an universal method to control skyrmion density.

cond-mat.mes-hall↗

arXiv2020

Fast Enhancement for Non-Uniform Illumination Images using Light-weight CNNs

This paper proposes a new light-weight convolutional neural network (5k parameters) for non-uniform illumination image enhancement to handle color, exposure, contrast, noise and artifacts, etc., simultaneously and effectively. More concretely, the input image is first enhanced using Retinex model from dual different aspects (enhancing under-exposure and suppressing over-exposure), respectively. Then, these two enhanced results and the original image are fused to obtain an image with satisfactory brightness, contrast and details. Finally, the extra noise and compression artifacts are removed to get the final result. To train this network, we propose a semi-supervised retouching solution and construct a new dataset (82k images) contains various scenes and light conditions. Our model can enhance 0.5 mega-pixel (like 600*800) images in real time (50 fps), which is faster than existing enhancement methods. Extensive experiments show that our solution is fast and effective to deal with non-uniform illumination images.

cs.CV↗

arXiv2020

Differential K-theory and localization formula for $η$-invariants

In this paper, we obtain a localization formula in differential K-theory for $S^1$-action. Then by combining an extension of Goette's result on the comparison of two types of equivariant $η$-invariants, we establish a version of localization formula for equivariant $η$-invariants. An important step of our approach is to construct a pre-$λ$-ring structure in differential K-theory which is interesting in its own right.

math.DG↗

arXiv2020

A Convolutional Neural Network with Parallel Multi-Scale Spatial Pooling to Detect Temporal Changes in SAR Images

In synthetic aperture radar (SAR) image change detection, it is quite challenging to exploit the changing information from the noisy difference image subject to the speckle. In this paper, we propose a multi-scale spatial pooling (MSSP) network to exploit the changed information from the noisy difference image. Being different from the traditional convolutional network with only mono-scale pooling kernels, in the proposed method, multi-scale pooling kernels are equipped in a convolutional network to exploit the spatial context information on changed regions from the difference image. Furthermore, to verify the generalization of the proposed method, we apply our proposed method to the cross-dataset bitemporal SAR image change detection, where the MSSP network (MSSP-Net) is trained on a dataset and then applied to an unknown testing dataset. We compare the proposed method with other state-of-arts and the comparisons are performed on four challenging datasets of bitemporal SAR images. Experimental results demonstrate that our proposed method obtains comparable results with S-PCA-Net on YR-A and YR-B dataset and outperforms other state-of-art methods, especially on the Sendai-A and Sendai-B datasets with more complex scenes. More important, MSSP-Net is more efficient than S-PCA-Net and convolutional neural networks (CNN) with less executing time in both training and testing phases.

cs.CV↗

arXiv2020

Effect of Dzyaloshinskii-Moriya interaction on magnetic vortex switching driven by radial spin waves

We theoretically investigate the radial-spin-wave induced magnetic vortex switching in the presence of Dzyaloshinskii-Moriya interaction (DMI). From micromagnetic simulations, we observe a circular-to-radial vortex phase transition by increasing the DMI strength. The radial spin-wave excitation spectrum for each magnetization configuration is analyzed, showing that the frequency of spin-wave mode with a given radial node number monotonically increases (decreases) with the DMI parameter of the radial (circular) vortex. Interestingly, we find that the DMI can significantly facilitate the polarity switching of the circular vortex driven by radial spin waves. Our work provides a new insight into the DMI effect on the vortex dynamics and is helpful for designing fast all-magnonic memory devices.

cond-mat.mes-hall↗

arXiv2020

Neutron spin resonance in a quasi-two-dimensional iron-based superconductor

Magnetically mediated Cooper pairing is generally regarded as a key to establish the unified mechanism of unconventional superconductivity. One crucial evidence is the neutron spin resonance arising in the superconducting state, which is commonly interpreted as a spin-exciton from collective particle-hole excitations confined below the superconducting pair-breaking gap ($2Δ$). Here, on the basis of inelastic neutron scattering measurements on a quasi-two-dimensional iron-based superconductor KCa$_2$Fe$_4$As$_4$F$_2$, we have discovered a two-dimensional spin resonant mode with downward dispersions, a behavior closely resembling the low branch of the hour-glass-type spin resonance in cuprates. The resonant intensity is predominant by two broad incommensurate peaks near $Q=$(0.5, 0.5) with a sharp energy peak at $E_R=16$ meV. The overall energy dispersion of the mode exceeds the measured maximum total gap $Δ_{\rm tot}=|Δ_k|+|Δ_{k+Q}|$. These experimental results deeply challenge the conventional understanding of the resonance modes as magnetic excitons regardless of underlining pairing symmetry schemes, and it also points out that when the iron-based superconductivity becomes very quasi-two-dimensional, the electronic behaviors are similar to those in cuprates.

cond-mat.supr-con↗

arXiv2020

DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs

A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty. We cast POMDP filtering and planning problems as two closely related Sequential Monte Carlo (SMC) processes, one over the real states and the other over the future optimal trajectories, and combine the merits of these two parts in a new model named the DualSMC network. In particular, we first introduce an adversarial particle filter that leverages the adversarial relationship between its internal components. Based on the filtering results, we then propose a planning algorithm that extends the previous SMC planning approach [Piche et al., 2018] to continuous POMDPs with an uncertainty-dependent policy. Crucially, not only can DualSMC handle complex observations such as image input but also it remains highly interpretable. It is shown to be effective in three continuous POMDP domains: the floor positioning domain, the 3D light-dark navigation domain, and a modified Reacher domain.

cs.LG↗

arXiv2020

Evolving the pulmonary nodules diagnosis from classical approaches to deep learning aided decision support: three decades development course and future prospect

Lung cancer is the commonest cause of cancer deaths worldwide, and its mortality can be reduced significantly by performing early diagnosis and screening. Since the 1960s, driven by the pressing needs to accurately and effectively interpret the massive volume of chest images generated daily, computer-assisted diagnosis of pulmonary nodule has opened up new opportunities to relax the limitation from physicians' subjectivity, experiences and fatigue. And the fair access to the reliable and affordable computer-assisted diagnosis will fight the inequalities in incidence and mortality between populations. It has been witnessed that significant and remarkable advances have been achieved since the 1980s, and consistent endeavors have been exerted to deal with the grand challenges on how to accurately detect the pulmonary nodules with high sensitivity at low false-positives rate as well as on how to precisely differentiate between benign and malignant nodules. There is a lack of comprehensive examination of the techniques' development which is evolving the pulmonary nodules diagnosis from classical approaches to machine learning-assisted decision support. The main goal of this investigation is to provide a comprehensive state-of-the-art review of the computer-assisted nodules detection and benign-malignant classification techniques developed over 3 decades, which have evolved from the complicated ad hoc analysis pipeline of conventional approaches to the simplified seamlessly integrated deep learning techniques. This review also identifies challenges and highlights opportunities for future work in learning models, learning algorithms and enhancement schemes for bridging current state to future prospect and satisfying future demand.

cs.CV↗

arXiv2020

Exploit Clues from Views: Self-Supervised and Regularized Learning for Multiview Object Recognition

Multiview recognition has been well studied in the literature and achieves decent performance in object recognition and retrieval task. However, most previous works rely on supervised learning and some impractical underlying assumptions, such as the availability of all views in training and inference time. In this work, the problem of multiview self-supervised learning (MV-SSL) is investigated, where only image to object association is given. Given this setup, a novel surrogate task for self-supervised learning is proposed by pursuing "object invariant" representation. This is solved by randomly selecting an image feature of an object as object prototype, accompanied with multiview consistency regularization, which results in view invariant stochastic prototype embedding (VISPE). Experiments shows that the recognition and retrieval results using VISPE outperform that of other self-supervised learning methods on seen and unseen data. VISPE can also be applied to semi-supervised scenario and demonstrates robust performance with limited data available. Code is available at https://github.com/chihhuiho/VISPE

cs.CV↗