arXiv subjects

Han Zhang

Publications and source records attributed to Han Zhang.

At least 415 records · Page 23Linked to original sources

arXiv2021

A Multiscale Graph Convolutional Network for Change Detection in Homogeneous and Heterogeneous Remote Sensing Images

Change detection (CD) in remote sensing images has been an ever-expanding area of research. To date, although many methods have been proposed using various techniques, accurately identifying changes is still a great challenge, especially in the high resolution or heterogeneous situations, due to the difficulties in effectively modeling the features from ground objects with different patterns. In this paper, a novel CD method based on the graph convolutional network (GCN) and multiscale object-based technique is proposed for both homogeneous and heterogeneous images. First, the object-wise high level features are obtained through a pre-trained U-net and the multiscale segmentations. Treating each parcel as a node, the graph representations can be formed and then, fed into the proposed multiscale graph convolutional network with each channel corresponding to one scale. The multiscale GCN propagates the label information from a small number of labeled nodes to the other ones which are unlabeled. Further, to comprehensively incorporate the information from the output channels of multiscale GCN, a fusion strategy is designed using the father-child relationships between scales. Extensive Experiments on optical, SAR and heterogeneous optical/SAR data sets demonstrate that the proposed method outperforms some state-of the-art methods in both qualitative and quantitative evaluations. Besides, the Influences of some factors are also discussed.

cs.CV↗

arXiv2021

Quantum Quench dynamics in XY spin chain with ferromagnetic and antiferromagnetic interactions

In this manuscript we investigate the one-dimensional anisotropic XY model with ferromagnetic and antiferromagnetic interactions, which gives more interesting phase diagrams and dynamic critical behaviors. By using quantum renormalization-group method, we find that there are three phases in the system: antiferromagnetic Ising phase ordered in "x direction", spin-fluid phase and ferromagnetic Ising phase ordered in "y direction". In order to study the dynamical critical behaviors of the system, two quantum quenching methods are used. In both cases, the concurrence, a measure of entanglement, oscillates periodically over time. We show that the periods are the same and can be used as a new order parameter for quantum phase transitions. For further discussion, we derive the scaling exponent, θ, and correlation length exponent, ν, from the scaling behavior of the evolution period.

cond-mat.stat-mech↗

arXiv2020

Modeling Heterogeneous Statistical Patterns in High-dimensional Data by Adversarial Distributions: An Unsupervised Generative Framework

Since the label collecting is prohibitive and time-consuming, unsupervised methods are preferred in applications such as fraud detection. Meanwhile, such applications usually require modeling the intrinsic clusters in high-dimensional data, which usually displays heterogeneous statistical patterns as the patterns of different clusters may appear in different dimensions. Existing methods propose to model the data clusters on selected dimensions, yet globally omitting any dimension may damage the pattern of certain clusters. To address the above issues, we propose a novel unsupervised generative framework called FIRD, which utilizes adversarial distributions to fit and disentangle the heterogeneous statistical patterns. When applying to discrete spaces, FIRD effectively distinguishes the synchronized fraudsters from normal users. Besides, FIRD also provides superior performance on anomaly detection datasets compared with SOTA anomaly detection methods (over 5% average AUC improvement). The significant experiment results on various datasets verify that the proposed method can better model the heterogeneous statistical patterns in high-dimensional data and benefit downstream applications.

cs.LG↗

arXiv2020

Improved Consistency Regularization for GANs

Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator. We improve on this technique in several ways. We first show that consistency regularization can introduce artifacts into the GAN samples and explain how to fix this issue. We then propose several modifications to the consistency regularization procedure designed to improve its performance. We carry out extensive experiments quantifying the benefit of our improvements. For unconditional image synthesis on CIFAR-10 and CelebA, our modifications yield the best known FID scores on various GAN architectures. For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11.48 to 9.21. Finally, on ImageNet-2012, we apply our technique to the original BigGAN model and improve the FID from 6.66 to 5.38, which is the best score at that model size.

stat.ML↗

arXiv2020

A Simple Semi-Supervised Learning Framework for Object Detection

Semi-supervised learning (SSL) has a potential to improve the predictive performance of machine learning models using unlabeled data. Although there has been remarkable recent progress, the scope of demonstration in SSL has mainly been on image classification tasks. In this paper, we propose STAC, a simple yet effective SSL framework for visual object detection along with a data augmentation strategy. STAC deploys highly confident pseudo labels of localized objects from an unlabeled image and updates the model by enforcing consistency via strong augmentations. We propose experimental protocols to evaluate the performance of semi-supervised object detection using MS-COCO and show the efficacy of STAC on both MS-COCO and VOC07. On VOC07, STAC improves the AP$^{0.5}$ from $76.30$ to $79.08$; on MS-COCO, STAC demonstrates $2{\times}$ higher data efficiency by achieving 24.38 mAP using only 5\% labeled data than supervised baseline that marks 23.86\% using 10\% labeled data. The code is available at https://github.com/google-research/ssl_detection/.

cs.CV↗

arXiv2020

From Spectrum Wavelet to Vertex Propagation: Graph Convolutional Networks Based on Taylor Approximation

Graph convolutional networks (GCN) have been recently utilized to extract the underlying structures of datasets with some labeled data and high-dimensional features. Existing GCNs mostly rely on a first-order Chebyshev approximation of graph wavelet-kernels. Such a generic propagation model does not always suit the various datasets and their features. This work revisits the fundamentals of graph wavelet and explores the utility of signal propagation in the vertex domain to approximate the spectral wavelet-kernels. We first derive the conditions for representing the graph wavelet-kernels via vertex propagation. We next propose alternative propagation models for GCN layers based on Taylor expansions. We further analyze the choices of detailed graph representations for TGCNs. Experiments on citation networks, multimedia datasets and synthetic graphs demonstrate the advantage of Taylor-based GCN (TGCN) in the node classification problems over the traditional GCN methods.

eess.SP↗

arXiv2020

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels using the model's predictions on weakly-augmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 -- just 4 labels per class. Since FixMatch bears many similarities to existing SSL methods that achieve worse performance, we carry out an extensive ablation study to tease apart the experimental factors that are most important to FixMatch's success. We make our code available at https://github.com/google-research/fixmatch.

cs.LG↗

arXiv2020

Transfer learning of chaotic systems

Can a neural network trained by the time series of system A be used to predict the evolution of system B? This problem, knowing as transfer learning in a broad sense, is of great importance in machine learning and data mining, yet has not been addressed for chaotic systems. Here we investigate transfer learning of chaotic systems from the perspective of synchronization-based state inference, in which a reservoir computer trained by chaotic system A is used to infer the unmeasured variables of chaotic system B, while A is different from B in either parameter or dynamics. It is found that if systems A and B are different in parameter, the reservoir computer can be well synchronized to system B. However, if systems A and B are different in dynamics, the reservoir computer fails to synchronize with system B in general. Knowledge transfer along a chain of coupled reservoir computers is also studied, and it is found that, although the reservoir computers are trained by different systems, the unmeasured variables of the driving system can be successfully inferred by the remote reservoir computer. Finally, by an experiment of chaotic pendulum, we show that the knowledge learned from the modeling system can be used to predict the evolution of the experimental system.

cs.NE↗

arXiv2020

Differentiable Combinatorial Losses through Generalized Gradients of Linear Programs

When samples have internal structure, we often see a mismatch between the objective optimized during training and the model's goal during inference. For example, in sequence-to-sequence modeling we are interested in high-quality translated sentences, but training typically uses maximum likelihood at the word level. The natural training-time loss would involve a combinatorial problem -- dynamic programming-based global sequence alignment -- but solutions to combinatorial problems are not differentiable with respect to their input parameters, so surrogate, differentiable losses are used instead. Here, we show how to perform gradient descent over combinatorial optimization algorithms that involve continuous parameters, for example edge weights, and can be efficiently expressed as linear programs. We demonstrate usefulness of gradient descent over combinatorial optimization in sequence-to-sequence modeling using differentiable encoder-decoder architecture with softmax or Gumbel-softmax, and in image classification in a weakly supervised setting where instead of the correct class for each photo, only groups of photos labeled with correct but unordered set of classes are available during training.

cs.LG↗

arXiv2020

Co-evolution of Functional Brain Network at Multiple Scales during Early Infancy

The human brains are organized into hierarchically modular networks facilitating efficient and stable information processing and supporting diverse cognitive processes during the course of development. While the remarkable reconfiguration of functional brain network has been firmly established in early life, all these studies investigated the network development from a "single-scale" perspective, which ignore the richness engendered by its hierarchical nature. To fill this gap, this paper leveraged a longitudinal infant resting-state functional magnetic resonance imaging dataset from birth to 2 years of age, and proposed an advanced methodological framework to delineate the multi-scale reconfiguration of functional brain network during early development. Our proposed framework is consist of two parts. The first part developed a novel two-step multi-scale module detection method that could uncover efficient and consistent modular structure for longitudinal dataset from multiple scales in a completely data-driven manner. The second part designed a systematic approach that employed the linear mixed-effect model to four global and nodal module-related metrics to delineate scale-specific age-related changes of network organization. By applying our proposed methodological framework on the collected longitudinal infant dataset, we provided the first evidence that, in the first 2 years of life, the brain functional network is co-evolved at different scales, where each scale displays the unique reconfiguration pattern in terms of modular organization.

q-bio.NC↗

arXiv2020

Three Dimensional Simulations of Solar Wind Preconditioning and the 23 July 2012 Interplanetary Coronal Mass Ejection

Predicting the large-scale eruptions from the solar corona and their propagation through interplanetary space remains an outstanding challenge in solar- and helio-physics research. In this article, we describe three dimensional magnetohydrodynamic simulations of the inner heliosphere leading up to and including the extreme interplanetary coronal mass ejection (ICME) of 23 July 2012, developed using the code PLUTO. The simulations are driven using the output of coronal models for Carrington rotations 2125 and 2126 and, given the uncertainties in the initial conditions, are able to reproduce an event of comparable magnitude to the 23 July ICME, with similar velocity and density profiles at 1 au. The launch-time of this event is then varied with regards to an initial 19 July ICME and the effects of solar wind preconditioning are found to be significant for an event of this magnitude and to decrease over a time-window consistent with the ballistic refilling of the depleted heliospheric sector. These results indicate that the 23 July ICME was mostly unaffected by events prior, but would have travelled even faster had it erupted closer in time to the 19 July event where it would have experienced even lower drag forces. We discuss this systematic study of solar wind preconditioning in the context of space weather forecasting.

astro-ph.SR↗

arXiv2020

Solving Missing-Annotation Object Detection with Background Recalibration Loss

This paper focuses on a novel and challenging detection scenario: A majority of true objects/instances is unlabeled in the datasets, so these missing-labeled areas will be regarded as the background during training. Previous art on this problem has proposed to use soft sampling to re-weight the gradients of RoIs based on the overlaps with positive instances, while their method is mainly based on the two-stage detector (i.e. Faster RCNN) which is more robust and friendly for the missing label scenario. In this paper, we introduce a superior solution called Background Recalibration Loss (BRL) that can automatically re-calibrate the loss signals according to the pre-defined IoU threshold and input image. Our design is built on the one-stage detector which is faster and lighter. Inspired by the Focal Loss formulation, we make several significant modifications to fit on the missing-annotation circumstance. We conduct extensive experiments on the curated PASCAL VOC and MS COCO datasets. The results demonstrate that our proposed method outperforms the baseline and other state-of-the-arts by a large margin. Code available: https://github.com/Dwrety/mmdetection-selective-iou.

cs.CV↗

arXiv2020

Improving NER's Performance with Massive financial corpus

Training large deep neural networks needs massive high quality annotation data, but the time and labor costs are too expensive for small business. We start a company-name recognition task with a small scale and low quality training data, then using skills to enhanced model training speed and predicting performance with minimum labor cost. The methods we use involve pre-training a lite language model such as Albert-small or Electra-small in financial corpus, knowledge of distillation and multi-stage learning. The result is that we raised the recall rate by nearly 20 points and get 4 times as fast as BERT-CRF model.

cs.CL↗

arXiv2020

Ferri-chiral compounds with potentially switchable Dresselhaus spin split-ting

Spin splitting of energy bands can be induced by relativistic spin-orbit interactions in materials without inversion symmetry. Whereas polar space group symmetries permit Rashba (R-1) spin splitting with helical spin textures in momentum space, which could be reversed upon switching a ferroelectric polarization via applied electric fields, the ordinary Dresselhaus effect (D-1A) is ac-tive only in materials exhibiting nonpolar noncentrosymmetric crystal classes with atoms occupy-ing exclusively non-polar lattice sites. Consequently, the spin-momentum locking induced by D-1A is not electric field-switchable. An alternative type of Dresselhaus symmetry, referred to as D-1B, exhibits crystal class constraints similar to D-1A (all dipoles add up to zero), but unlike D-1A, at least one polar site is occupied. We find that this behavior exists in a class of ferri-chiral crys-tals, which in principle could be reversed upon a change in chirality. Focusing on alkali metal chalcogenides, we identify NaCu5S3 in the nonentiamorphic ferri-chiral structure, which exhibits CuS3 chiral units differing in the magnitude of their Cu displacements. We then synthesize NaCu5S3 (space group P6322) and confirm its ferri-chiral structure with power x-ray diffraction. Our electronic structure calculations demonstrate it exhibits D-1B spin splitting as well as a ferri-chiral phase transition, revealing spin splitting interdependent on chirality. Our electronic struc-ture calculations show that a few percent biaxial tensile strain can reduce (or nearly quench) the switching barrier separating the monodomain ferri-chiral P6322 states. We discuss what type of external stimuli might switch the chirality so as to reverse the (non-helical) Dresselhaus D-1B spin texture, offering an alternative to the traditional reversal of the (helical) Rashba spin texture.

cond-mat.mtrl-sci↗

arXiv2020

A Hybrid Evolutionary Algorithm for Reliable Facility Location Problem

The reliable facility location problem (RFLP) is an important research topic of operational research and plays a vital role in the decision-making and management of modern supply chain and logistics. Through solving RFLP, the decision-maker can obtain reliable location decisions under the risk of facilities' disruptions or failures. In this paper, we propose a novel model for the RFLP. Instead of assuming allocating a fixed number of facilities to each customer as in the existing works, we set the number of allocated facilities as an independent variable in our proposed model, which makes our model closer to the scenarios in real life but more difficult to be solved by traditional methods. To handle it, we propose EAMLS, a hybrid evolutionary algorithm, which combines a memorable local search (MLS) method and an evolutionary algorithm (EA). Additionally, a novel metric called l3-value is proposed to assist the analysis of the algorithm's convergence speed and exam the process of evolution. The experimental results show the effectiveness and superior performance of our EAMLS, compared to a CPLEX solver and a Genetic Algorithm (GA), on large-scale problems.

cs.NE↗

arXiv2020

Distilling Effective Supervision from Severe Label Noise

Collecting large-scale data with clean labels for supervised training of neural networks is practically challenging. Although noisy labels are usually cheap to acquire, existing methods suffer a lot from label noise. This paper targets at the challenge of robust training at high label noise regimes. The key insight to achieve this goal is to wisely leverage a small trusted set to estimate exemplar weights and pseudo labels for noisy data in order to reuse them for supervised training. We present a holistic framework to train deep neural networks in a way that is highly invulnerable to label noise. Our method sets the new state of the art on various types of label noise and achieves excellent performance on large-scale datasets with real-world label noise. For instance, on CIFAR100 with a $40\%$ uniform noise ratio and only 10 trusted labeled data per class, our method achieves $80.2{\pm}0.3\%$ classification accuracy, where the error rate is only $1.4\%$ higher than a neural network trained without label noise. Moreover, increasing the noise ratio to $80\%$, our method still maintains a high accuracy of $75.5{\pm}0.2\%$, compared to the previous best accuracy $48.2\%$. Source code available: https://github.com/google-research/google-research/tree/master/ieg

cs.LG↗

arXiv2020

ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation

Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks. To address this issue, we propose an enhanced multi-flow sequence to sequence pre-training and fine-tuning framework named ERNIE-GEN, which bridges the discrepancy between training and inference with an infilling generation mechanism and a noise-aware generation method. To make generation closer to human writing patterns, this framework introduces a span-by-span generation flow that trains the model to predict semantically-complete spans consecutively rather than predicting word by word. Unlike existing pre-training methods, ERNIE-GEN incorporates multi-granularity target sampling to construct pre-training data, which enhances the correlation between encoder and decoder. Experimental results demonstrate that ERNIE-GEN achieves state-of-the-art results with a much smaller amount of pre-training data and parameters on a range of language generation tasks, including abstractive summarization (Gigaword and CNN/DailyMail), question generation (SQuAD), dialogue generation (Persona-Chat) and generative question answering (CoQA).

cs.CL↗

arXiv2020

Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning

Nowadays e-commerce search has become an integral part of many people's shopping routines. Two critical challenges stay in today's e-commerce search: how to retrieve items that are semantically relevant but not exact matching to query terms, and how to retrieve items that are more personalized to different users for the same search query. In this paper, we present a novel approach called DPSR, which stands for Deep Personalized and Semantic Retrieval, to tackle this problem. Explicitly, we share our design decisions on how to architect a retrieval system so as to serve industry-scale traffic efficiently and how to train a model so as to learn query and item semantics accurately. Based on offline evaluations and online A/B test with live traffics, we show that DPSR model outperforms existing models, and DPSR system can retrieve more personalized and semantically relevant items to significantly improve users' search experience by +1.29% conversion rate, especially for long tail queries by +10.03%. As a result, our DPSR system has been successfully deployed into JD.com's search production since 2019.

cs.IR↗