arXiv subjects

Xiang Li

Publications and source records attributed to Xiang Li.

At least 1,369 records · Page 76Linked to original sources

arXiv2018

Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift

This paper first answers the question "why do the two most powerful techniques Dropout and Batch Normalization (BN) often lead to a worse performance when they are combined together?" in both theoretical and statistical aspects. Theoretically, we find that Dropout would shift the variance of a specific neural unit when we transfer the state of that network from train to test. However, BN would maintain its statistical variance, which is accumulated from the entire learning procedure, in the test phase. The inconsistency of that variance (we name this scheme as "variance shift") causes the unstable numerical behavior in inference that leads to more erroneous predictions finally, when applying Dropout before BN. Thorough experiments on DenseNet, ResNet, ResNeXt and Wide ResNet confirm our findings. According to the uncovered mechanism, we next explore several strategies that modifies Dropout and try to overcome the limitations of their combination by avoiding the variance shift risks.

cs.LG↗

arXiv2018

GRB 111005A at Z = 0.0133 and the Prospect of Establishing Long-short GRB/GW Association

GRB 111005A, one long duration gamma-ray burst (GRB) occurred within a metal-rich environment that lacks massive stars with $M_{\rm ZAMS}\geq 15M_\odot$, is not coincident with supernova emission down to stringent limit and thus should be classified as a "long-short" GRB (lsGRB; also known as SN-less long GRB or hybrid GRB), like GRB 060505 and GRB 060614. In this work we show that in the neutron star merger model, the non-detection of the optical/infrared emission of GRB 111005A requires a sub-relativistic neutron-rich ejecta with the mass of $\leq 0.01~M_\odot$, (significantly) less massive than that of GRB 130603B, GRB 060614 and GRB 050709. The lsGRBs are found to have a high rate density and the neutron star merger origin model can be unambiguously tested by the joint observations of the second generation gravitational wave (GW) detectors and the full-sky gamma-ray monitors such as Fermi-GBM and the proposing GECAM. If no lsGRB/GW association is observed in 2020s, alternative scenarios have to be systematically investigated. With the detailed environmental information achievable for the very-nearby events, a novel kind of merger or explosion origin may be identified.

astro-ph.HE↗

arXiv2017

Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal

Understanding shadows from a single image spontaneously derives into two types of task in previous studies, containing shadow detection and shadow removal. In this paper, we present a multi-task perspective, which is not embraced by any existing work, to jointly learn both detection and removal in an end-to-end fashion that aims at enjoying the mutually improved benefits from each other. Our framework is based on a novel STacked Conditional Generative Adversarial Network (ST-CGAN), which is composed of two stacked CGANs, each with a generator and a discriminator. Specifically, a shadow image is fed into the first generator which produces a shadow detection mask. That shadow image, concatenated with its predicted mask, goes through the second generator in order to recover its shadow-free image consequently. In addition, the two corresponding discriminators are very likely to model higher level relationships and global scene characteristics for the detected shadow region and reconstruction via removing shadows, respectively. More importantly, for multi-task learning, our design of stacked paradigm provides a novel view which is notably different from the commonly used one as the multi-branch version. To fully evaluate the performance of our proposed framework, we construct the first large-scale benchmark with 1870 image triplets (shadow image, shadow mask image, and shadow-free image) under 135 scenes. Extensive experimental results consistently show the advantages of ST-CGAN over several representative state-of-the-art methods on two large-scale publicly available datasets and our newly released one.

cs.CV↗

arXiv2017

Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders

Unsupervised Image-to-Image Translation achieves spectacularly advanced developments nowadays. However, recent approaches mainly focus on one model with two domains, which may face heavy burdens with large cost of $O(n^2)$ training time and model parameters, under such a requirement that $n$ domains are freely transferred to each other in a general setting. To address this problem, we propose a novel and unified framework named Domain-Bank, which consists of a global shared auto-encoder and $n$ domain-specific encoders/decoders, assuming that a universal shared-latent sapce can be projected. Thus, we yield $O(n)$ complexity in model parameters along with a huge reduction of the time budgets. Besides the high efficiency, we show the comparable (or even better) image translation results over state-of-the-arts on various challenging unsupervised image translation tasks, including face image translation, fashion-clothes translation and painting style translation. We also apply the proposed framework to domain adaptation and achieve state-of-the-art performance on digit benchmark datasets. Further, thanks to the explicit representation of the domain-specific decoders as well as the universal shared-latent space, it also enables us to conduct incremental learning to add a new domain encoder/decoder. Linear combination of different domains' representations is also obtained by fusing the corresponding decoders.

cs.CV↗

arXiv2017

GW170817/GRB 170817A/AT2017gfo association: some implications for physics and astrophysics

On 17 August 2017, a gravitational wave event (GW170817) and an associated short gamma-ray burst (GRB 170817A) from a binary neutron star merger had been detected. The followup optical/infrared observations also identified the macronova/kilonova emission (AT2017gfo). In this work we discuss some implications of the remarkable GW170817/GRB 170817A/AT2017gfo association. We show that the $\sim 1.7$s time delay between the gravitational wave (GW) and GRB signals imposes very tight constraint on the superluminal movement of gravitational waves (i.e., the relative departure of GW velocity from the speed of light is $\leq 4.3\times 10^{-16}$) or the possible violation of weak equivalence principle (i.e., the difference of the gamma-ray and GW trajectories in the gravitational field of the galaxy and the local universe should be within a factor of $\sim 3.4\times 10^{-9}$). The so-called Dark Matter Emulators and a class of contender models for cosmic acceleration ("Covariant Galileon") are ruled out, too. The successful identification of Lanthanide elements in the macronova/kilonova spectrum also excludes the possibility that the progenitors of GRB 170817A are a binary strange star system. The high neutron star merger rate (inferred from both the local sGRB data and the gravitational wave data) together with the significant ejected mass strongly suggest that such mergers are the prime sites of heavy r-process nucleosynthesis.

astro-ph.HE↗

arXiv2017

Interpretations of the DAMPE electron data

The DArk Matter Particle Explorer (DAMPE), a high energy cosmic ray and $γ$-ray detector in space, has recently reported the new measurement of the total electron plus positron flux between 25 GeV and 4.6 TeV. A spectral softening at $\sim0.9$ TeV and a tentative peak at $\sim1.4$ TeV have been reported. We study the physical implications of the DAMPE data in this work. The presence of the spectral break significantly tightens the constraints on the model parameters to explain the electron/positron excesses. The spectral softening can either be explained by the maximum acceleration limits of electrons by astrophysical sources, or a breakdown of the common assumption of continuous distribution of electron sources at TeV energies in space and time. The tentive peak at $\sim1.4$ TeV implies local sources of electrons/positrons with quasi-monochromatic injection spectrum. We find that the cold, ultra-relativistic $e^+e^-$ winds from pulsars may give rise to such a structure. The pulsar is requird to be middle-aged, relatively slowly-rotated, mildly magnetized, and isolated in a density cavity. The annihilation of DM particles ($m_χ\sim1.5$ TeV) into $e^+e^-$ pairs in a nearby clump or an over-density region may also explain the data. In the DM scenario, the inferred clump mass (or density enhancement) is about $10^7-10^8$ M$_\odot$ (or $17-35$ times of the canonical local density) assuming a thermal production cross section, which is relatively extreme compared with the expectation from numerical simulations. A moderate enhancement of the annihilation cross section via, e.g., the Sommerfeld mechanism or non-thermal production, is thus needed.

astro-ph.HE↗

arXiv2017

A parametrized compactness theorem under bounded Ricci curvature

We prove a parametrized compactness theorem on manifolds of bounded Ricci curvature, upper bounded diameter and lower bounded injectivity radius.

math.DG↗

arXiv2017

The stable property of Newton slopes for general Witt towers

Any polynomial $f(x)\in\mathbb{Z}_q[x]$ defines a Witt vector $[f]\in W(\mathbb{F}_q[x])$. Consider the Artin-Schreier-Witt tower $y^F-y=[f]$. This is a tower of curves over $\mathbb{F}_q$, with total Galois group $\mathbb{Z}_p$. We want to study the Newton slopes of zeta functions of this tower. We reduce it to the Newton polygons of L-functions associated with characters on the Galois groups. We prove that, when the conductors are large enough, these Newton slopes are unions of arithmetic progressions which are changing proportionally as the conductor increases. This is a generalization of the result of \cite{Da}, where they get the same result in the case the non-zero coefficients of $f(x)$ are roots of unity. To overcome the new difficulty in our process, we apply some $(p^θ,T)$-topology.

math.NT↗

arXiv2017

Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes

Image analysis using more than one modality (i.e. multi-modal) has been increasingly applied in the field of biomedical imaging. One of the challenges in performing the multimodal analysis is that there exist multiple schemes for fusing the information from different modalities, where such schemes are application-dependent and lack a unified framework to guide their designs. In this work we firstly propose a conceptual architecture for the image fusion schemes in supervised biomedical image analysis: fusing at the feature level, fusing at the classifier level, and fusing at the decision-making level. Further, motivated by the recent success in applying deep learning for natural image analysis, we implement the three image fusion schemes above based on the Convolutional Neural Network (CNN) with varied structures, and combined into a single framework. The proposed image segmentation framework is capable of analyzing the multi-modality images using different fusing schemes simultaneously. The framework is applied to detect the presence of soft tissue sarcoma from the combination of Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and Positron Emission Tomography (PET) images. It is found from the results that while all the fusion schemes outperform the single-modality schemes, fusing at the feature level can generally achieve the best performance in terms of both accuracy and computational cost, but also suffers from the decreased robustness in the presence of large errors in any image modalities.

cs.CV↗

arXiv2017

Image Segmentation and Classification for Sickle Cell Disease using Deformable U-Net

Reliable cell segmentation and classification from biomedical images is a crucial step for both scientific research and clinical practice. A major challenge for more robust segmentation and classification methods is the large variations in the size, shape and viewpoint of the cells, combining with the low image quality caused by noise and artifacts. To address this issue, in this work we propose a learning-based, simultaneous cell segmentation and classification method based on the deep U-Net structure with deformable convolution layers. The U-Net architecture for deep learning has been shown to offer a precise localization for image semantic segmentation. Moreover, deformable convolution layer enables the free form deformation of the feature learning process, thus makes the whole network more robust to various cell morphologies and image settings. The proposed method is tested on microscopic red blood cell images from patients with sickle cell disease. The results show that U-Net with deformable convolution achieves the highest accuracy for segmentation and classification, comparing with original U-Net structure.

q-bio.CB↗

arXiv2017

On better training the infinite restricted Boltzmann machines

The infinite restricted Boltzmann machine (iRBM) is an extension of the classic RBM. It enjoys a good property of automatically deciding the size of the hidden layer according to specific training data. With sufficient training, the iRBM can achieve a competitive performance with that of the classic RBM. However, the convergence of learning the iRBM is slow, due to the fact that the iRBM is sensitive to the ordering of its hidden units, the learned filters change slowly from the left-most hidden unit to right. To break this dependency between neighboring hidden units and speed up the convergence of training, a novel training strategy is proposed. The key idea of the proposed training strategy is randomly regrouping the hidden units before each gradient descent step. Potentially, a mixing of infinite many iRBMs with different permutations of the hidden units can be achieved by this learning method, which has a similar effect of preventing the model from over-fitting as the dropout. The original iRBM is also modified to be capable of carrying out discriminative training. To evaluate the impact of our method on convergence speed of learning and the model's generalization ability, several experiments have been performed on the binarized MNIST and CalTech101 Silhouettes datasets. Experimental results indicate that the proposed training strategy can greatly accelerate learning and enhance generalization ability of iRBMs.

cs.LG↗

arXiv2017

Can Early Joining Participants Contribute More? - Timeliness Sensitive Incentivization for Crowdsensing

This paper investigates the incentive mechanism design from a novel and practically important perspective in which mobile users as contributors do not join simultaneously and a requester desires large efforts from early contributors. A two-stage Tullock contest framework is constructed:at the second stage the potential contributors compete for splittable reward by exerting efforts, and at the first stage the requester can orchestrate the incentive mechanism to maximize his crowdsensing efficiency given the rewarding budget. A general reward discrimination mechanism is developed for timeliness sensitive crowdsensing where an earlier contributor usually has a larger maximum achievable reward and thus allocates more efforts. Owning to the lack of joining time information, two practical implementations, namely earliest-n and termination time, are announced to the contributors. For each of them, we formulate a Stackelberg Bayesian game in which the joining time of a contributor is his type and not available to his opponents. The uniqueness of Bayesian Nash equilibrium (BNE) is proved in each strategy. To maximize the requester's efficiency, we compute the optimal number of rewarded contributors in the earliest-n scheme and the optimal deadline in the termination time scheme. Our contest framework is applicable not only to the closed crowdsensing with fixed number of contributors, but also to the open crowdsensing that the arrival of contributors is governed by a stochastic process. Extensive simulations manifest that with appropriate reward discriminations, the requester is able to achieve a much higher efficiency with the optimal selection of the number of rewarded contributiors and the termination time.

cs.GT↗

arXiv2017

Scaling of Dzyaloshinskii Moriya interaction at heavy metal and ferromagnetic metal interfaces

The Dzyaloshinskii Moriya Interaction (DMI) at the heavy metal (HM) and ferromagnetic metal (FM) interface has been recognized as a key ingredient in spintronic applications. Here we investigate the chemical trend of DMI on the 5d band filling (5d^3~5d^10) of the HM element in HM/CoFeB/MgO multilayer thin films. DMI is quantitatively evaluated by measuring asymmetric spin wave dispersion using Brillouin light scattering. Sign reversal and 20 times modification of the DMI coefficient D have been measured as the 5d HM element is varied. The chemical trend can be qualitatively understood by considering the 5d and 3d bands alignment at the HM/FM interface and the subsequent orbital hybridization around the Fermi level. Furthermore, a positive correlation is observed between DMI and spin mixing conductance at the HM/FM interfaces. Our results provide new insights into the interfacial DMI for designing future spintronic devices.

cond-mat.mtrl-sci↗

arXiv2017

Distributed rank-1 dictionary learning: Towards fast and scalable solutions for fMRI big data analytics

The use of functional brain imaging for research and diagnosis has benefitted greatly from the recent advancements in neuroimaging technologies, as well as the explosive growth in size and availability of fMRI data. While it has been shown in literature that using multiple and large scale fMRI datasets can improve reproducibility and lead to new discoveries, the computational and informatics systems supporting the analysis and visualization of such fMRI big data are extremely limited and largely under-discussed. We propose to address these shortcomings in this work, based on previous success in using dictionary learning method for functional network decomposition studies on fMRI data. We presented a distributed dictionary learning framework based on rank-1 matrix decomposition with sparseness constraint (D-r1DL framework). The framework was implemented using the Spark distributed computing engine and deployed on three different processing units: an in-house server, in-house high performance clusters, and the Amazon Elastic Compute Cloud (EC2) service. The whole analysis pipeline was integrated with our neuroinformatics system for data management, user input/output, and real-time visualization. Performance and accuracy of D-r1DL on both individual and group-wise fMRI Human Connectome Project (HCP) dataset shows that the proposed framework is highly scalable. The resulting group-wise functional network decompositions are highly accurate, and the fast processing time confirm this claim. In addition, D-r1DL can provide real-time user feedback and results visualization which are vital for large-scale data analysis.

cs.DS↗

arXiv2017

Improved Representation Learning for Predicting Commonsense Ontologies

Recent work in learning ontologies (hierarchical and partially-ordered structures) has leveraged the intrinsic geometry of spaces of learned representations to make predictions that automatically obey complex structural constraints. We explore two extensions of one such model, the order-embedding model for hierarchical relation learning, with an aim towards improved performance on text data for commonsense knowledge representation. Our first model jointly learns ordering relations and non-hierarchical knowledge in the form of raw text. Our second extension exploits the partial order structure of the training data to find long-distance triplet constraints among embeddings which are poorly enforced by the pairwise training procedure. We find that both incorporating free text and augmented training constraints improve over the original order-embedding model and other strong baselines.

cs.CL↗

arXiv2017

Developing Knowledge-enhanced Chronic Disease Risk Prediction Models from Regional EHR Repositories

Precision medicine requires the precision disease risk prediction models. In literature, there have been a lot well-established (inter-)national risk models, but when applying them into the local population, the prediction performance becomes unsatisfactory. To address the localization issue, this paper exploits the way to develop knowledge-enhanced localized risk models. On the one hand, we tune models by learning from regional Electronic Health Record (EHR) repositories, and on the other hand, we propose knowledge injection into the EHR data learning process. For experiments, we leverage the Pooled Cohort Equations (PCE, as recommended in ACC/AHA guidelines to estimate the risk of ASCVD) to develop a localized ASCVD risk prediction model in diabetes. The experimental results show that, if directly using the PCE algorithm on our cohort, the AUC is only 0.653, while our knowledge-enhanced localized risk model can achieve higher prediction performance with AUC of 0.723 (improved by 10.7%).

cs.AI↗

arXiv2017

Dictionary Learning and Sparse Coding-based Denoising for High-Resolution Task Functional Connectivity MRI Analysis

We propose a novel denoising framework for task functional Magnetic Resonance Imaging (tfMRI) data to delineate the high-resolution spatial pattern of the brain functional connectivity via dictionary learning and sparse coding (DLSC). In order to address the limitations of the unsupervised DLSC-based fMRI studies, we utilize the prior knowledge of task paradigm in the learning step to train a data-driven dictionary and to model the sparse representation. We apply the proposed DLSC-based method to Human Connectome Project (HCP) motor tfMRI dataset. Studies on the functional connectivity of cerebrocerebellar circuits in somatomotor networks show that the DLSC-based denoising framework can significantly improve the prominent connectivity patterns, in comparison to the temporal non-local means (tNLM)-based denoising method as well as the case without denoising, which is consistent and neuroscientifically meaningful within motor area. The promising results show that the proposed method can provide an important foundation for the high-resolution functional connectivity analysis, and provide a better approach for fMRI preprocessing.

cs.LG↗