arXiv subjects

Kai Zhang

Publications and source records attributed to Kai Zhang.

At least 685 records · Page 38Linked to original sources

arXiv2016

Economic information from Smart Meter: Nexus Between Demand Profile and Electricity Retail Price Between Demand Profile and Electricity Retail Price

In this paper, we demonstrate that a consumer's marginal system impact is only determined by their demand profile rather than their demand level. Demand profile clustering is identical to cluster consumers according to their marginal impacts on system costs. A profile-based uniform-rate price is economically efficient as real-time pricing. We develop a criteria system to evaluate the economic efficiency of an implemented retail price scheme in a distribution system by comparing profile clustering and daily-average clustering. Our criteria system can examine the extent of a retail price scheme's inefficiency even without information about the distribution system's daily cost structure. We analyze data from a real distribution system in China. In this system, targeting each consumer's high-impact days is more efficient than target high-impact consumers.

econ.GN↗

arXiv2016

The quenching time scale and quenching rate of galaxies

The average star formation rate (SFR) in galaxies has been declining since redshift of 2. A fraction of galaxies quench and become quiescent. We constrain two key properties of the quenching process: the quenching time scale and the quenching rate among galaxies. We achieve this by analyzing the galaxy number density profile in NUV-u color space and the distribution in NUV-u v.s. u-i color-color diagram with a simple toy-model framework. We focus on galaxies in three mass bins between 10 to 10 and 10 to 10.6 solar mass. In the NUV-u v.s. u-i color-color diagram, the red u-i galaxies exhibit a different slope from the slope traced by the star-forming galaxies. This angled distribution and the number density profile of galaxies in NUV-u space strongly suggest that the decline of the SFR in galaxies has to accelerate before they turn quiescent. We model this color-color distribution with a two-phase exponential decline star formation history. The models with an e-folding time in the second phase (the quenching phase) of 0.5 Gyr best fit the data. We further use the NUV-u number density profile to constrain the quenching rate among star-forming galaxies as a function of mass. Adopting an e-folding time of 0.5 Gyr in the second phase (or the quenching phase), we found the quenching rate to be 19%/Gyr, 25%/Gyr and 33%/Gyr for the three mass bins. These are upper limits of quenching rate as the transition zone could also be populated by rejuvenated red-sequence galaxies.

astro-ph.GA↗

arXiv2016

Entity Embedding-based Anomaly Detection for Heterogeneous Categorical Events

Anomaly detection plays an important role in modern data-driven security applications, such as detecting suspicious access to a socket from a process. In many cases, such events can be described as a collection of categorical values that are considered as entities of different types, which we call heterogeneous categorical events. Due to the lack of intrinsic distance measures among entities, and the exponentially large event space, most existing work relies heavily on heuristics to calculate abnormal scores for events. Different from previous work, we propose a principled and unified probabilistic model APE (Anomaly detection via Probabilistic pairwise interaction and Entity embedding) that directly models the likelihood of events. In this model, we embed entities into a common latent space using their observed co-occurrence in different events. More specifically, we first model the compatibility of each pair of entities according to their embeddings. Then we utilize the weighted pairwise interactions of different entity types to define the event probability. Using Noise-Contrastive Estimation with "context-dependent" noise distribution, our model can be learned efficiently regardless of the large event space. Experimental results on real enterprise surveillance data show that our methods can accurately detect abnormal events compared to other state-of-the-art abnormal detection techniques.

cs.LG↗

arXiv2016

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Discriminative model learning for image denoising has been recently attracting considerable attentions due to its favorable denoising performance. In this paper, we take one step forward by investigating the construction of feed-forward denoising convolutional neural networks (DnCNNs) to embrace the progress in very deep architecture, learning algorithm, and regularization method into image denoising. Specifically, residual learning and batch normalization are utilized to speed up the training process as well as boost the denoising performance. Different from the existing discriminative denoising models which usually train a specific model for additive white Gaussian noise (AWGN) at a certain noise level, our DnCNN model is able to handle Gaussian denoising with unknown noise level (i.e., blind Gaussian denoising). With the residual learning strategy, DnCNN implicitly removes the latent clean image in the hidden layers. This property motivates us to train a single DnCNN model to tackle with several general image denoising tasks such as Gaussian denoising, single image super-resolution and JPEG image deblocking. Our extensive experiments demonstrate that our DnCNN model can not only exhibit high effectiveness in several general image denoising tasks, but also be efficiently implemented by benefiting from GPU computing.

cs.CV↗

arXiv2016

GID: Graph-based Intrusion Detection on Massive Process Traces for Enterprise Security Systems

Intrusion detection system (IDS) is an important part of enterprise security system architecture. In particular, anomaly-based IDS has been widely applied to detect abnormal process behaviors that deviate from the majority. However, such abnormal behavior usually consists of a series of low-level heterogeneous events. The gap between the low-level events and the high-level abnormal behaviors makes it hard to infer which single events are related to the real abnormal activities, especially considering that there are massive "noisy" low-level events happening in between. Hence, the existing work that focus on detecting single entities/events can hardly achieve high detection accuracy. Different from previous work, we design and implement GID, an efficient graph-based intrusion detection technique that can identify abnormal event sequences from a massive heterogeneous process traces with high accuracy. GID first builds a compact graph structure to capture the interactions between different system entities. The suspiciousness or anomaly score of process paths is then measured by leveraging random walk technique to the constructed acyclic directed graph. To eliminate the score bias from the path length, the Box-Cox power transformation based approach is introduced to normalize the anomaly scores so that the scores of paths of different lengths have the same distribution. The efficiency of suspicious path discovery is further improved by the proposed optimization scheme. We fully implement our GID algorithm and deploy it into a real enterprise security system, and it greatly helps detect the advanced threats, and optimize the incident response. Executing GID on system monitoring datasets showing that GID is efficient (about 2 million records per minute) and accurate (higher than 80% in terms of detection rate).

cs.CR↗

arXiv2016

The Infrared Imaging Spectrograph (IRIS) for TMT: Instrument Overview

IRIS is a near-infrared (0.84 to 2.4 microns) integral field spectrograph and wide-field imager being developed for first light with the Thirty Meter (TMT). It mounts to the advanced optics (AO) system NFIRAOS and has integrated on-instrument wavefront sensors (OIWFS) to achieve diffraction-limited spatial resolution at wavelengths longer than 1 micron. With moderate spectral resolution (R ~4,000 - 8,000) and large bandpass over a continuous field of view, IRIS will open new opportunities in virtually every area of astrophysical science. It will be able to resolve surface features tens of kilometers across Titan, while also mapping the distant galaxies at the scale of an individual star forming region. This paper summarizes the entire design and capabilities, and includes the results from the nearly completed preliminary design phase.

astro-ph.IM↗

arXiv2016

The InfraRed Imaging Spectrograph (IRIS) for TMT: latest science cases and simulations

The Thirty Meter Telescope (TMT) first light instrument IRIS (Infrared Imaging Spectrograph) will complete its preliminary design phase in 2016. The IRIS instrument design includes a near-infrared (0.85 - 2.4 micron) integral field spectrograph (IFS) and imager that are able to conduct simultaneous diffraction-limited observations behind the advanced adaptive optics system NFIRAOS. The IRIS science cases have continued to be developed and new science studies have been investigated to aid in technical performance and design requirements. In this development phase, the IRIS science team has paid particular attention to the selection of filters, gratings, sensitivities of the entire system, and science cases that will benefit from the parallel mode of the IFS and imaging camera. We present new science cases for IRIS using the latest end-to-end data simulator on the following topics: Solar System bodies, the Galactic center, active galactic nuclei (AGN), and distant gravitationally-lensed galaxies. We then briefly discuss the necessity of an advanced data management system and data reduction pipeline.

astro-ph.IM↗

arXiv2016

The Data Reduction Pipeline for the SDSS-IV MaNGA IFU Galaxy Survey

Mapping Nearby Galaxies at Apache Point Observatory (MaNGA) is an optical fiber-bundle integral-field unit (IFU) spectroscopic survey that is one of three core programs in the fourth-generation Sloan Digital Sky Survey (SDSS-IV). With a spectral coverage of 3622 - 10,354 Angstroms and an average footprint of ~ 500 arcsec^2 per IFU the scientific data products derived from MaNGA will permit exploration of the internal structure of a statistically large sample of 10,000 low redshift galaxies in unprecedented detail. Comprising 174 individually pluggable science and calibration IFUs with a near-constant data stream, MaNGA is expected to obtain ~ 100 million raw-frame spectra and ~ 10 million reduced galaxy spectra over the six-year lifetime of the survey. In this contribution, we describe the MaNGA Data Reduction Pipeline (DRP) algorithms and centralized metadata framework that produces sky-subtracted, spectrophotometrically calibrated spectra and rectified 3-D data cubes that combine individual dithered observations. For the 1390 galaxy data cubes released in Summer 2016 as part of SDSS-IV Data Release 13 (DR13), we demonstrate that the MaNGA data have nearly Poisson-limited sky subtraction shortward of ~ 8500 Angstroms and reach a typical 10-sigma limiting continuum surface brightness mu = 23.5 AB/arcsec^2 in a five arcsec diameter aperture in the g band. The wavelength calibration of the MaNGA data is accurate to 5 km/s rms, with a median spatial resolution of 2.54 arcsec FWHM (1.8 kpc at the median redshift of 0.037) and a median spectral resolution of sigma = 72 km/s.

astro-ph.IM↗

arXiv2016

SDSS-IV MaNGA IFS Galaxy Survey --- Survey Design, Execution, and Initial Data Quality

The MaNGA Survey (Mapping Nearby Galaxies at Apache Point Observatory) is one of three core programs in the Sloan Digital Sky Survey IV. It is obtaining integral field spectroscopy (IFS) for 10K nearby galaxies at a spectral resolution of R~2000 from 3,622-10,354A. The design of the survey is driven by a set of science requirements on the precision of estimates of the following properties: star formation rate surface density, gas metallicity, stellar population age, metallicity, and abundance ratio, and their gradients; stellar and gas kinematics; and enclosed gravitational mass as a function of radius. We describe how these science requirements set the depth of the observations and dictate sample selection. The majority of targeted galaxies are selected to ensure uniform spatial coverage in units of effective radius (Re) while maximizing spatial resolution. About 2/3 of the sample is covered out to 1.5Re (Primary sample), and 1/3 of the sample is covered to 2.5Re (Secondary sample). We describe the survey execution with details that would be useful in the design of similar future surveys. We also present statistics on the achieved data quality, specifically, the point spread function, sampling uniformity, spectral resolution, sky subtraction, and flux calibration. For our Primary sample, the median r-band signal-to-noise ratio is ~73 per 1.4A pixel for spectra stacked between 1-1.5 Re. Measurements of various galaxy properties from the first year data show that we are meeting or exceeding the defined requirements for the majority of our science goals.

astro-ph.GA↗

arXiv2016

Seeing the Forest from the Trees in Two Looks: Matrix Sketching by Cascaded Bilateral Sampling

Matrix sketching is aimed at finding close approximations of a matrix by factors of much smaller dimensions, which has important applications in optimization and machine learning. Given a matrix A of size m by n, state-of-the-art randomized algorithms take O(m * n) time and space to obtain its low-rank decomposition. Although quite useful, the need to store or manipulate the entire matrix makes it a computational bottleneck for truly large and dense inputs. Can we sketch an m-by-n matrix in O(m + n) cost by accessing only a small fraction of its rows and columns, without knowing anything about the remaining data? In this paper, we propose the cascaded bilateral sampling (CABS) framework to solve this problem. We start from demonstrating how the approximation quality of bilateral matrix sketching depends on the encoding powers of sampling. In particular, the sampled rows and columns should correspond to the code-vectors in the ground truth decompositions. Motivated by this analysis, we propose to first generate a pilot-sketch using simple random sampling, and then pursue more advanced, "follow-up" sampling on the pilot-sketch factors seeking maximal encoding powers. In this cascading process, the rise of approximation quality is shown to be lower-bounded by the improvement of encoding powers in the follow-up sampling step, thus theoretically guarantees the algorithmic boosting property. Computationally, our framework only takes linear time and space, and at the same time its performance rivals the quality of state-of-the-art algorithms consuming a quadratic amount of resources. Empirical evaluations on benchmark data fully demonstrate the potential of our methods in large scale matrix sketching and related areas.

cs.LG↗

arXiv2016

The effects of cooling rate on particle rearrangement statistics: Rapidly cooled glasses are more ductile and less reversible

Amorphous solids, such as metallic, polymeric, and colloidal glasses, display complex spatiotemporal response to applied deformations. In contrast to crystalline solids, during loading, amorphous solids exhibit a smooth crossover from elastic response to plastic flow. In this study, we investigate the mechanical response of binary Lennard-Jones glasses to athermal, quasistatic pure shear as a function of the cooling rate used to prepare them. We find several key results concerning the connection between strain-induced particle rearrangements and mechanical response. We show that more rapidly cooled glasses undergo more frequent and larger particle rearrangements than slowly cooled glasses. We find that the ratio of the shear to bulk moduli decreases with increasing cooling rate, which suggests that more rapidly cooled glasses are more ductile than slowly cooled samples. In addition, we characterized the degree of reversibility of particle motion during cyclic shear. We find that irreversible particle motion occurs even in the putative linear regime of stress versus strain. However, slowly cooled glasses, which undergo smaller rearrangements, are more reversible under cyclic shear than rapidly cooled glasses. Thus, we show that more ductile glasses are also less reversible.

cond-mat.soft↗

arXiv2016

On the Concept of Static Structure Factor

We clarify the confusion in the expression of the static structure factor S(k) in the study of condensed matters and discuss its explicit form that can be directly used in calculations and computer simulations.

cond-mat.soft↗

arXiv2016

Suppressing star formation in quiescent galaxies with supermassive black hole winds

Quiescent galaxies with little or no ongoing star formation dominate the galaxy population above $M_{*}\sim 2 \times 10^{10}~M_{\odot}$, where their numbers have increased by a factor of $\sim25$ since $z\sim2$. Once star formation is initially shut down, perhaps during the quasar phase of rapid accretion onto a supermassive black hole, an unknown mechanism must remove or heat subsequently accreted gas from stellar mass loss or mergers that would otherwise cool to form stars. Energy output from a black hole accreting at a low rate has been proposed, but observational evidence for this in the form of expanding hot gas shells is indirect and limited to radio galaxies at the centers of clusters, which are too rare to explain the vast majority of the quiescent population. Here we report bisymmetric emission features co-aligned with strong ionized gas velocity gradients from which we infer the presence of centrally-driven winds in typical quiescent galaxies that host low-luminosity active nuclei. These galaxies are surprisingly common, accounting for as much as $10\%$ of the population at $M_* \sim 2 \times 10^{10}~ M_{\odot}$. In a prototypical example, we calculate that the energy input from the galaxy's low-level active nucleus is capable of driving the observed wind, which contains sufficient mechanical energy to heat ambient, cooler gas (also detected) and thereby suppress star formation.

astro-ph.GA↗

arXiv2016

Distributed Flexible Nonlinear Tensor Factorization

Tensor factorization is a powerful tool to analyse multi-way data. Compared with traditional multi-linear methods, nonlinear tensor factorization models are capable of capturing more complex relationships in the data. However, they are computationally expensive and may suffer severe learning bias in case of extreme data sparsity. To overcome these limitations, in this paper we propose a distributed, flexible nonlinear tensor factorization model. Our model can effectively avoid the expensive computations and structural restrictions of the Kronecker-product in existing TGP formulations, allowing an arbitrary subset of tensorial entries to be selected to contribute to the training. At the same time, we derive a tractable and tight variational evidence lower bound (ELBO) that enables highly decoupled, parallel computations and high-quality inference. Based on the new bound, we develop a distributed inference algorithm in the MapReduce framework, which is key-value-free and can fully exploit the memory cache mechanism in fast MapReduce systems such as SPARK. Experimental results fully demonstrate the advantages of our method over several state-of-the-art approaches, in terms of both predictive performance and computational efficiency. Moreover, our approach shows a promising potential in the application of Click-Through-Rate (CTR) prediction for online advertising.

cs.LG↗

arXiv2016

Equilibrium Phase Behavior of a Continuous-Space Microphase Former

Periodic microphases universally emerge in systems for which short-range inter-particle attraction is frustrated by long-range repulsion. The morphological richness of these phases makes them desirable material targets, but our relatively coarse understanding of even simple models limits our grasp of their assembly. We report here the solution of the equilibrium phase behavior of a microscopic microphase former through specialized Monte Carlo simulations. The results for the ordered regime qualitatively agree with a Landau-type free energy description and highlight the nontrivial dynamical interplay between micelle, gel and microphase formation.

cond-mat.soft↗

arXiv2016

A testing-based approach to the discovery of differentially correlated variable sets

Given data obtained under two sampling conditions, it is often of interest to identify variables that behave differently in one condition than in the other. We introduce a method for differential analysis of second-order behavior called Differential Correlation Mining (DCM). The DCM method identifies differentially correlated sets of variables, with the property that the average pairwise correlation between variables in a set is higher under one sample condition than the other. DCM is based on an iterative search procedure that adaptively updates the size and elements of a candidate variable set. Updates are performed via hypothesis testing of individual variables, based on the asymptotic distribution of their average differential correlation. We investigate the performance of DCM by applying it to simulated data as well as recent experimental datasets in genomics and brain imaging.

stat.ME↗

arXiv2015

Generalized derivations of $3$-Lie algebras

Generalized derivations, quasiderivations and quasicentroid of $3$-algebras are introduced, and basic relations between them are studied. Structures of quasiderivations and quasicentroid of $3$-Lie algebras, which contains a maximal diagonalized tours, are systematically investigated. The main results are: for all $3$-Lie algebra $A$, 1) the generalized derivation algebra $GDer(A)$ is the sum of quasiderivation algebra $QDer(A)$ and quasicentroid $QΓ(A)$; 2) quasiderivations of $A$ can be embedded as derivations in a larger algebra; 3) quasiderivation algebra $QDer(A)$ normalizer quasicentroid, that is, $[QDer(A), QΓ(A)]\subseteq QΓ(A)$; 4) if $A$ contains a maximal diagonalized tours $T$, then $QDer(A)$ and $QΓ(A)$ are diagonalized $T$-modules, that is, as $T$-modules, $(T, T)$ semi-simplely acts on $QDer(A)$ and $QΓ(A)$, respectively.

math.RA↗

arXiv2015

Weak antilocalization and electron-electron interaction in coupled multiple-channel transport in a Bi$_2$Se$_3$ thin film

Electron transport properties of a topological insulator Bi$_2$Se$_3$ thin film are studied in Hall-bar geometry. The film with a thickness of 10 nm is grown by van der Waals epitaxy on fluorophlogopite mica and Hall-bar devices are fabricated from the as-grown film directly on the mica substrate. Weak antilocalization and electron-electron interaction effects are observed and analyzed at low temperatures. The phase-coherence length extracted from the measured weak antilocalization characteristics shows a strong power-law increase with decreasing temperature and the transport in the film is shown to occur via coupled multiple (topological surface and bulk states) channels. The conductivity of the film shows a logarithmically decrease with decreasing temperature and thus the electron-electron interaction plays a dominant role in quantum corrections to the conductivity of the film at low temperatures.

cond-mat.mes-hall↗