arXiv subjects

Chao Zhang

Publications and source records attributed to Chao Zhang.

At least 865 records · Page 48Linked to original sources

arXiv2021

Ground states of 2D tilted dipolar bosons with density-induced hopping

Motivated by recent experiments with ultracold magnetic atoms trapped in optical lattices where the orientation of atomic dipoles can be fully controlled by external fields, we study the ground state properties of dipolar bosons trapped in a two-dimensional lattice with density-induced hopping and where the dipoles are tilted along the $xz$ plane. We present ground state phase diagrams of the above system at different tilt angles. We find that, as the dipolar interaction increases, the superfluid phase at half filling factor is destroyed in favor of either a checkerboard or stripe solid phase for tilt angle $θ\lesssim 35.3^{\circ}$ or $θ\gtrsim 35.3^{\circ}$ respectively. More interesting physics happens at tilt angles $θ\gtrsim 58^{\circ}$, where we find that, as the dipolar interaction strength increases, solid phases first appear at filling factor lower than $0.5$. Moreover, unlike what observed at lower tilt angles, we find that, at half filling, a stripe supersolid intervenes between the superfluid and stripe solid phase.

cond-mat.quant-gas↗

arXiv2021

A Universal End-to-End Approach to Portfolio Optimization via Deep Learning

We propose a universal end-to-end framework for portfolio optimization where asset distributions are directly obtained. The designed framework circumvents the traditional forecasting step and avoids the estimation of the covariance matrix, lifting the bottleneck for generalizing to a large amount of instruments. Our framework has the flexibility of optimizing various objective functions including Sharpe ratio, mean-variance trade-off etc. Further, we allow for short selling and study several constraints attached to objective functions. In particular, we consider cardinality, maximum position for individual instrument and leverage. These constraints are formulated into objective functions by utilizing several neural layers and gradient ascent can be adopted for optimization. To ensure the robustness of our framework, we test our methods on two datasets. Firstly, we look at a synthetic dataset where we demonstrate that weights obtained from our end-to-end approach are better than classical predictive methods. Secondly, we apply our framework on a real-life dataset with historical observations of hundreds of instruments with a testing period of more than 20 years.

q-fin.PM↗

arXiv2021

When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting

Accurate and trustworthy epidemic forecasting is an important problem that has impact on public health planning and disease mitigation. Most existing epidemic forecasting models disregard uncertainty quantification, resulting in mis-calibrated predictions. Recent works in deep neural models for uncertainty-aware time-series forecasting also have several limitations; e.g. it is difficult to specify meaningful priors in Bayesian NNs, while methods like deep ensembling are computationally expensive in practice. In this paper, we fill this important gap. We model the forecasting task as a probabilistic generative process and propose a functional neural process model called EPIFNP, which directly models the probability density of the forecast value. EPIFNP leverages a dynamic stochastic correlation graph to model the correlations between sequences in a non-parametric way, and designs different stochastic latent variables to capture functional uncertainty from different perspectives. Our extensive experiments in a real-time flu forecasting setting show that EPIFNP significantly outperforms previous state-of-the-art models in both accuracy and calibration metrics, up to 2.5x in accuracy and 2.4x in calibration. Additionally, due to properties of its generative process,EPIFNP learns the relations between the current season and similar patterns of historical seasons,enabling interpretable forecasts. Beyond epidemic forecasting, the EPIFNP can be of independent interest for advancing principled uncertainty quantification in deep sequential models for predictive analytics

cs.LG↗

arXiv2021

Approximating Optimal Transport via Low-rank and Sparse Factorization

Optimal transport (OT) naturally arises in a wide range of machine learning applications but may often become the computational bottleneck. Recently, one line of works propose to solve OT approximately by searching the \emph{transport plan} in a low-rank subspace. However, the optimal transport plan is often not low-rank, which tends to yield large approximation errors. For example, when Monge's \emph{transport map} exists, the transport plan is full rank. This paper concerns the computation of the OT distance with adequate accuracy and efficiency. A novel approximation for OT is proposed, in which the transport plan can be decomposed into the sum of a low-rank matrix and a sparse one. We theoretically analyze the approximation error. An augmented Lagrangian method is then designed to efficiently calculate the transport plan.

cs.LG↗

arXiv2021

Diversity-Promoting Human Motion Interpolation via Conditional Variational Auto-Encoder

In this paper, we present a deep generative model based method to generate diverse human motion interpolation results. We resort to the Conditional Variational Auto-Encoder (CVAE) to learn human motion conditioned on a pair of given start and end motions, by leveraging the Recurrent Neural Network (RNN) structure for both the encoder and the decoder. Additionally, we introduce a regularization loss to further promote sample diversity. Once trained, our method is able to generate multiple plausible coherent motions by repetitively sampling from the learned latent space. Experiments on the publicly available dataset demonstrate the effectiveness of our method, in terms of sample plausibility and diversity.

cs.CV↗

arXiv2021

GraSSNet: Graph Soft Sensing Neural Networks

In the era of big data, data-driven based classification has become an essential method in smart manufacturing to guide production and optimize inspection. The industrial data obtained in practice is usually time-series data collected by soft sensors, which are highly nonlinear, nonstationary, imbalanced, and noisy. Most existing soft-sensing machine learning models focus on capturing either intra-series temporal dependencies or pre-defined inter-series correlations, while ignoring the correlation between labels as each instance is associated with multiple labels simultaneously. In this paper, we propose a novel graph based soft-sensing neural network (GraSSNet) for multivariate time-series classification of noisy and highly-imbalanced soft-sensing data. The proposed GraSSNet is able to 1) capture the inter-series and intra-series dependencies jointly in the spectral domain; 2) exploit the label correlations by superimposing label graph that built from statistical co-occurrence information; 3) learn features with attention mechanism from both textual and numerical domain; and 4) leverage unlabeled data and mitigate data imbalance by semi-supervised learning. Comparative studies with other commonly used classifiers are carried out on Seagate soft sensing data, and the experimental results validate the competitive performance of our proposed method.

cs.LG↗

arXiv2021

Soft-Sensing ConFormer: A Curriculum Learning-based Convolutional Transformer

Over the last few decades, modern industrial processes have investigated several cost-effective methodologies to improve the productivity and yield of semiconductor manufacturing. While playing an essential role in facilitating real-time monitoring and control, the data-driven soft-sensors in industries have provided a competitive edge when augmented with deep learning approaches for wafer fault-diagnostics. Despite the success of deep learning methods across various domains, they tend to suffer from bad performance on multi-variate soft-sensing data domains. To mitigate this, we propose a soft-sensing ConFormer (CONvolutional transFORMER) for wafer fault-diagnostic classification task which primarily consists of multi-head convolution modules that reap the benefits of fast and light-weight operations of convolutions, and also the ability to learn the robust representations through multi-head design alike transformers. Another key issue is that traditional learning paradigms tend to suffer from low performance on noisy and highly-imbalanced soft-sensing data. To address this, we augment our soft-sensing ConFormer model with a curriculum learning-based loss function, which effectively learns easy samples in the early phase of training and difficult ones later. To further demonstrate the utility of our proposed architecture, we performed extensive experiments on various toolsets of Seagate Technology's wafer manufacturing process which are shared openly along with this work. To the best of our knowledge, this is the first time that curriculum learning-based soft-sensing ConFormer architecture has been proposed for soft-sensing data and our results show strong promise for future use in soft-sensing research domain.

cs.LG↗

arXiv2021

Soft Sensing Model Visualization: Fine-tuning Neural Network from What Model Learned

The growing availability of the data collected from smart manufacturing is changing the paradigms of production monitoring and control. The increasing complexity and content of the wafer manufacturing process in addition to the time-varying unexpected disturbances and uncertainties, make it infeasible to do the control process with model-based approaches. As a result, data-driven soft-sensing modeling has become more prevalent in wafer process diagnostics. Recently, deep learning has been utilized in soft sensing system with promising performance on highly nonlinear and dynamic time-series data. Despite its successes in soft-sensing systems, however, the underlying logic of the deep learning framework is hard to understand. In this paper, we propose a deep learning-based model for defective wafer detection using a highly imbalanced dataset. To understand how the proposed model works, the deep visualization approach is applied. Additionally, the model is then fine-tuned guided by the deep visualization. Extensive experiments are performed to validate the effectiveness of the proposed system. The results provide an interpretation of how the model works and an instructive fine-tuning method based on the interpretation.

cs.LG↗

arXiv2021

Trade or Trick? Detecting and Characterizing Scam Tokens on Uniswap Decentralized Exchange

The prosperity of the cryptocurrency ecosystem drives the need for digital asset trading platforms. Beyond centralized exchanges (CEXs), decentralized exchanges (DEXs) are introduced to allow users to trade cryptocurrency without transferring the custody of their digital assets to the middlemen, thus eliminating the security and privacy issues of traditional CEX. Uniswap, as the most prominent cryptocurrency DEX, is continuing to attract scammers, with fraudulent cryptocurrencies flooding in the ecosystem. In this paper, we take the first step to detect and characterize scam tokens on Uniswap. We first collect all the transactions related to Uniswap V2 exchange and investigate the landscape of cryptocurrency trading on Uniswap from different perspectives. Then, we propose an accurate approach for flagging scam tokens on Uniswap based on a guilt-by-association heuristic and a machine-learning powered technique. We have identified over 10K scam tokens listed on Uniswap, which suggests that roughly 50% of the tokens listed on Uniswap are scam tokens. All the scam tokens and liquidity pools are created specialized for the "rug pull" scams, and some scam tokens have embedded tricks and backdoors in the smart contracts. We further observe that thousands of collusion addresses help carry out the scams in league with the scam token/pool creators. The scammers have gained a profit of at least \$16 million from 39,762 potential victims. Our observations in this paper suggest the urgency to identify and stop scams in the decentralized finance ecosystem, and our approach can act as a whistleblower that identifies scam tokens at their early stages.

cs.CR↗

arXiv2021

Soft Sensing Transformer: Hundreds of Sensors are Worth a Single Word

With the rapid development of AI technology in recent years, there have been many studies with deep learning models in soft sensing area. However, the models have become more complex, yet, the data sets remain limited: researchers are fitting million-parameter models with hundreds of data samples, which is insufficient to exercise the effectiveness of their models and thus often fail to perform when implemented in industrial applications. To solve this long-lasting problem, we are providing large scale, high dimensional time series manufacturing sensor data from Seagate Technology to the public. We demonstrate the challenges and effectiveness of modeling industrial big data by a Soft Sensing Transformer model on these data sets. Transformer is used because, it has outperformed state-of-the-art techniques in Natural Language Processing, and since then has also performed well in the direct application to computer vision without introduction of image-specific inductive biases. We observe the similarity of a sentence structure to the sensor readings and process the multi-variable sensor readings in a time series in a similar manner of sentences in natural language. The high-dimensional time-series data is formatted into the same shape of embedded sentences and fed into the transformer model. The results show that transformer model outperforms the benchmark models in soft sensing field based on auto-encoder and long short-term memory (LSTM) models. To the best of our knowledge, we are the first team in academia or industry to benchmark the performance of original transformer model with large-scale numerical soft sensing data.

cs.LG↗

arXiv2021

Training Algorithm Matters for the Performance of Neural Network Potential: A Case Study of Adam and the Kalman Filter Optimizers

One hidden yet important issue for developing neural network potentials (NNPs) is the choice of training algorithm. Here we compare the performance of two popular training algorithms, the adaptive moment estimation algorithm (Adam) and the Extended Kalman Filter algorithm (EKF), using the Behler-Parrinello neural network (BPNN) and two publicly accessible datasets of liquid water [Proc. Natl. Acad. Sci. U.S.A. 2016, 113, 8368-8373 and Proc. Natl. Acad. Sci. U.S.A. 2019, 116, 1110-1115]. This is achieved by implementing EKF in TensorFlow. It is found that NNPs trained with EKF are more transferable and less sensitive to the value of the learning rate, as compared to Adam. In both cases, error metrics of the validation set do not always serve as a good indicator for the actual performance of NNPs. Instead, we show that their performance correlates well with a Fisher information based similarity measure.

physics.chem-ph↗

arXiv2021

HRFormer: High-Resolution Transformer for Dense Prediction

We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense prediction tasks, in contrast to the original Vision Transformer that produces low-resolution representations and has high memory and computational cost. We take advantage of the multi-resolution parallel design introduced in high-resolution convolutional networks (HRNet), along with local-window self-attention that performs self-attention over small non-overlapping image windows, for improving the memory and computation efficiency. In addition, we introduce a convolution into the FFN to exchange information across the disconnected image windows. We demonstrate the effectiveness of the High-Resolution Transformer on both human pose estimation and semantic segmentation tasks, e.g., HRFormer outperforms Swin transformer by $1.3$ AP on COCO pose estimation with $50\%$ fewer parameters and $30\%$ fewer FLOPs. Code is available at: https://github.com/HRNet/HRFormer.

cs.CV↗

arXiv2021

ATOMS: ALMA Three-millimeter Observations of Massive Star-forming regions -- V. Hierarchical fragmentation and gas dynamics in IRDC G034.43+00.24

We present new 3-mm continuum and molecular lines observations from the ATOMS survey towards the massive protostellar clump, MM1, located in the filamentary infrared dark cloud (IRDC), G034.43+00.24 (G34). The lines observed are the tracers of either dense gas (e.g. HCO+/H13CO+ J = 1-0) or outflows (e.g. CS J = 2-1). The most complete picture to date of seven cores in MM1 is revealed by dust continuum emission. These cores are found to be gravitationally bound, with virial parameter, $α_{vir}<2$. At least four outflows are identified in MM1 with a total outflowing mass of $\sim 45 M_\odot$, and a total energy of $\sim 1\times 10^{47}$ erg, typical of outflows from a B0-type star. Evidence of hierarchical fragmentation, where turbulence dominates over thermal pressure, is observed at both the cloud and the clump scales. This could be linked to the scale-dependent, dynamical mass inflow/accretion on clump and core scales. We therefore suggest that the G34 cloud could be undergoing a dynamical mass inflow/accretion process linked to the multiscale fragmentation, which leads to the sequential formation of fragments of the initial cloud, clumps, and ultimately dense cores, the sites of star formation.

astro-ph.GA↗

arXiv2021

Boundedness of differential transforms for heat semigroups generated by fractional Laplacian

In this paper we analyze the convergence of the following type of series \begin{equation*} T_N f(x)=\sum_{j=N_1}^{N_2} v_j\Big(e^{-a_{j+1}(-Δ)^α} f(x)-e^{-a_{j}(-Δ)^α} f(x)\Big),\quad x\in \mathbb R^n, \end{equation*} where $\{e^{-t(-Δ)^α} \}_{t>0}$ is the heat semigroup of the fractional Laplacian $(-Δ)^α,$ $N=(N_1, N_2)\in \mathbb Z^2$ with $N_1<N_2,$ $\{v_j\}_{j\in \mathbb Z}$ is a bounded real sequences and $\{a_j\}_{j\in \mathbb Z}$ is an increasing real sequence. Our analysis will consist in the boundedness, in $L^p(\mathbb{R}^n)$ and in $BMO(\mathbb{R}^n)$, of the operators $T_N$ and its maximal operator $\displaystyle T^*f(x)= \sup_N |T_N f(x)|.$ It is also shown that the local size of the maximal differential transform operators is the same with the order of a singular integral for functions $f$ having local support.

math.CA↗

arXiv2021

Lightweight Mobile Automated Assistant-to-physician for Global Lower-resource Areas

Importance: Lower-resource areas in Africa and Asia face a unique set of healthcare challenges: the dual high burden of communicable and non-communicable diseases; a paucity of highly trained primary healthcare providers in both rural and densely populated urban areas; and a lack of reliable, inexpensive internet connections. Objective: To address these challenges, we designed an artificial intelligence assistant to help primary healthcare providers in lower-resource areas document demographic and medical sign/symptom data and to record and share diagnostic data in real-time with a centralized database. Design: We trained our system using multiple data sets, including US-based electronic medical records (EMRs) and open-source medical literature and developed an adaptive, general medical assistant system based on machine learning algorithms. Main outcomes and Measure: The application collects basic information from patients and provides primary care providers with diagnoses and prescriptions suggestions. The application is unique from existing systems in that it covers a wide range of common diseases, signs, and medication typical in lower-resource countries; the application works with or without an active internet connection. Results: We have built and implemented an adaptive learning system that assists trained primary care professionals by means of an Android smartphone application, which interacts with a central database and collects real-time data. The application has been tested by dozens of primary care providers. Conclusions and Relevance: Our application would provide primary healthcare providers in lower-resource areas with a tool that enables faster and more accurate documentation of medical encounters. This application could be leveraged to automatically populate local or national EMR systems.

cs.LG↗

arXiv2021

Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization

Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards their transferability. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of EGI (Ego-Graph Information maximization) to analytically achieve this goal. Secondly, when node features are structure-relevant, we conduct an analysis of EGI transferability regarding the difference between the local graph Laplacians of the source and target graphs. We conduct controlled synthetic experiments to directly justify our theoretical conclusions. Comprehensive experiments on two real-world network datasets show consistent results in the analyzed setting of direct-transfering, while those on large-scale knowledge graphs show promising results in the more practical setting of transfering with fine-tuning.

cs.LG↗

arXiv2021

Unrestricted Adversarial Attacks on ImageNet Competition

Many works have investigated the adversarial attacks or defenses under the settings where a bounded and imperceptible perturbation can be added to the input. However in the real-world, the attacker does not need to comply with this restriction. In fact, more threats to the deep model come from unrestricted adversarial examples, that is, the attacker makes large and visible modifications on the image, which causes the model classifying mistakenly, but does not affect the normal observation in human perspective. Unrestricted adversarial attack is a popular and practical direction but has not been studied thoroughly. We organize this competition with the purpose of exploring more effective unrestricted adversarial attack algorithm, so as to accelerate the academical research on the model robustness under stronger unbounded attacks. The competition is held on the TianChi platform (\url{https://tianchi.aliyun.com/competition/entrance/531853/introduction}) as one of the series of AI Security Challengers Program.

cs.CV↗

arXiv2021

Parameter estimation for space-based gravitational wave detectors with ringdown signals

Unlike ground-based gravitational wave detectors, space-based gravitational wave detectors can detect the ringdown signals from massive black hole mergers with large signal-to-noise ratios, help to localize sources and extract their parameters. To reduce the computation time in the Fisher information matrix analysis, we derive the analytical formulas of frequency-domain ringdown signals for both heliocentric and geocentric detectors by considering the effects of the harmonic phases, the rotation period of the geocentric detector, and the detector's arm length. We explore median errors of the parameter estimation and source localization with ringdown signals from binaries with different masses and different redshifts. Using a binary source with the total mass $M=10^7\ M_\odot$ at the redshift $z=1$, we analyze the dependence of these errors on the sky position. We find that the network of space-based gravitational wave detectors can significantly improve the source localization at the ringdown stage. The results of the Fisher matrix approximation are also checked by Bayesian inference method.

gr-qc↗