Search arXiv⌕ Search

arXiv subjects

Meng Li

Publications and source records attributed to Meng Li.

At least 361 records · Page 20Linked to original sources
arXiv2020

Multi-branch and Multi-scale Attention Learning for Fine-Grained Visual Categorization

ImageNet Large Scale Visual Recognition Challenge (ILSVRC) is one of the most authoritative academic competitions in the field of Computer Vision (CV) in recent years. But applying ILSVRC's annual champion directly to fine-grained visual categorization (FGVC) tasks does not achieve good performance. To FGVC tasks, the small inter-class variations and the large intra-class variations make it a challenging problem. Our attention object location module (AOLM) can predict the position of the object and attention part proposal module (APPM) can propose informative part regions without the need of bounding-box or part annotations. The obtained object images not only contain almost the entire structure of the object, but also contains more details, part images have many different scales and more fine-grained features, and the raw images contain the complete object. The three kinds of training images are supervised by our multi-branch network. Therefore, our multi-branch and multi-scale learning network(MMAL-Net) has good classification ability and robustness for images of different scales. Our approach can be trained end-to-end, while provides short inference time. Through the comprehensive experiments demonstrate that our approach can achieves state-of-the-art results on CUB-200-2011, FGVC-Aircraft and Stanford Cars datasets. Our code will be available at https://github.com/ZF1044404254/MMAL-Net

cs.CV↗
arXiv2020

Unsaturated Single Atoms on Monolayer Transition Metal Dichalcogenides for Ultrafast Hydrogen Evolution

Large scale implementation of electrochemical water splitting for hydrogen evolution requires cheap and efficient catalysts to replace expensive platinum. Molybdenum disulfide is one of the most promising alternative catalysts but its intrinsic activity is still inferior to platinum. There is therefore a need to explore new active site origins in molybdenum disulfide with ultrafast reaction kinetics and to understand their mechanisms. Here, we report a universal cold hydrogen plasma reduction method for synthesizing different single atoms sitting on two-dimensional monolayers. In case of molybdenum disulfide, we design and identify a new type of active site, i.e., unsaturated Mo single atoms on cogenetic monolayer molybdenum disulfide. The catalyst shows exceptional intrinsic activity with a Tafel slope of 35.1 mV dec-1 and a turnover frequency of ~10^3 s-1 at 100 mV, based on single flake microcell measurements. Theoretical studies indicate that coordinately unsaturated Mo single atoms sitting on molybdenum disulfide increase the bond strength between adsorbed hydrogen atoms and the substrates through hybridization, leading to fast hydrogen adsorption/desorption kinetics and superior hydrogen evolution activity. This work shines fresh light on preparing highly-efficient electrocatalysts for water splitting and other electrochemical processes, as well as provides a general method to synthesize single atoms on two-dimensional monolayers.

cond-mat.mtrl-sci↗
arXiv2020

Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent

Designing energy-efficient networks is of critical importance for enabling state-of-the-art deep learning in mobile and edge settings where the computation and energy budgets are highly limited. Recently, Liu et al. (2019) framed the search of efficient neural architectures into a continuous splitting process: it iteratively splits existing neurons into multiple off-springs to achieve progressive loss minimization, thus finding novel architectures by gradually growing the neural network. However, this method was not specifically tailored for designing energy-efficient networks, and is computationally expensive on large-scale benchmarks. In this work, we substantially improve Liu et al. (2019) in two significant ways: 1) we incorporate the energy cost of splitting different neurons to better guide the splitting process, thereby discovering more energy-efficient network architectures; 2) we substantially speed up the splitting process of Liu et al. (2019), which requires expensive eigen-decomposition, by proposing a highly scalable Rayleigh-quotient stochastic gradient algorithm. Our fast algorithm allows us to reduce the computational cost of splitting to the same level of typical back-propagation updates and enables efficient implementation on GPU. Extensive empirical results show that our method can train highly accurate and energy-efficient networks on challenging datasets such as ImageNet, improving a variety of baselines, including the pruning-based methods and expert-designed architectures.

cs.LG↗
arXiv2020

Single-pixel coherent diffraction imaging

Complex-field imaging is indispensable for numerous applications at wavelengths from X-ray to THz, with amplitude describing transmittance (or reflectivity) and phase revealing intrinsic structure of the target object. Coherent diffraction imaging (CDI) employs iterative phase retrieval algorithms to process diffraction measurements and is the predominant non-interferometric method to image complex fields. However, the working spectrum of CDI is quite narrow, because the diffraction measurements on which it relies require dense array detection with ultra-high dynamic range. Here we report a single-pixel CDI technique that works for a wide waveband. A single-pixel detector instead of an array sensor is employed in the far field for detection. It repeatedly records the DC-only component of the diffracted wavefront scattered from an object as it is illuminated by a sequence of binary modulation patterns. This decreases the measurements' dynamic range by several orders of magnitude. We employ an efficient single-pixel phase-retrieval algorithm to jointly recover the object's 2D amplitude and phase maps from the 1D intensity-only measurements. No a priori object information is needed in the recovery process. We validate the technique's quantitative phase imaging nature using both calibrated phase objects and biological samples, and demonstrate its wide working spectrum with both 488-nm visible light and 980-nm near-infrared light. Our approach paves the way for complex-field imaging in a wider waveband where 2D detector arrays are not available, with broad applications in life and material sciences.

eess.IV↗
arXiv2020

SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks

Machine learning techniques have been widely applied in Internet companies for various tasks, acting as an essential driving force, and feature engineering has been generally recognized as a crucial tache when constructing machine learning systems. Recently, a growing effort has been made to the development of automatic feature engineering methods, so that the substantial and tedious manual effort can be liberated. However, for industrial tasks, the efficiency and scalability of these methods are still far from satisfactory. In this paper, we proposed a staged method named SAFE (Scalable Automatic Feature Engineering), which can provide excellent efficiency and scalability, along with requisite interpretability and promising performance. Extensive experiments are conducted and the results show that the proposed method can provide prominent efficiency and competitive effectiveness when comparing with other methods. What's more, the adequate scalability of the proposed method ensures it to be deployed in large scale industrial tasks.

cs.LG↗
arXiv2020

TimingCamouflage+: Netlist Security Enhancement with Unconventional Timing (with Appendix)

With recent advances in reverse engineering, attackers can reconstruct a netlist to counterfeit chips by opening the die and scanning all layers of authentic chips. This relatively easy counterfeiting is made possible by the use of the standard simple clocking scheme, where all combinational blocks function within one clock period, so that a netlist of combinational logic gates and flip-flops is sufficient to duplicate a design. In this paper, we propose to invalidate the assumption that a netlist completely represents the function of a circuit with unconventional timing. With the introduced wave-pipelining paths, attackers have to capture gate and interconnect delays during reverse engineering, or to test a huge number of combinational paths to identify the wave-pipelining paths. To hinder the test-based attack, we construct false paths with wave-pipelining to increase the counterfeiting challenge. Experimental results confirm that wave-pipelining true paths and false paths can be constructed in benchmark circuits successfully with only a negligible cost, thus thwarting the potential attack techniques.

cs.CR↗
arXiv2020

Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization

Neural network accelerator is a key enabler for the on-device AI inference, for which energy efficiency is an important metric. The data-path energy, including the computation energy and the data movement energy among the arithmetic units, claims a significant part of the total accelerator energy. By revisiting the basic physics of the arithmetic logic circuits, we show that the data-path energy is highly correlated with the bit flips when streaming the input operands into the arithmetic units, defined as the hamming distance of the input operand matrices. Based on the insight, we propose a post-training optimization algorithm and a hamming-distance-aware training algorithm to co-design and co-optimize the accelerator and the network synergistically. The experimental results based on post-layout simulation with MobileNetV2 demonstrate on average 2.85X data-path energy reduction and up to 8.51X data-path energy reduction for certain layers.

cs.CV↗
arXiv2020

Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks

Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs). However, it remains an open problem, how to integrate NAS with Application-Specific Integrated Circuits (ASICs), despite them being the most powerful AI accelerating platforms. The major bottleneck comes from the large design freedom associated with ASIC designs. Moreover, with the consideration that multiple DNNs will run in parallel for different workloads with diverse layer operations and sizes, integrating heterogeneous ASIC sub-accelerators for distinct DNNs in one design can significantly boost performance, and at the same time further complicate the design space. To address these challenges, in this paper we build ASIC template set based on existing successful designs, described by their unique dataflows, so that the design space is significantly reduced. Based on the templates, we further propose a framework, namely NASAIC, which can simultaneously identify multiple DNN architectures and the associated heterogeneous ASIC accelerator design, such that the design specifications (specs) can be satisfied, while the accuracy can be maximized. Experimental results show that compared with successive NAS and ASIC design optimizations which lead to design spec violations, NASAIC can guarantee the results to meet the design specs with 17.77%, 2.49x, and 2.32x reductions on latency, energy, and area and with 0.76% accuracy loss. To the best of the authors' knowledge, this is the first work on neural architecture and ASIC accelerator design co-exploration.

cs.LG↗
arXiv2020

Scalable Double Regularization for 3D Nano-CT Reconstruction

Nano-CT (computerized tomography) has emerged as a non-destructive high-resolution cross-sectional imaging technique to effectively study the sub-$μ$m pore structure of shale, which is of fundamental importance to the evaluation and development of shale oil and gas. Nano-CT poses unique challenges to the inverse problem of reconstructing the 3D structure due to the lower signal-to-noise ratio (than Micro-CT) at the nano-scale, increased sensitivity to the misaligned geometry caused by the movement of object manipulator, limited sample size, and a larger volume of data at higher resolution. In this paper, we propose a scalable double regularization (SDR) method to utilize the entire dataset for simultaneous 3D structural reconstruction across slices through total variation regularization within slices and $L_1$ regularization between adjacent slices. SDR allows information borrowing both within and between slices, contrasting with the traditional methods that usually build on slice by slice reconstruction. We develop a scalable and memory-efficient algorithm by exploiting the systematic sparsity and consistent geometry induced by such Nano-CT data. We illustrate the proposed method using synthetic data and two Nano-CT imaging datasets of Jiulaodong (JLD) shale and Longmaxi (LMX) shale acquired in the Sichuan Basin. These numerical experiments show that the proposed method substantially outperforms selected alternatives both visually and quantitatively.

eess.IV↗
arXiv2019

RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing

Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embedding operations with unique irregular memory access patterns that pose a fundamental challenge to accelerate. This paper proposes a lightweight, commodity DRAM compliant, near-memory processing solution to accelerate personalized recommendation inference. The in-depth characterization of production-grade recommendation models shows that embedding operations with high model-, operator- and data-level parallelism lead to memory bandwidth saturation, limiting recommendation inference performance. We propose RecNMP which provides a scalable solution to improve system throughput, supporting a broad range of sparse embedding models. RecNMP is specifically tailored to production environments with heavy co-location of operators on a single server. Several hardware/software co-optimization techniques such as memory-side caching, table-aware packet scheduling, and hot entry profiling are studied, resulting in up to 9.8x memory latency speedup over a highly-optimized baseline. Overall, RecNMP offers 4.2x throughput improvement and 45.8% memory energy savings.

cs.DC↗
arXiv2019

Aggregate Power Control of Heterogeneous TCL Populations Governed by Fokker-Planck Equations

This paper addresses the modeling and control of heterogeneous populations of thermostatically controlled loads (TCLs) operated by model predictive control (MPC) schemes at level of each TCL. It is shown that the dynamics of such TCLs populations can be described by a pair of Fokker-Planck equations coupled via the actions in the domain. The technique of input-output feedback linearization is used in the design of aggregate power control, which leads to a nonlinear system in closed loop. Well-posedness analysis is carried out to validate the developed control scheme. The closed-loop stability of the system is assessed by a rigorous analysis. A simulation study is contacted, and the obtained results confirm the validity and the effectiveness of the proposed approach.

math.OC↗
arXiv2019

Experimental realization of state transfer by quantum walks with two coins

Quantum state transfer between different sites is a significant problem for quantum networks and quantum computers. By selecting quantum walks with two coins as the basic model and two coin spaces as the communication carriers, we successfully implement quantum state transfer on various graphs (EPL, \textbf{124} (2018) 60009) \cite{Shang_2019}. Here, we demonstrate the experimental implementation of this scheme using IBM quantum experience platform. In particular, we show the transfer of Bell state, GHZ state and W state on complete graph on the quantum device. Also, we observe that our protocol has high fidelity by preforming quantum state tomography.

quant-ph↗
arXiv2019

PGU-net+: Progressive Growing of U-net+ for Automated Cervical Nuclei Segmentation

Automated cervical nucleus segmentation based on deep learning can effectively improve the quantitative analysis of cervical cancer. However, accurate nuclei segmentation is still challenging. The classic U-net has not achieved satisfactory results on this task, because it mixes the information of different scales that affect each other, which limits the segmentation accuracy of the model. To solve this problem, we propose a progressive growing U-net (PGU-net+) model, which uses two paradigms to extract image features at different scales in a more independent way. First, we add residual modules between different scales of U-net, which enforces the model to learn the approximate shape of the annotation in the coarser scale, and to learn the residual between the annotation and the approximate shape in the finer scale. Second, we start to train the model with the coarsest part and then progressively add finer part to the training until the full model is included. When we train a finer part, we will reduce the learning rate of the previous coarser part, which further ensures that the model independently extracts information from different scales. We conduct several comparative experiments on the Herlev dataset. The experimental results show that the PGU-net+ has superior accuracy than the previous state-of-the-art methods on cervical nuclei segmentation.

eess.IV↗
arXiv2019

Function-on-Scalar Quantile Regression with Application to Mass Spectrometry Proteomics Data

Mass spectrometry proteomics, characterized by spiky, spatially heterogeneous functional data, can be used to identify potential cancer biomarkers. Existing mass spectrometry analyses utilize mean regression to detect spectral regions that are differentially expressed across groups. However, given the inter-patient heterogeneity that is a key hallmark of cancer, many biomarkers are only present at aberrant levels for a subset of, not all, cancer samples. Differences in these biomarkers can easily be missed by mean regression, but might be more easily detected by quantile-based approaches. Thus, we propose a unified Bayesian framework to perform quantile regression on functional responses. Our approach utilizes an asymmetric Laplace working likelihood, represents the functional coefficients with basis representations which enable borrowing of strength from nearby locations, and places a global-local shrinkage prior on the basis coefficients to achieve adaptive regularization. Different types of basis transform and continuous shrinkage priors can be used in our framework. A scalable Gibbs sampler is developed to generate posterior samples that can be used to perform Bayesian estimation and inference while accounting for multiple testing. Our framework performs quantile regression and coefficient regularization in a unified manner, allowing them to inform each other and leading to improvement in performance over competing methods as demonstrated by simulation studies. We also introduce an adjustment procedure to the model to improve its frequentist properties of posterior inference. We apply our model to identify proteomic biomarkers of pancreatic cancer that are differentially expressed for a subset of cancer patients compared to the normal controls, which were missed by previous mean-regression based approaches. Supplementary materials for this article are available online.

stat.ME↗
arXiv2019

Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images

Automatic and accurate segmentation for retinal and choroidal layers of Optical Coherence Tomography (OCT) is crucial for detection of various ocular diseases. However, because of the variations in different equipments, OCT data obtained from different manufacturers might encounter appearance discrepancy, which could lead to performance fluctuation to a deep neural network. In this paper, we propose an uncertainty-guided domain alignment method to aim at alleviating this problem to transfer discriminative knowledge across distinct domains. We disign a novel uncertainty-guided cross-entropy loss for boosting the performance over areas with high uncertainty. An uncertainty-guided curriculum transfer strategy is developed for the self-training (ST), which regards uncertainty as efficient and effective guidance to optimize the learning process in target domain. Adversarial learning with feature recalibration module (FRM) is applied to transfer informative knowledge from the domain feature spaces adaptively. The experiments on two OCT datasets show that the proposed methods can obtain significant segmentation improvements compared with the baseline models.

cs.CV↗
arXiv2019

Deep Instance-Level Hard Negative Mining Model for Histopathology Images

Histopathology image analysis can be considered as a Multiple instance learning (MIL) problem, where the whole slide histopathology image (WSI) is regarded as a bag of instances (i.e, patches) and the task is to predict a single class label to the WSI. However, in many real-life applications such as computational pathology, discovering the key instances that trigger the bag label is of great interest because it provides reasons for the decision made by the system. In this paper, we propose a deep convolutional neural network (CNN) model that addresses the primary task of a bag classification on a WSI and also learns to identify the response of each instance to provide interpretable results to the final prediction. We incorporate the attention mechanism into the proposed model to operate the transformation of instances and learn attention weights to allow us to find key patches. To perform a balanced training, we introduce adaptive weighing in each training bag to explicitly adjust the weight distribution in order to concentrate more on the contribution of hard samples. Based on the learned attention weights, we further develop a solution to boost the classification performance by generating the bags with hard negative instances. We conduct extensive experiments on colon and breast cancer histopathology data and show that our framework achieves state-of-the-art performance.

cs.CV↗
arXiv2019

Limits of Sums for Binomial and Eulerian Numbers and their Associated Distributions

We provide a unified, probabilistic approach using renewal theory to derive some novel limits of sums for the normalized binomial coefficients and for the normalized Eulerian numbers. We also investigate some corresponding results for their associated distributions -- the binomial distributions for the binomial coefficients and the Irwin-Hall distributions (uniform B-splines) for the Eulerian numbers.

math.CO↗