arXiv subjects

Cong Liu

Publications and source records attributed to Cong Liu.

At least 181 records · Page 10Linked to original sources

arXiv2022

Deep Partial Updating: Towards Communication Efficient Updating for On-device Inference

Emerging edge intelligence applications require the server to retrain and update deep neural networks deployed on remote edge nodes to leverage newly collected data samples. Unfortunately, it may be impossible in practice to continuously send fully updated weights to these edge nodes due to the highly constrained communication resource. In this paper, we propose the weight-wise deep partial updating paradigm, which smartly selects a small subset of weights to update in each server-to-edge communication round, while achieving a similar performance compared to full updating. Our method is established through analytically upper-bounding the loss difference between partial updating and full updating, and only updates the weights which make the largest contributions to the upper bound. Extensive experimental results demonstrate the efficacy of our partial updating methodology which achieves a high inference accuracy while updating a rather small number of weights.

cs.LG↗

arXiv2022

Halo Properties and Mass Functions of Groups/Clusters from the DESI Legacy Imaging Surveys DR9

Based on a large group/cluster catalog recently constructed from the DESI Legacy Imaging Surveys DR9 using an extended halo-based group finder, we measure and model the group-galaxy weak lensing signals for groups/clusters in a few redshift bins within redshift range $0.1 \leqslant z<0.6$. Here, the background shear signals are obtained based on the DECaLS survey shape catalog derived with the \textsc{Fourier\_Quad} method. We divide the lens samples into 5 equispaced redshift bins and 7 mass bins, which allow us to probe the redshift and mass dependence of the lensing signals and hence the resulting halo properties. In addition to these sample selections, we have also checked the signals around different group centers, e.g., brightest central galaxy (BCG), luminosity weighted center and number weighted center. We use a lensing model that includes off-centering to describe the lensing signals we measure for all mass and redshift bins. The results demonstrate that our model predictions for the halo masses, bias and concentrations are stable and self-consistent among different samples for different group centers. Taking advantage of the very large and complete sample of groups/clusters, as well as the reliable estimation of their halo masses, we provide measurements of the cumulative halo mass functions up to redshift $z=0.6$, with a mass precision at $0.03\sim0.09$ dex.

astro-ph.CO↗

arXiv2022

Xronos: Predictable Coordination for Safety-Critical Distributed Embedded Systems

Asynchronous frameworks for distributed embedded systems, like ROS and MQTT, are increasingly used in safety-critical applications such as autonomous driving, where the cost of unintended behavior is high. The coordination mechanism between the components in these frameworks, however, gives rise to nondeterminism, where factors such as communication timing can lead to arbitrary ordering in the handling of messages. In this paper, we demonstrate the significance of this problem in an open-source full-stack autonomous driving software, Autoware.Auto 1.0, which relies on ROS 2. We give an alternative: Xronos, an open-source framework for distributed embedded systems that has a novel coordination strategy with predictable properties under clearly stated assumptions. If these assumptions are violated, Xronos provides for application-specific fault handlers to be invoked. We port Autoware.Auto to Xronos and show that it avoids the identified problems with manageable cost in end-to-end latency. Furthermore, we compare the maximum throughput of Xronos to ROS 2 and MQTT using microbenchmarks under different settings, including on three different hardware configurations, and find that it can match or exceed those frameworks in terms of throughput.

cs.DC↗

arXiv2022

Brief Industry Paper: The Necessity of Adaptive Data Fusion in Infrastructure-Augmented Autonomous Driving System

This paper is the first to provide a thorough system design overview along with the fusion methods selection criteria of a real-world cooperative autonomous driving system, named Infrastructure-Augmented Autonomous Driving or IAAD. We present an in-depth introduction of the IAAD hardware and software on both road-side and vehicle-side computing and communication platforms. We extensively characterize the IAAD system in the context of real-world deployment scenarios and observe that the network condition that fluctuates along the road is currently the main technical roadblock for cooperative autonomous driving. To address this challenge, we propose new fusion methods, dubbed "inter-frame fusion" and "planning fusion" to complement the current state-of-the-art "intra-frame fusion". We demonstrate that each fusion method has its own benefit and constraint.

cs.DC↗

arXiv2022

JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding

This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model~(PLM) for effectively understanding and representing mathematical problems. Unlike other standard NLP tasks, mathematical texts are difficult to understand, since they involve mathematical terminology, symbols and formulas in the problem statement. Typically, it requires complex mathematical logic and background knowledge for solving mathematical problems. Considering the complex nature of mathematical texts, we design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses. Specially, we first perform token-level pre-training based on a position-biased masking strategy, and then design logic-based pre-training tasks that aim to recover the shuffled sentences and formulas, respectively. Finally, we introduce a more difficult pre-training task that enforces the PLM to detect and correct the errors in its generated solutions. We conduct extensive experiments on offline evaluation (including nine math-related tasks) and online $A/B$ test. Experimental results demonstrate the effectiveness of our approach compared with a number of competitive baselines. Our code is available at: \textcolor{blue}{\url{https://github.com/RUCAIBox/JiuZhang}}.

cs.CL↗

arXiv2022

Shear Measurement with Poorly Resolved Images

Weak lensing studies typically require excellent seeing conditions for the purpose of maximizing the number density of well-resolved galaxy images. It is interesting to ask to what extent the seeing size limits the usefulness of the astronomical images in weak lensing. In this work, we study this issue with the data of the DECam Legacy Survey (DECaLS), which is a part of the target selection program for the Dark Energy Spectroscopic Instrument (DESI). Using the Fourier Quad shear measurement pipeline, we demonstrate that images with relatively poor seeing conditions (around 1.5 arcsec) can still yield accurate shear estimators. We do not find any correlation between systematic shear error and the image resolution.

astro-ph.CO↗

arXiv2022

Dynamic Group Transformer: A General Vision Transformer Backbone with Dynamic Group Attention

Recently, Transformers have shown promising performance in various vision tasks. To reduce the quadratic computation complexity caused by each query attending to all keys/values, various methods have constrained the range of attention within local regions, where each query only attends to keys/values within a hand-crafted window. However, these hand-crafted window partition mechanisms are data-agnostic and ignore their input content, so it is likely that one query maybe attends to irrelevant keys/values. To address this issue, we propose a Dynamic Group Attention (DG-Attention), which dynamically divides all queries into multiple groups and selects the most relevant keys/values for each group. Our DG-Attention can flexibly model more relevant dependencies without any spatial constraint that is used in hand-crafted window based attention. Built on the DG-Attention, we develop a general vision transformer backbone named Dynamic Group Transformer (DGT). Extensive experiments show that our models can outperform the state-of-the-art methods on multiple common vision tasks, including image classification, semantic segmentation, object detection, and instance segmentation.

cs.CV↗

arXiv2022

Resonance induced by higher-order coupling diversity

The studies of collective oscillations induced by higher-order interactions point out the necessity of group effect in coupling modelization. As yet the related advances are mainly concentrated on nonlinear coupling patterns and cannot be straightforwardly extended to the linear ones. In present work, we introduce the standard deviation of dynamic behavior for the interacting group to complement the higher-order effect that beyond pairwise in diffusive coupling. By doing so, the higher-order effect can be flexibly extended to the linearly coupled system. We leverage this modelization to embrace the influence of heterogeneous higher-order coupling, including promoting and inhibiting effects, on the signal response for two conventional models, the globally coupled overdamped bistable oscillators and excitable FitzHugh-Nagumo neurons. Particularly, we numerically and analytically reveal that the optimal signal response can be obtained by an intermediate degree of higher-order coupling diversity for both systems. This resonant signal response stems from the competition between dispersion and aggregation induced by heterogeneous higher-order and positive pairwise couplings, respectively. Our results contribute to a better understanding of the signal propagation in linearly coupled systems.

nlin.AO↗

arXiv2022

NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Models

Neural image caption generation (NICG) models have received massive attention from the research community due to their excellent performance in visual understanding. Existing work focuses on improving NICG model accuracy while efficiency is less explored. However, many real-world applications require real-time feedback, which highly relies on the efficiency of NICG models. Recent research observed that the efficiency of NICG models could vary for different inputs. This observation brings in a new attack surface of NICG models, i.e., An adversary might be able to slightly change inputs to cause the NICG models to consume more computational resources. To further understand such efficiency-oriented threats, we propose a new attack approach, NICGSlowDown, to evaluate the efficiency robustness of NICG models. Our experimental results show that NICGSlowDown can generate images with human-unnoticeable perturbations that will increase the NICG model latency up to 483.86%. We hope this research could raise the community's concern about the efficiency robustness of NICG models.

cs.CV↗

arXiv2022

Neural Mean Discrepancy for Efficient Out-of-Distribution Detection

Various approaches have been proposed for out-of-distribution (OOD) detection by augmenting models, input examples, training sets, and optimization objectives. Deviating from existing work, we have a simple hypothesis that standard off-the-shelf models may already contain sufficient information about the training set distribution which can be leveraged for reliable OOD detection. Our empirical study on validating this hypothesis, which measures the model activation's mean for OOD and in-distribution (ID) mini-batches, surprisingly finds that activation means of OOD mini-batches consistently deviate more from those of the training data. In addition, training data's activation means can be computed offline efficiently or retrieved from batch normalization layers as a 'free lunch'. Based upon this observation, we propose a novel metric called Neural Mean Discrepancy (NMD), which compares neural means of the input examples and training data. Leveraging the simplicity of NMD, we propose an efficient OOD detector that computes neural means by a standard forward pass followed by a lightweight classifier. Extensive experiments show that NMD outperforms state-of-the-art OOD approaches across multiple datasets and model architectures in terms of both detection accuracy and computational cost.

cs.LG↗

arXiv2022

AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysis

Deep neural networks (DNNs) are proved to be vulnerable against backdoor attacks. A backdoor is often embedded in the target DNNs through injecting a backdoor trigger into training examples, which can cause the target DNNs misclassify an input attached with the backdoor trigger. Existing backdoor detection methods often require the access to the original poisoned training data, the parameters of the target DNNs, or the predictive confidence for each given input, which are impractical in many real-world applications, e.g., on-device deployed DNNs. We address the black-box hard-label backdoor detection problem where the DNN is fully black-box and only its final output label is accessible. We approach this problem from the optimization perspective and show that the objective of backdoor detection is bounded by an adversarial objective. Further theoretical and empirical studies reveal that this adversarial objective leads to a solution with highly skewed distribution; a singularity is often observed in the adversarial map of a backdoor-infected example, which we call the adversarial singularity phenomenon. Based on this observation, we propose the adversarial extreme value analysis(AEVA) to detect backdoors in black-box neural networks. AEVA is based on an extreme value analysis of the adversarial map, computed from the monte-carlo gradient estimation. Evidenced by extensive experiments across multiple popular tasks and backdoor attacks, our approach is shown effective in detecting backdoor attacks under the black-box hard-label scenarios.

cs.LG↗

arXiv2022

EREBA: Black-box Energy Testing of Adaptive Neural Networks

Recently, various Deep Neural Network (DNN) models have been proposed for environments like embedded systems with stringent energy constraints. The fundamental problem of determining the robustness of a DNN with respect to its energy consumption (energy robustness) is relatively unexplored compared to accuracy-based robustness. This work investigates the energy robustness of Adaptive Neural Networks (AdNNs), a type of energy-saving DNNs proposed for many energy-sensitive domains and have recently gained traction. We propose EREBA, the first black-box testing method for determining the energy robustness of an AdNN. EREBA explores and infers the relationship between inputs and the energy consumption of AdNNs to generate energy surging samples. Extensive implementation and evaluation using three state-of-the-art AdNNs demonstrate that test inputs generated by EREBA could degrade the performance of the system substantially. The test inputs generated by EREBA can increase the energy consumption of AdNNs by 2,000% compared to the original inputs. Our results also show that test inputs generated via EREBA are valuable in detecting energy surging inputs.

cs.LG↗

arXiv2021

Natural language processing to identify lupus nephritis phenotype in electronic health records

Systemic lupus erythematosus (SLE) is a rare autoimmune disorder characterized by an unpredictable course of flares and remission with diverse manifestations. Lupus nephritis, one of the major disease manifestations of SLE for organ damage and mortality, is a key component of lupus classification criteria. Accurately identifying lupus nephritis in electronic health records (EHRs) would therefore benefit large cohort observational studies and clinical trials where characterization of the patient population is critical for recruitment, study design, and analysis. Lupus nephritis can be recognized through procedure codes and structured data, such as laboratory tests. However, other critical information documenting lupus nephritis, such as histologic reports from kidney biopsies and prior medical history narratives, require sophisticated text processing to mine information from pathology reports and clinical notes. In this study, we developed algorithms to identify lupus nephritis with and without natural language processing (NLP) using EHR data. We developed four algorithms: a rule-based algorithm using only structured data (baseline algorithm) and three algorithms using different NLP models. The three NLP models are based on regularized logistic regression and use different sets of features including positive mention of concept unique identifiers (CUIs), number of appearances of CUIs, and a mixture of three components respectively. The baseline algorithm and the best performed NLP algorithm were external validated on a dataset from Vanderbilt University Medical Center (VUMC). Our best performing NLP model incorporating features from both structured data, regular expression concepts, and mapped CUIs improved F measure in both the NMEDW (0.41 vs 0.79) and VUMC (0.62 vs 0.96) datasets compared to the baseline lupus nephritis algorithm.

cs.LG↗

arXiv2021

Invariant in variants

The coronavirus Covid-19 mutates quickly in the pandemic, leaves people struggling to verify and improve the effectiveness of the vaccine based on biochemistry. Is there any physical invariant in the variants of such kind of pathogen that could be taken advantage to ease the tensions? To this point, extensive numerical experiments based on continuity mechanics were carried out to discover the vibration modes and the range of natural frequency of coronavirus Covid-19. Such invariant could help us in developing some flexible technique to deactivate the coronavirus, like as resonantly breaking the viral spike by ultrasound wave. The fundamental mechanisms governing such process are demonstrated via solving the coupled equations of acoustics and dynamics and thereafter the technique strategies proposed to efficiently realize the concept.

physics.bio-ph↗

arXiv2021

Auto robust relative radiometric normalization via latent change noise modelling

Relative radiometric normalization(RRN) of different satellite images of the same terrain is necessary for change detection, object classification/segmentation, and map-making tasks. However, traditional RRN models are not robust, disturbing by object change, and RRN models precisely considering object change can not robustly obtain the no-change set. This paper proposes auto robust relative radiometric normalization methods via latent change noise modeling. They utilize the prior knowledge that no change points possess small-scale noise under relative radiometric normalization and that change points possess large-scale radiometric noise after radiometric normalization, combining the stochastic expectation maximization method to quickly and robustly extract the no-change set to learn the relative radiometric normalization mapping functions. This makes our model theoretically grounded regarding the probabilistic theory and mathematics deduction. Specifically, when we select histogram matching as the relative radiometric normalization learning scheme integrating with the mixture of Gaussian noise(HM-RRN-MoG), the HM-RRN-MoG model achieves the best performance. Our model possesses the ability to robustly against clouds/fogs/changes. Our method naturally generates a robust evaluation indicator for RRN that is the no-change set root mean square error. We apply the HM-RRN-MoG model to the latter vegetation/water change detection task, which reduces the radiometric contrast and NDVI/NDWI differences on the no-change set, generates consistent and comparable results. We utilize the no-change set into the building change detection task, efficiently reducing the pseudo-change and boosting the precision.

cs.CV↗

arXiv2021

Game-environment feedback dynamics for voluntary prisoner's dilemma games

Recently, the eco-evolutionary game theory which describes the coupled dynamics of strategies and environment have attracted great attention. At the same time, most of the current work is focused on the classic two-player two-strategy game. In this work, we study multi-strategy eco-evolutionary game theory which is an extension of the framework. For simplicity, we'll focus on the voluntary participation Prisoner's dilemma game. For the general class of payoff-dependent feedback dynamics, we show the conditions for the existence and stability of internal equilibrium by using the replicator dynamics, respectively. Where internal equilibrium points, such as, two-strategy coexistence states, three-strategy coexistence states, persistent oscillation states and interior saddle points. These states are determined by the relative feedback strength and payoff matrix, and are independent of the relative feedback speed and initial state. In particular, the three-strategy coexistence provides a new mechanism for maintaining biodiversity in biology, ecology, and sociology. Besides, we find that this three-strategy model return to the persistent oscillation state of the two-strategy model when there is no defective strategy at the initial moment.

physics.soc-ph↗

arXiv2021

High-performance Ba1-xKxFe2As2 superconducting joints for persistent current operation

Superconducting joints are one of the key technologies to make Ba1-xKxFe2As2 (Ba-122) superconducting wires or tapes for high-field applications. Herein, superconducting joints were fabricated by a simple cold-pressing method, and the joint resistance of the iron-based superconducting joint was estimated for the first time. The superconducting properties, microstructures, and elements distribution in the joint regions were investigated. At 4.2 K and 10 T, a transport critical current Ic of 105 A for the joint was obtained, and the critical current ratio (CCR= Ic-joint/Ic-tape) of the joint was 94.6%. On the other hand, the joint show very low joint resistance of 2.7x10^-13 ohm in self-field at 4.2 K. Among iron-based superconductors (IBS), this work is the first to successfully realize a superconducting joint with such high CCR and low joint resistance. This work shows great potential to apply Ba-122 in a range of practical applications, where superconducting joints are essential.

cond-mat.supr-con↗

arXiv2021

Point-wise posteriori phase estimation in high-precision fringe projection profilometry

In fringe projection profilometry, the high-order harmonics information of non-sinusoidal fringes will lead to errors in the phase estimation. In order to solve this problem, a point-wise posterior phase estimation (PWPPE) method based on deep learning technique is proposed in this paper. The complex nonlinear mapping relationship between the multiple gray values and the sine / cosine value of the phase is constructed by using the feedforward neural network model. After the model training, it can estimate the phase values of each pixel location, and the accuracy is higher than the point-wise least-square (PWLS) method. To further verify the effectiveness of this method, a face mask is measured, the traditional PWLS method and the proposed PWPPE method are employed, respectively. The comparison results show that the traditional method is with periodic phase errors, while the proposed PWPPE method can effectively eliminate such phase errors caused by non-sinusoidal fringes.

eess.IV↗