arXiv subjects

Chang Xu

Publications and source records attributed to Chang Xu.

At least 289 records · Page 16Linked to original sources

arXiv2022

DyRep: Bootstrapping Training with Dynamic Re-parameterization

Structural re-parameterization (Rep) methods achieve noticeable improvements on simple VGG-style networks. Despite the prevalence, current Rep methods simply re-parameterize all operations into an augmented network, including those that rarely contribute to the model's performance. As such, the price to pay is an expensive computational overhead to manipulate these unnecessary behaviors. To eliminate the above caveats, we aim to bootstrap the training with minimal cost by devising a dynamic re-parameterization (DyRep) method, which encodes Rep technique into the training process that dynamically evolves the network structures. Concretely, our proposal adaptively finds the operations which contribute most to the loss in the network, and applies Rep to enhance their representational capacity. Besides, to suppress the noisy and redundant operations introduced by Rep, we devise a de-parameterization technique for a more compact re-parameterization. With this regard, DyRep is more efficient than Rep since it smoothly evolves the given network instead of constructing an over-parameterized network. Experimental results demonstrate our effectiveness, e.g., DyRep improves the accuracy of ResNet-18 by $2.04\%$ on ImageNet and reduces $22\%$ runtime over the baseline. Code is available at: https://github.com/hunto/DyRep.

cs.CV↗

arXiv2022

$q$-Supercongruences on triple and quadruple sums

Inspired by the recent work of El Bachraoui, we present some new $q$-supercongruences on triple and quadruple sums of basic hypergeometric series. In particular, we give a $q$-supercongruence modulo the fifth power of a cyclotomic polynomial, which is a $q$-analogue of the quadruple sum of Van Hamme's supercongruence (G.2).

math.NT↗

arXiv2022

SimMatch: Semi-supervised Learning with Similarity Matching

Learning with few labeled data has been a longstanding problem in the computer vision and machine learning research community. In this paper, we introduced a new semi-supervised learning framework, SimMatch, which simultaneously considers semantic similarity and instance similarity. In SimMatch, the consistency regularization will be applied on both semantic-level and instance-level. The different augmented views of the same instance are encouraged to have the same class prediction and similar similarity relationship respected to other instances. Next, we instantiated a labeled memory buffer to fully leverage the ground truth labels on instance-level and bridge the gaps between the semantic and instance similarities. Finally, we proposed the \textit{unfolding} and \textit{aggregation} operation which allows these two similarities be isomorphically transformed with each other. In this way, the semantic and instance pseudo-labels can be mutually propagated to generate more high-quality and reliable matching targets. Extensive experimental results demonstrate that SimMatch improves the performance of semi-supervised learning tasks across different benchmark datasets and different settings. Notably, with 400 epochs of training, SimMatch achieves 67.2\%, and 74.4\% Top-1 Accuracy with 1\% and 10\% labeled examples on ImageNet, which significantly outperforms the baseline methods and is better than previous semi-supervised learning frameworks. Code and pre-trained models are available at https://github.com/KyleZheng1997/simmatch.

cs.CV↗

arXiv2022

Some new results about $q$-trinomial coefficients

In this paper, we present several new congruences on the $q$-trinomial coefficients introduced by Andrews and Baxter. A new congruence on sums of central $q$-binomial coefficients is also established.

math.NT↗

arXiv2022

Relational Surrogate Loss Learning

Evaluation metrics in machine learning are often hardly taken as loss functions, as they could be non-differentiable and non-decomposable, e.g., average precision and F1 score. This paper aims to address this problem by revisiting the surrogate loss learning, where a deep neural network is employed to approximate the evaluation metrics. Instead of pursuing an exact recovery of the evaluation metric through a deep neural network, we are reminded of the purpose of the existence of these evaluation metrics, which is to distinguish whether one model is better or worse than another. In this paper, we show that directly maintaining the relation of models between surrogate losses and metrics suffices, and propose a rank correlation-based optimization method to maximize this relation and learn surrogate losses. Compared to previous works, our method is much easier to optimize and enjoys significant efficiency and performance gains. Extensive experiments show that our method achieves improvements on various tasks including image classification and neural machine translation, and even outperforms state-of-the-art methods on human pose estimation and machine reading comprehension tasks. Code is available at: https://github.com/hunto/ReLoss.

cs.LG↗

arXiv2022

New q-supercongruences from the Bailey transformation

Inspired by the recent work of Guo, we establish some new q-supercongruences including q-analogues of some Ramannujan-type supercongruences, by using the Bailey transformation formula and the `creative microscoping' method recently introduced by Guo and Zudilin.

math.NT↗

arXiv2022

GhostNets on Heterogeneous Devices via Cheap Operations

Deploying convolutional neural networks (CNNs) on mobile devices is difficult due to the limited memory and computation resources. We aim to design efficient neural networks for heterogeneous devices including CPU and GPU, by exploiting the redundancy in feature maps, which has rarely been investigated in neural architecture design. For CPU-like devices, we propose a novel CPU-efficient Ghost (C-Ghost) module to generate more feature maps from cheap operations. Based on a set of intrinsic feature maps, we apply a series of linear transformations with cheap cost to generate many ghost feature maps that could fully reveal information underlying intrinsic features. The proposed C-Ghost module can be taken as a plug-and-play component to upgrade existing convolutional neural networks. C-Ghost bottlenecks are designed to stack C-Ghost modules, and then the lightweight C-GhostNet can be easily established. We further consider the efficient networks for GPU devices. Without involving too many GPU-inefficient operations (e.g.,, depth-wise convolution) in a building stage, we propose to utilize the stage-wise feature redundancy to formulate GPU-efficient Ghost (G-Ghost) stage structure. The features in a stage are split into two parts where the first part is processed using the original block with fewer output channels for generating intrinsic features, and the other are generated using cheap operations by exploiting stage-wise redundancy. Experiments conducted on benchmarks demonstrate the effectiveness of the proposed C-Ghost module and the G-Ghost stage. C-GhostNet and G-GhostNet can achieve the optimal trade-off of accuracy and latency for CPU and GPU, respectively. Code is available at https://github.com/huawei-noah/CV-Backbones.

cs.CV↗

arXiv2022

Subtle Contact Nuances in the Delivery of Human-to-Human Touch Distinguish Emotional Sentiment

We routinely communicate distinct social and emotional sentiments through nuanced touch. For example, we might gently hold another's arm to offer a sense of calm, yet intensively hold another's arm to express excitement or anxiety. As this example indicates, distinct sentiments may be shaped by the subtlety in one's touch delivery. This work investigates how slight distinctions in skin-to-skin contact influence both the recognition of cued emotional messages (e.g., anger, sympathy) and the rating of emotional content (i.e., arousal, valence). By self-selecting preferred gestures (e.g., holding, stroking), touchers convey distinct messages by touching the receiver's forearm. Skin-to-skin contact attributes (e.g., velocity, depth, area) are optically tracked in high resolution. Contact is then examined within gesture, between messages. The results indicate touchers subtly, but significantly, vary contact attributes of a gesture to communicate distinct messages, which are recognizable by receivers. This tuning also correlates with receivers' arousal and valence. For instance, arousal increases with velocity for stroking, and depth for holding. Moreover, as shown here with human-to-human touch, valence is tied with velocity, which is the same trend as reported with brushes. The findings indicate that subtle nuance in skin-to-skin contact is important in conveying social messages and inducing emotions.

cs.HC↗

arXiv2021

An Empirical Study of Adder Neural Networks for Object Detection

Adder neural networks (AdderNets) have shown impressive performance on image classification with only addition operations, which are more energy efficient than traditional convolutional neural networks built with multiplications. Compared with classification, there is a strong demand on reducing the energy consumption of modern object detectors via AdderNets for real-world applications such as autonomous driving and face detection. In this paper, we present an empirical study of AdderNets for object detection. We first reveal that the batch normalization statistics in the pre-trained adder backbone should not be frozen, since the relatively large feature variance of AdderNets. Moreover, we insert more shortcut connections in the neck part and design a new feature fusion architecture for avoiding the sparse features of adder layers. We present extensive ablation studies to explore several design choices of adder detectors. Comparisons with state-of-the-arts are conducted on COCO and PASCAL VOC benchmarks. Specifically, the proposed Adder FCOS achieves a 37.8\% AP on the COCO val set, demonstrating comparable performance to that of the convolutional counterpart with an about $1.4\times$ energy reduction.

cs.CV↗

arXiv2021

Some q-supercongruences on double and triple sums

In this paper, we investigate a number of $q$-supercongruences on double and triple sums. By means of a lemma devised by El Bachraoui and its generalization, we transform some $q$-supercongruences on double and triple sums into the $q$-supercongruences of the square and cube of truncated basic hypergeometric series.Our main tools still involve the `creative microscoping' method introduced by Guo and Zudilin, a lemma designed by Guo and Li and the Chinese remainder theorem.

math.NT↗

arXiv2021

ViTAS: Vision Transformer Architecture Search

Vision transformers (ViTs) inherited the success of NLP but their structures have not been sufficiently investigated and optimized for visual tasks. One of the simplest solutions is to directly search the optimal one via the widely used neural architecture search (NAS) in CNNs. However, we empirically find this straightforward adaptation would encounter catastrophic failures and be frustratingly unstable for the training of superformer. In this paper, we argue that since ViTs mainly operate on token embeddings with little inductive bias, imbalance of channels for different architectures would worsen the weight-sharing assumption and cause the training instability as a result. Therefore, we develop a new cyclic weight-sharing mechanism for token embeddings of the ViTs, which enables each channel could more evenly contribute to all candidate architectures. Besides, we also propose identity shifting to alleviate the many-to-one issue in superformer and leverage weak augmentation and regularization techniques for more steady training empirically. Based on these, our proposed method, ViTAS, has achieved significant superiority in both DeiT- and Twins-based ViTs. For example, with only $1.4$G FLOPs budget, our searched architecture has $3.3\%$ ImageNet-$1$k accuracy than the baseline DeiT. With $3.0$G FLOPs, our results achieve $82.0\%$ accuracy on ImageNet-$1$k, and $45.9\%$ mAP on COCO$2017$ which is $2.4\%$ superior than other ViTs.

cs.CV↗

arXiv2021

Hire-MLP: Vision MLP via Hierarchical Rearrangement

Previous vision MLPs such as MLP-Mixer and ResMLP accept linearly flattened image patches as input, making them inflexible for different input sizes and hard to capture spatial information. Such approach withholds MLPs from getting comparable performance with their transformer-based counterparts and prevents them from becoming a general backbone for computer vision. This paper presents Hire-MLP, a simple yet competitive vision MLP architecture via \textbf{Hi}erarchical \textbf{re}arrangement, which contains two levels of rearrangements. Specifically, the inner-region rearrangement is proposed to capture local information inside a spatial region, and the cross-region rearrangement is proposed to enable information communication between different regions and capture global context by circularly shifting all tokens along spatial directions. Extensive experiments demonstrate the effectiveness of Hire-MLP as a versatile backbone for various vision tasks. In particular, Hire-MLP achieves competitive results on image classification, object detection and semantic segmentation tasks, e.g., 83.8% top-1 accuracy on ImageNet, 51.7% box AP and 44.8% mask AP on COCO val2017, and 49.9% mIoU on ADE20K, surpassing previous transformer-based and MLP-based models with better trade-off for accuracy and throughput. Code is available at https://github.com/ggjy/Hire-Wave-MLP.pytorch.

cs.CV↗

arXiv2021

Learning Frequency Domain Approximation for Binary Neural Networks

Binary neural networks (BNNs) represent original full-precision weights and activations into 1-bit with sign function. Since the gradient of the conventional sign function is almost zero everywhere which cannot be used for back-propagation, several attempts have been proposed to alleviate the optimization difficulty by using approximate gradient. However, those approximations corrupt the main direction of factual gradient. To this end, we propose to estimate the gradient of sign function in the Fourier frequency domain using the combination of sine functions for training BNNs, namely frequency domain approximation (FDA). The proposed approach does not affect the low-frequency information of the original sign function which occupies most of the overall energy, and high-frequency coefficients will be ignored to avoid the huge computational overhead. In addition, we embed a noise adaptation module into the training phase to compensate the approximation error. The experiments on several benchmark datasets and neural architectures illustrate that the binary network learned using our method achieves the state-of-the-art accuracy. Code will be available at \textit{https://gitee.com/mindspore/models/tree/master/research/cv/FDA-BNN}.

cs.CV↗

arXiv2021

Extraction of the second and fourth radial moments of nuclear charge density from the elastic electron-nucleus scattering

The expression for the radial moments $\left\langle r^{n}\right\rangle_{c}$ of the nuclear charge density has been discussed under the plane wave Born approximation (PWBA) method recently, which is significant to investigate the nuclear surface thickness and neutron distribution radius. In this paper, we extend the studies of extracting second-order moment $\left\langle r^{2}\right\rangle_{c}$ and fourth-order moment $\left\langle r^{4}\right\rangle_{c}$ from the Coulomb form factors $|F_{C}(q)|^2$ by the distorted wave Born approximation (DWBA) at the small momentum transfer $ q $ region. Based on the relativistic mean-field (RMF) calculations, the DWBA form factors $F_{C}^{DW}(q)$ are expanded into $ q^4 $, where the corresponding charge distributions are corrected by the contributions of neutron and spin-orbit densities. In the small $ q $ region, it is found that the experimental $|F_{C}(q)|^2$ can be well reproduced by considering the contributions of the $\left\langle r^{4}\right\rangle_{c}$ at the small $ q $ region. Through further analyzing the second-order and fourth-order expansion coefficients of the $F_{C}^{DW}(q)$, the relationship between the expansion coefficients and proton number $ Z $ is obtained. By the relationship, we extract the $\left\langle r^{2}\right\rangle_{c}$ and $\left\langle r^{4}\right\rangle_{c}$ from the limited experimental data of form factors at the small $ q $ region. Within the permissible range of error, the extracted $\left\langle r^{n}\right\rangle_{c}$ are consistent with the experimental data in this paper.

nucl-th↗

arXiv2021

Pre-Trained Image Processing Transformer

As the computing power of modern hardware is increasing strongly, pre-trained deep learning models (e.g., BERT, GPT-3) learned on large-scale datasets have shown their effectiveness over conventional methods. The big progress is mainly contributed to the representation ability of transformer and its variant architectures. In this paper, we study the low-level computer vision task (e.g., denoising, super-resolution and deraining) and develop a new pre-trained model, namely, image processing transformer (IPT). To maximally excavate the capability of transformer, we present to utilize the well-known ImageNet benchmark for generating a large amount of corrupted image pairs. The IPT model is trained on these images with multi-heads and multi-tails. In addition, the contrastive learning is introduced for well adapting to different image processing tasks. The pre-trained model can therefore efficiently employed on desired task after fine-tuning. With only one pre-trained model, IPT outperforms the current state-of-the-art methods on various low-level benchmarks. Code is available at https://github.com/huawei-noah/Pretrained-IPT and https://gitee.com/mindspore/mindspore/tree/master/model_zoo/research/cv/IPT

cs.CV↗

arXiv2021

The role of crosslinking density in surface stress and surface energy of soft solids

Surface stress and surface energy are two fundamental parameters that determine the surface properties of any materials. While it is commonly believed that the surface stress and surface energy of liquids are identical, the relationship between the two parameters in soft polymeric gels remains debatable. In this work, we measured the surface stress and surface energy of soft silicone gels with varying weight ratios of crosslinkers in soft wetting experiments. Above a critical density, $k_0$, the surface stress was found to increase significantly with crosslinking density while the surface energy remained unchanged. In this regime, we can estimate a non-zero surface elastic modulus that also increases with the ratio of crosslinkers. By comparing the surface mechanics of the soft gels with their bulk rheology, the surface properties near the critical density $k_0$ were found to be closely related to the underlying percolation transition of the polymer networks.

cond-mat.soft↗

arXiv2021

PP-PicoDet: A Better Real-Time Object Detector on Mobile Devices

The better accuracy and efficiency trade-off has been a challenging problem in object detection. In this work, we are dedicated to studying key optimizations and neural network architecture choices for object detection to improve accuracy and efficiency. We investigate the applicability of the anchor-free strategy on lightweight object detection models. We enhance the backbone structure and design the lightweight structure of the neck, which improves the feature extraction ability of the network. We improve label assignment strategy and loss function to make training more stable and efficient. Through these optimizations, we create a new family of real-time object detectors, named PP-PicoDet, which achieves superior performance on object detection for mobile devices. Our models achieve better trade-offs between accuracy and latency compared to other popular models. PicoDet-S with only 0.99M parameters achieves 30.6% mAP, which is an absolute 4.8% improvement in mAP while reducing mobile CPU inference latency by 55% compared to YOLOX-Nano, and is an absolute 7.1% improvement in mAP compared to NanoDet. It reaches 123 FPS (150 FPS using Paddle Lite) on mobile ARM CPU when the input size is 320. PicoDet-L with only 3.3M parameters achieves 40.9% mAP, which is an absolute 3.7% improvement in mAP and 44% faster than YOLOv5s. As shown in Figure 1, our models far outperform the state-of-the-art results for lightweight object detection. Code and pre-trained models are available at https://github.com/PaddlePaddle/PaddleDetection.

cs.CV↗

arXiv2021

Burst Eddy Current Testing with a Diamond Magnetometry

In this work, a burst eddy current testing technique based on the employment of a diamond nitrogen vacancy (NV) center magnetometer with the Hahn echo (HE) sequence is demonstrated. With the confocal experiment apparatus, the HE-based NV magnetometer attained a magnetic sensitivity of $4.3 ~ \mathrm{nT} / \sqrt{\mathrm{Hz}}$ and a volume-normalized sensitivity of $3.6 ~ \mathrm{pT} / \sqrt{\mathrm{Hz} \cdot \mathrm{mm}^{-3}}$, which are 5 times better than the already existing method under the same conditions. Based on the proposed magnetometer configuration, a burst eddy current (BEC) testing prototype achieves a minimum detectable sample smaller than ${300~μ \mathrm{m}}$ and measurement accuracy of $9.85~\mathrmμ \mathrm{m}$., which is employed to image different metallic specimens and detect the layered internal structures. Since our prototype comprises superb high sensitivity, it exhibits various potential applications in the fields of deformation monitoring, security screening, and quality control. Moreover, its biocompatibility and promising nanoscale resolution paves the way for electromagnetic testing in the fields of biomaterials.

physics.app-ph↗