arXiv subjects

Xinyu Zhang

Publications and source records attributed to Xinyu Zhang.

At least 433 records · Page 24Linked to original sources

arXiv2021

Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation

Visual object tracking aims to precisely estimate the bounding box for the given target, which is a challenging problem due to factors such as deformation and occlusion. Many recent trackers adopt the multiple-stage tracking strategy to improve the quality of bounding box estimation. These methods first coarsely locate the target and then refine the initial prediction in the following stages. However, existing approaches still suffer from limited precision, and the coupling of different stages severely restricts the method's transferability. This work proposes a novel, flexible, and accurate refinement module called Alpha-Refine (AR), which can significantly improve the base trackers' box estimation quality. By exploring a series of design options, we conclude that the key to successful refinement is extracting and maintaining detailed spatial information as much as possible. Following this principle, Alpha-Refine adopts a pixel-wise correlation, a corner prediction head, and an auxiliary mask head as the core components. Comprehensive experiments on TrackingNet, LaSOT, GOT-10K, and VOT2020 benchmarks with multiple base trackers show that our approach significantly improves the base trackers' performance with little extra latency. The proposed Alpha-Refine method leads to a series of strengthened trackers, among which the ARSiamRPN (AR strengthened SiamRPNpp) and the ARDiMP50 (ARstrengthened DiMP50) achieve good efficiency-precision trade-off, while the ARDiMPsuper (AR strengthened DiMP-super) achieves very competitive performance at a real-time speed. Code and pretrained models are available at https://github.com/MasterBin-IIAU/AlphaRefine.

cs.CV↗

arXiv2021

Non-Local Porous Media Equations with Fractional Time Derivative

In this paper we investigate existence of solutions for the system: \begin{equation*} \left\{ \begin{array}{l} D^α_tu=\textrm{div}(u \nabla p),\\ D^α_tp=-(-Δ)^{s}p+u^{2}, \end{array} \right. \end{equation*} in $\mathbb{T}^3$ for $0< s \leq 1$, and $0< α\le 1$. The term $D^α_t u$ denotes the Caputo derivative, which models memory effects in time. The fractional Laplacian $(-Δ)^{s}$ represents the Lévy diffusion. We prove global existence of nonnegative weak solutions that satisfy a variational inequality. The proof uses several approximations steps, including an implicit Euler time discretization. We show that the proposed discrete Caputo derivative satisfies several important properties, including positivity preserving, convexity and rigorous convergence towards the continuous Caputo derivative. Most importantly, we give a strong compactness criteria for piecewise constant functions, in the spirit of Aubin-Lions theorem, based on bounds of the discrete Caputo derivative.

math.AP↗

arXiv2021

Lite-FPN for Keypoint-based Monocular 3D Object Detection

3D object detection with a single image is an essential and challenging task for autonomous driving. Recently, keypoint-based monocular 3D object detection has made tremendous progress and achieved great speed-accuracy trade-off. However, there still exists a huge gap with LIDAR-based methods in terms of accuracy. To improve their performance without sacrificing efficiency, we propose a sort of lightweight feature pyramid network called Lite-FPN to achieve multi-scale feature fusion in an effective and efficient way, which can boost the multi-scale detection capability of keypoint-based detectors. Besides, the misalignment between classification score and localization precision is further relieved by introducing a novel regression loss named attention loss. With the proposed loss, predictions with high confidence but poor localization are treated with more attention during the training phase. Comparative experiments based on several state-of-the-art keypoint-based detectors on the KITTI dataset show that our proposed methods manage to achieve significant improvements in both accuracy and frame rate. The code and pretrained models will be released at \url{https://github.com/yanglei18/Lite-FPN}.

cs.CV↗

arXiv2021

IPS300+: a Challenging Multimodal Dataset for Intersection Perception System

Due to the high complexity and occlusion, insufficient perception in the crowded urban intersection can be a serious safety risk for both human drivers and autonomous algorithms, whereas CVIS (Cooperative Vehicle Infrastructure System) is a proposed solution for full-participants perception in this scenario. However, the research on roadside multimodal perception is still in its infancy, and there is no open-source dataset for such scenario. Accordingly, this paper fills the gap. Through an IPS (Intersection Perception System) installed at the diagonal of the intersection, this paper proposes a high-quality multimodal dataset for the intersection perception task. The center of the experimental intersection covers an area of 3000m2, and the extended distance reaches 300m, which is typical for CVIS. The first batch of open-source data includes 14198 frames, and each frame has an average of 319.84 labels, which is 9.6 times larger than the most crowded dataset (H3D dataset in 2019) by now. In order to facilitate further study, this dataset tries to keep the label documents consistent with the KITTI dataset, and a standardized benchmark is created for algorithm evaluation. Our dataset is available at: http://www.openmpd.com/column/other_datasets.

cs.CV↗

arXiv2021

Early Exiting with Ensemble Internal Classifiers

As a simple technique to accelerate inference of large-scale pre-trained models, early exiting has gained much attention in the NLP community. It allows samples to exit early at internal classifiers without passing through the entire model. Most existing work usually trains the internal classifiers independently and employs an exiting strategy to decide whether or not to exit based on the confidence of the current internal classifier. However, none of these works takes full advantage of the fact that the internal classifiers are trained to solve the same task therefore can be used to construct an ensemble. In this paper, we show that a novel objective function for the training of the ensemble internal classifiers can be naturally induced from the perspective of ensemble learning and information theory. The proposed training objective consists of two terms: one for accuracy and the other for the diversity of the internal classifiers. In contrast, the objective used in prior work is exactly the accuracy term of our training objective therefore only optimizes the accuracy but not diversity. Further, we propose a simple voting-based strategy that considers predictions of all the past internal classifiers to infer the correct label and decide whether to exit. Experimental results on various NLP tasks show that our proposed objective function and voting-based strategy can achieve better accuracy-speed trade-offs.

cs.CL↗

arXiv2021

Emotion Eliciting Machine: Emotion Eliciting Conversation Generation based on Dual Generator

Recent years have witnessed great progress on building emotional chatbots. Tremendous methods have been proposed for chatbots to generate responses with given emotions. However, the emotion changes of the user during the conversation has not been fully explored. In this work, we study the problem of positive emotion elicitation, which aims to generate responses that can elicit positive emotion of the user, in human-machine conversation. We propose a weakly supervised Emotion Eliciting Machine (EEM) to address this problem. Specifically, we first collect weak labels of user emotion status changes in a conversion based on a pre-trained emotion classifier. Then we propose a dual encoder-decoder structure to model the generation of responses in both positive and negative side based on the changes of the user's emotion status in the conversation. An emotion eliciting factor is introduced on top of the dual structure to balance the positive and negative emotional impacts on the generated response during emotion elicitation. The factor also provides a fine-grained controlling manner for emotion elicitation. Experimental results on a large real-world dataset show that EEM outperforms the existing models in generating responses with positive emotion elicitation.

cs.CL↗

arXiv2021

A novel multimodal fusion network based on a joint coding model for lane line segmentation

There has recently been growing interest in utilizing multimodal sensors to achieve robust lane line segmentation. In this paper, we introduce a novel multimodal fusion architecture from an information theory perspective, and demonstrate its practical utility using Light Detection and Ranging (LiDAR) camera fusion networks. In particular, we develop, for the first time, a multimodal fusion network as a joint coding model, where each single node, layer, and pipeline is represented as a channel. The forward propagation is thus equal to the information transmission in the channels. Then, we can qualitatively and quantitatively analyze the effect of different fusion approaches. We argue the optimal fusion architecture is related to the essential capacity and its allocation based on the source and channel. To test this multimodal fusion hypothesis, we progressively determine a series of multimodal models based on the proposed fusion methods and evaluate them on the KITTI and the A2D2 datasets. Our optimal fusion network achieves 85%+ lane line accuracy and 98.7%+ overall. The performance gap among the models will inform continuing future research into development of optimal fusion algorithms for the deep multimodal learning community.

cs.CV↗

arXiv2021

S-AT GCN: Spatial-Attention Graph Convolution Network based Feature Enhancement for 3D Object Detection

3D object detection plays a crucial role in environmental perception for autonomous vehicles, which is the prerequisite of decision and control. This paper analyses partition-based methods' inherent drawbacks. In the partition operation, a single instance such as a pedestrian is sliced into several pieces, which we call it the partition effect. We propose the Spatial-Attention Graph Convolution (S-AT GCN), forming the Feature Enhancement (FE) layers to overcome this drawback. The S-AT GCN utilizes the graph convolution and the spatial attention mechanism to extract local geometrical structure features. This allows the network to have more meaningful features for the foreground. Our experiments on the KITTI 3D object and bird's eye view detection show that S-AT Conv and FE layers are effective, especially for small objects. FE layers boost the pedestrian class performance by 3.62\% and cyclist class by 4.21\% 3D mAP. The time cost of these extra FE layers are limited. PointPillars with FE layers can achieve 48 PFS, satisfying the real-time requirement.

cs.CV↗

arXiv2021

DNA: Deeply-supervised Nonlinear Aggregation for Salient Object Detection

Recent progress on salient object detection mainly aims at exploiting how to effectively integrate multi-scale convolutional features in convolutional neural networks (CNNs). Many popular methods impose deep supervision to perform side-output predictions that are linearly aggregated for final saliency prediction. In this paper, we theoretically and experimentally demonstrate that linear aggregation of side-output predictions is suboptimal, and it only makes limited use of the side-output information obtained by deep supervision. To solve this problem, we propose Deeply-supervised Nonlinear Aggregation (DNA) for better leveraging the complementary information of various side-outputs. Compared with existing methods, it i) aggregates side-output features rather than predictions, and ii) adopts nonlinear instead of linear transformations. Experiments demonstrate that DNA can successfully break through the bottleneck of current linear approaches. Specifically, the proposed saliency detector, a modified U-Net architecture with DNA, performs favorably against state-of-the-art methods on various datasets and evaluation metrics without bells and whistles.

cs.CV↗

arXiv2021

A New Window Loss Function for Bone Fracture Detection and Localization in X-ray Images with Point-based Annotation

Object detection methods are widely adopted for computer-aided diagnosis using medical images. Anomalous findings are usually treated as objects that are described by bounding boxes. Yet, many pathological findings, e.g., bone fractures, cannot be clearly defined by bounding boxes, owing to considerable instance, shape and boundary ambiguities. This makes bounding box annotations, and their associated losses, highly ill-suited. In this work, we propose a new bone fracture detection method for X-ray images, based on a labor effective and flexible annotation scheme suitable for abnormal findings with no clear object-level spatial extents or boundaries. Our method employs a simple, intuitive, and informative point-based annotation protocol to mark localized pathology information. To address the uncertainty in the fracture scales annotated via point(s), we convert the annotations into pixel-wise supervision that uses lower and upper bounds with positive, negative, and uncertain regions. A novel Window Loss is subsequently proposed to only penalize the predictions outside of the uncertain regions. Our method has been extensively evaluated on 4410 pelvic X-ray images of unique patients. Experiments demonstrate that our method outperforms previous state-of-the-art image classification and object detection baselines by healthy margins, with an AUROC of 0.983 and FROC score of 89.6%.

cs.CV↗

arXiv2020

Diverse Knowledge Distillation for End-to-End Person Search

Person search aims to localize and identify a specific person from a gallery of images. Recent methods can be categorized into two groups, i.e., two-step and end-to-end approaches. The former views person search as two independent tasks and achieves dominant results using separately trained person detection and re-identification (Re-ID) models. The latter performs person search in an end-to-end fashion. Although the end-to-end approaches yield higher inference efficiency, they largely lag behind those two-step counterparts in terms of accuracy. In this paper, we argue that the gap between the two kinds of methods is mainly caused by the Re-ID sub-networks of end-to-end methods. To this end, we propose a simple yet strong end-to-end network with diverse knowledge distillation to break the bottleneck. We also design a spatial-invariant augmentation to assist model to be invariant to inaccurate detection results. Experimental results on the CUHK-SYSU and PRW datasets demonstrate the superiority of our method against existing approaches -- it achieves on par accuracy with state-of-the-art two-step methods while maintaining high efficiency due to the single joint model. Code is available at: https://git.io/DKD-PersonSearch.

cs.CV↗

arXiv2020

Peer-to-Peer Localization for Single-Antenna Devices

Some important indoor localization applications, such as localizing a lost kid in a shopping mall, call for a new peer-to-peer localization technique that can localize an individual's smartphone or wearables by directly using another's on-body devices in unknown indoor environments. However, current localization solutions either require pre-deployed infrastructures or multiple antennas in both transceivers, impending their wide-scale application. In this paper, we present P2PLocate, a peer-to-peer localization system that enables a single-antenna device co-located with a batteryless backscatter tag to localize another single-antenna device with decimeter-level accuracy. P2PLocate leverages the multipath variations intentionally created by an on-body backscatter tag, coupled with spatial information offered by user movements, to accomplish this objective without relying on any pre-deployed infrastructures or pre-training. P2PLocate incorporates novel algorithms to address two major challenges: (i) interference with strong direct-path signal while extracting multipath variations, and (ii) lack of direction information while using single-antenna transceivers. We implement P2PLocate on commercial off-the-shelf Google Nexus 6p, Intel 5300 WiFi card, and Raspberry Pi B4. Real-world experiments reveal that P2PLocate can localize both static and mobile targets with a median accuracy of 0.88 m.

eess.SP↗

arXiv2020

Part-Guided Attention Learning for Vehicle Instance Retrieval

Vehicle instance retrieval often requires one to recognize the fine-grained visual differences between vehicles. Besides the holistic appearance of vehicles which is easily affected by the viewpoint variation and distortion, vehicle parts also provide crucial cues to differentiate near-identical vehicles. Motivated by these observations, we introduce a Part-Guided Attention Network (PGAN) to pinpoint the prominent part regions and effectively combine the global and part information for discriminative feature learning. PGAN first detects the locations of different part components and salient regions regardless of the vehicle identity, which serve as the bottom-up attention to narrow down the possible searching regions. To estimate the importance of detected parts, we propose a Part Attention Module (PAM) to adaptively locate the most discriminative regions with high-attention weights and suppress the distraction of irrelevant parts with relatively low weights. The PAM is guided by the instance retrieval loss and therefore provides top-down attention that enables attention to be calculated at the level of car parts and other salient regions. Finally, we aggregate the global appearance and part features to improve the feature performance further. The PGAN combines part-guided bottom-up and top-down attention, global and part visual features in an end-to-end framework. Extensive experiments demonstrate that the proposed method achieves new state-of-the-art vehicle instance retrieval performance on four large-scale benchmark datasets.

cs.CV↗

arXiv2020

Electronic structure of pristine and Ni-substituted LaFeO$_3$ from near edge x-ray absorption fine structure experiments and first-principles simulations

We present a joint theoretical and experimental study of the oxygen $K$-edge spectra for LaFeO$_3$ and homovalent Ni-substituted LaFeO$_3$ (LaFe$_{0.75}$Ni$_{0.25}$O$_3$), using first-principles simulations based on density-functional theory with extended Hubbard functionals and x-ray absorption near edge structure (XANES) measurements. Ground-state and excited-state XANES calculations employ Hubbard on-site $U$ and inter-site $V$ parameters determined from first principles and the Lanczos recursive method to obtain absorption cross sections, which allows for a reliable description of XANES spectra in transition-metal compounds in a very broad energy range, with an accuracy comparable to that of hybrid functionals but at a substantially lower cost. We show that standard gradient-corrected exchange-correlation functionals fail in capturing accurately the electronic properties of both materials. In particular, for LaFe$_{0.75}$Ni$_{0.25}$O$_3$ they do not reproduce its semiconducting behaviour and provide a poor description of the pre-edge features at the O $K$ edge. The inclusion of Hubbard interactions leads to a drastic improvement, accounting for the semiconducting ground state of LaFe$_{0.75}$Ni$_{0.25}$O$_3$ and for a good agreement between calculated and measured XANES spectra. We show that the partial substitution of Fe for Ni affects the conduction-band bottom by generating a strongly hybridized O($2p$)-Ni($3d$) minority-spin empty electronic state. The present work, based on a consistent correction of self-interaction errors, outlines the crucial role of extended Hubbard functionals to describe the electronic structure of complex transition-metal oxides such as LaFeO$_3$ and LaFe$_{0.75}$Ni$_{0.25}$O$_3$ and paves the way to future studies on similar systems.

cond-mat.str-el↗

arXiv2020

Error Bounds for Generalized Group Sparsity

In high-dimensional statistical inference, sparsity regularizations have shown advantages in consistency and convergence rates for coefficient estimation. We consider a generalized version of Sparse-Group Lasso which captures both element-wise sparsity and group-wise sparsity simultaneously. We state one universal theorem which is proved to obtain results on consistency and convergence rates for different forms of double sparsity regularization. The universality of the results lies in an generalization of various convergence rates for single regularization cases such as LASSO and group LASSO and also double regularization cases such as sparse-group LASSO. Our analysis identifies a generalized norm of $ε$-norm, which provides a dual formulation for our double sparsity regularization.

stat.ML↗

arXiv2020

Towards Embodied Scene Description

Embodiment is an important characteristic for all intelligent agents (creatures and robots), while existing scene description tasks mainly focus on analyzing images passively and the semantic understanding of the scenario is separated from the interaction between the agent and the environment. In this work, we propose the Embodied Scene Description, which exploits the embodiment ability of the agent to find an optimal viewpoint in its environment for scene description tasks. A learning framework with the paradigms of imitation learning and reinforcement learning is established to teach the intelligent agent to generate corresponding sensorimotor activities. The proposed framework is tested on both the AI2Thor dataset and a real world robotic platform demonstrating the effectiveness and extendability of the developed method.

cs.RO↗

arXiv2020

Effective gravitational couplings of four-dimensional $\mathcal{N}=2$ supersymmetric gauge theories

The low energy effective couplings of a four-dimensional $\mathcal{N}=2$ supersymmetric gauge theory to topological invariants of the background gravitational field are described by two functions $A$ and $B$. These two functions play an important role in the study of topologically twisted four-dimensional $\mathcal{N}=2$ supersymmetric gauge theories and in the computation of central charges of $\mathcal{N}=2$ superconformal theories. In this paper, we compute $A$ and $B$ from the partition function in the $Ω$-background for $\mathrm{SU}(2)$ gauge theories. Our results not only confirm the predicted expressions of the effective gravitational couplings, but also give the previously undetermined overall multiplicative factors. We also analyze $A$ and $B$ for the $\mathrm{SU}(N)$ super-Yang-Mills theory, and confirm all the previous predictions.

hep-th↗

arXiv2020

Thermal conductivity and Lorenz ratio of metals at intermediate temperature: a first-principles analysis

Electronic and phononic thermal conductivity are involved in the thermal conduction for metals and Wiedemann-Franz law is usually employed to predict them separately. However, Wiedemann-Franz law is shown to be invalid at intermediate temperatures. Here, to obtain the accurate thermal conductivity and Lorenz ratio for metals, the momentum relaxation time is used for electrical conductivity and energy relaxation time for electronic thermal conductivity. The mode-level first-principles calculation is conducted on two representative metals copper and aluminum. It is shown that the method can correctly predict electrical transport coefficients from 6 to 300 K. Also, the anomalous Lorenz ratio is observed within the present scheme, which has significant departure from the Sommerfeld value. The calculation scheme can be expanded to other metallic systems and is valuable in a better understanding of the electron dynamics and transport properties of metals.

physics.app-ph↗