arXiv subjects

Shuo Yang

Publications and source records attributed to Shuo Yang.

At least 343 records · Page 19Linked to original sources

arXiv2022

Objects in Semantic Topology

A more realistic object detection paradigm, Open-World Object Detection, has arisen increasing research interests in the community recently. A qualified open-world object detector can not only identify objects of known categories, but also discover unknown objects, and incrementally learn to categorize them when their annotations progressively arrive. Previous works rely on independent modules to recognize unknown categories and perform incremental learning, respectively. In this paper, we provide a unified perspective: Semantic Topology. During the life-long learning of an open-world object detector, all object instances from the same category are assigned to their corresponding pre-defined node in the semantic topology, including the `unknown' category. This constraint builds up discriminative feature representations and consistent relationships among objects, thus enabling the detector to distinguish unknown objects out of the known categories, as well as making learned features of known objects undistorted when learning new categories incrementally. Extensive experiments demonstrate that semantic topology, either randomly-generated or derived from a well-trained language model, could outperform the current state-of-the-art open-world object detectors by a large margin, e.g., the absolute open-set error is reduced from 7832 to 2546, exhibiting the inherent superiority of semantic topology on open-world object detection.

cs.CV↗

arXiv2022

Enhanced quantum sensing with room-temperature solid-state masers

Quantum sensing with solid-state systems finds broad applications in diverse areas ranging from material and biomedical sciences to fundamental physics. Several solid-state spin sensors have been developed, facilitating the ultra-sensitive detection of physical quantities such as magnetic and electric fields and temperature. Exploiting collective behaviour of non-interacting spins holds the promise of pushing the detection limit to even lower levels, while to date, those levels are scarcely reached due to the broadened linewidth and inefficient readout of solid-state spin ensembles. Here, we experimentally demonstrate that such drawbacks can be overcome by newly reborn maser technology at room temperature in the solid state. Owing to maser action, we observe a 4-fold reduction in the inhomogeneously broadened linewidth of a molecular spin ensemble, which is narrower than the same measured from single spins at cryogenic temperatures. The maser-based readout applied to magnetometry showcases a signal-to-noise ratio (SNR) of 30 dB for single shots. This technique would be a significant addition to the toolbox for boosting the sensitivity of solid-state ensemble spin sensors.

quant-ph↗

arXiv2021

Numerical analysis of the LDG method for large deformations of prestrained plates

A local discontinuous Galerkin (LDG) method for approximating large deformations of prestrained plates is introduced and tested on several insightful numerical examples in our previous computational work. This paper presents a numerical analysis of this LDG method, focusing on the free boundary case. The problem consists of minimizing a fourth order bending energy subject to a nonlinear and nonconvex metric constraint. The energy is discretized using LDG and a discrete gradient flow is used for computing discrete minimizers. We first show $Γ$-convergence of the discrete energy to the continuous one. Then we prove that the discrete gradient flow decreases the energy at each step and computes discrete minimizers with control of the metric constraint defect. We also present a numerical scheme for initialization of the gradient flow, and discuss the conditional stability of it.

math.NA↗

arXiv2021

Semantically Contrastive Learning for Low-light Image Enhancement

Low-light image enhancement (LLE) remains challenging due to the unfavorable prevailing low-contrast and weak-visibility problems of single RGB images. In this paper, we respond to the intriguing learning-related question -- if leveraging both accessible unpaired over/underexposed images and high-level semantic guidance, can improve the performance of cutting-edge LLE models? Here, we propose an effective semantically contrastive learning paradigm for LLE (namely SCL-LLE). Beyond the existing LLE wisdom, it casts the image enhancement task as multi-task joint learning, where LLE is converted into three constraints of contrastive learning, semantic brightness consistency, and feature preservation for simultaneously ensuring the exposure, texture, and color consistency. SCL-LLE allows the LLE model to learn from unpaired positives (normal-light)/negatives (over/underexposed), and enables it to interact with the scene semantics to regularize the image enhancement network, yet the interaction of high-level semantic knowledge and the low-level signal prior is seldom investigated in previous methods. Training on readily available open data, extensive experiments demonstrate that our method surpasses the state-of-the-arts LLE models over six independent cross-scenes datasets. Moreover, SCL-LLE's potential to benefit the downstream semantic segmentation under extremely dark conditions is discussed. Source Code: https://github.com/LingLIx/SCL-LLE.

cs.CV↗

arXiv2021

Searching for axion-like particles at future electron-positron colliders

We investigate the prospects for discovering axion-like particles (ALPs) via a light-by-light (LBL) scattering at two colliders, the future circular collider (FCC-ee) and circular electron-positron collider (CEPC). The promising sensitivities to the effective ALP-photon coupling $g_{aγγ}$ are obtained. Our numerical results show that the FCC-ee and CEPC might be more sensitive to the ALPs with mass 2 GeV $\sim$ 10 GeV than the LHC and CLIC.

hep-ph↗

arXiv2021

Does Preprocessing Help Training Over-parameterized Neural Networks?

Deep neural networks have achieved impressive performance in many areas. Designing a fast and provable method for training neural networks is a fundamental question in machine learning. The classical training method requires paying $Ω(mnd)$ cost for both forward computation and backward computation, where $m$ is the width of the neural network, and we are given $n$ training points in $d$-dimensional space. In this paper, we propose two novel preprocessing ideas to bypass this $Ω(mnd)$ barrier: $\bullet$ First, by preprocessing the initial weights of the neural networks, we can train the neural network in $\widetilde{O}(m^{1-Θ(1/d)} n d)$ cost per iteration. $\bullet$ Second, by preprocessing the input data points, we can train the neural network in $\widetilde{O} (m^{4/5} nd )$ cost per iteration. From the technical perspective, our result is a sophisticated combination of tools in different fields, greedy-type convergence analysis in optimization, sparsity observation in practical work, high-dimensional geometric search in data structure, concentration and anti-concentration in probability. Our results also provide theoretical insights for a large number of previously established fast training methods. In addition, our classical algorithm can be generalized to the Quantum computation model. Interestingly, we can get a similar sublinear cost per iteration but avoid preprocessing initial weights or input data points.

cs.LG↗

arXiv2021

Speeding up Deep Model Training by Sharing Weights and Then Unsharing

We propose a simple and efficient approach for training the BERT model. Our approach exploits the special structure of BERT that contains a stack of repeated modules (i.e., transformer encoders). Our proposed approach first trains BERT with the weights shared across all the repeated modules till some point. This is for learning the commonly shared component of weights across all repeated layers. We then stop weight sharing and continue training until convergence. We present theoretic insights for training by sharing weights then unsharing with analysis for simplified models. Empirical experiments on the BERT model show that our method yields better performance of trained models, and significantly reduces the number of training iterations.

cs.LG↗

arXiv2021

Equality Constrained Linear Optimal Control With Factor Graphs

This paper presents a novel factor graph-based approach to solve the discrete-time finite-horizon Linear Quadratic Regulator problem subject to auxiliary linear equality constraints within and across time steps. We represent such optimal control problems using constrained factor graphs and optimize the factor graphs to obtain the optimal trajectory and the feedback control policies using the variable elimination algorithm with a modified Gram-Schmidt process. We prove that our approach has the same order of computational complexity as the state-of-the-art dynamic programming approach. Furthermore, current dynamic programming approaches can only handle equality constraints between variables at the same time step, but ours can handle equality constraints among any combination of variables at any time step while maintaining linear complexity with respect to trajectory length. Our approach can be used to efficiently generate trajectories and feedback control policies to achieve periodic motion or repetitive manipulation.

cs.RO↗

arXiv2021

Tensor network representations of fermionic crystalline topological phases on two-dimensional lattices

We investigate the tensor network representations of fermionic crystalline symmetry-protected topological (SPT) phases on two-dimensional lattices. As a mapping from virtual indices to physical indices, projected entangled-pair state (PEPS) serves as a concrete way to construct the wavefunctions of 2D crystalline fermionic SPT (fSPT) phases protected by 17 wallpaper group symmetries, for both spinless and spin-1/2 fermions. Based on PEPS, the full classification of 2D crystalline fSPT phases with wallpaper groups can be obtained. Tensor network states provide a natural framework for studying 2D crystalline fSPT phases.

cond-mat.str-el↗

arXiv2021

Free Lunch for Few-shot Learning: Distribution Calibration

Learning from a limited number of samples is challenging since the learned model can easily become overfitted based on the biased distribution formed by only a few training examples. In this paper, we calibrate the distribution of these few-sample classes by transferring statistics from the classes with sufficient examples, then an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. We assume every dimension in the feature representation follows a Gaussian distribution so that the mean and the variance of the distribution can borrow from that of similar classes whose statistics are better estimated with an adequate number of samples. Our method can be built on top of off-the-shelf pretrained feature extractors and classification models without extra parameters. We show that a simple logistic regression classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy on two datasets (~5% improvement on miniImageNet compared to the next best). The visualization of these generated features demonstrates that our calibrated distribution is an accurate estimation.

cs.LG↗

arXiv2021

Semi-TCL: Semi-Supervised Track Contrastive Representation Learning

Online tracking of multiple objects in videos requires strong capacity of modeling and matching object appearances. Previous methods for learning appearance embedding mostly rely on instance-level matching without considering the temporal continuity provided by videos. We design a new instance-to-track matching objective to learn appearance embedding that compares a candidate detection to the embedding of the tracks persisted in the tracker. It enables us to learn not only from videos labeled with complete tracks, but also unlabeled or partially labeled videos. We implement this learning objective in a unified form following the spirit of constrastive loss. Experiments on multiple object tracking datasets demonstrate that our method can effectively learning discriminative appearance embeddings in a semi-supervised fashion and outperform state of the art methods on representative benchmarks.

cs.CV↗

arXiv2021

Positive-Congruent Training: Towards Regression-Free Model Updates

Reducing inconsistencies in the behavior of different versions of an AI system can be as important in practice as reducing its overall error. In image classification, sample-wise inconsistencies appear as "negative flips": A new model incorrectly predicts the output for a test sample that was correctly classified by the old (reference) model. Positive-congruent (PC) training aims at reducing error rate while at the same time reducing negative flips, thus maximizing congruency with the reference model only on positive predictions, unlike model distillation. We propose a simple approach for PC training, Focal Distillation, which enforces congruence with the reference model by giving more weights to samples that were correctly classified. We also found that, if the reference model itself can be chosen as an ensemble of multiple deep neural networks, negative flips can be further reduced without affecting the new model's accuracy.

cs.CV↗

arXiv2021

Compatibility-aware Heterogeneous Visual Search

We tackle the problem of visual search under resource constraints. Existing systems use the same embedding model to compute representations (embeddings) for the query and gallery images. Such systems inherently face a hard accuracy-efficiency trade-off: the embedding model needs to be large enough to ensure high accuracy, yet small enough to enable query-embedding computation on resource-constrained platforms. This trade-off could be mitigated if gallery embeddings are generated from a large model and query embeddings are extracted using a compact model. The key to building such a system is to ensure representation compatibility between the query and gallery models. In this paper, we address two forms of compatibility: One enforced by modifying the parameters of each model that computes the embeddings. The other by modifying the architectures that compute the embeddings, leading to compatibility-aware neural architecture search (CMP-NAS). We test CMP-NAS on challenging retrieval tasks for fashion images (DeepFashion2), and face images (IJB-C). Compared to ordinary (homogeneous) visual search using the largest embedding model (paragon), CMP-NAS achieves 80-fold and 23-fold cost reduction while maintaining accuracy within 0.3% and 1.6% of the paragon on DeepFashion2 and IJB-C respectively.

cs.CV↗

arXiv2021

Detecting anomaly in vector boson scattering

Measuring the vector boson scattering (VBS) precisely is an important step towards understanding the electroweak symmetry breaking of the standard model (SM) and detecting new physics beyond the SM. We propose a neural network which compress the features of the VBS into three dimensional latent space. The consistency of the SM prediction and the experimental data is tested by the binned log-likelihood analysis in the latent space. We will show that the network is capable of distinguish different polarization modes of $WWjj$ production in both dileptonic channel and semi-leptonic channel. The method is also applied to constrain the effective field theory and two Higgs Doublet Model. The results demonstrate that the method is sensitive to generic new physics contributing to the VBS.

hep-ph↗

arXiv2021

Optimal Control for Structurally Sparse Systems using Graphical Inference

Dynamical systems with a distributed yet interconnected structure, like multi-rigid-body robots or large-scale multi-agent systems, introduce valuable sparsity into the system dynamics that can be exploited in an optimal control setting for speeding up computation and improving numerical conditioning. Conventional approaches for solving the Optimal Control Problem (OCP) rarely capitalize on such structural sparsity, and hence suffer from a cubic computational complexity growth as the dimensionality of the system scales. In this paper, we present an OCP formulation that relies on graphical models to capture the sparsely-interconnected nature of the system dynamics. Such a representational choice allows the use of contemporary graphical inference algorithms that enable our solver to achieve a linear time complexity in the state and control dimensions as well as the time horizon. We demonstrate the numerical and computational advantages of our approach on a canonical dynamical system in simulation.

cs.RO↗

arXiv2021

Gamma-convergent projection-free finite element methods for nematic liquid crystals: The Ericksen model

The Ericksen model for nematic liquid crystals couples a director field with a scalar degree of orientation variable, and allows the formation of various defects with finite energy. We propose a simple but novel finite element approximation of the problem that can be implemented easily within standard finite element packages. Our scheme is projection-free and thus circumvents the use of weakly acute meshes, which are quite restrictive in 3D but are required by recent algorithms for convergence. We prove stability and $Γ$-convergence properties of the new method in the presence of defects. We also design an effective nested gradient flow algorithm for computing minimizers that controls the violation of the unit-length constraint of the director. We present several simulations in 2D and 3D that document the performance of the proposed scheme and its ability to capture quite intriguing defects.

math.NA↗

arXiv2021

Single-View 3D Object Reconstruction from Shape Priors in Memory

Existing methods for single-view 3D object reconstruction directly learn to transform image features into 3D representations. However, these methods are vulnerable to images containing noisy backgrounds and heavy occlusions because the extracted image features do not contain enough information to reconstruct high-quality 3D shapes. Humans routinely use incomplete or noisy visual cues from an image to retrieve similar 3D shapes from their memory and reconstruct the 3D shape of an object. Inspired by this, we propose a novel method, named Mem3D, that explicitly constructs shape priors to supplement the missing information in the image. Specifically, the shape priors are in the forms of "image-voxel" pairs in the memory network, which is stored by a well-designed writing strategy during training. We also propose a voxel triplet loss function that helps to retrieve the precise 3D shapes that are highly related to the input image from shape priors. The LSTM-based shape encoder is introduced to extract information from the retrieved 3D shapes, which are useful in recovering the 3D shape of an object that is heavily occluded or in complex environments. Experimental results demonstrate that Mem3D significantly improves reconstruction quality and performs favorably against state-of-the-art methods on the ShapeNet and Pix3D datasets.

cs.CV↗

arXiv2021

Combinatorial Bandits without Total Order for Arms

We consider the combinatorial bandits problem, where at each time step, the online learner selects a size-$k$ subset $s$ from the arms set $\mathcal{A}$, where $\left|\mathcal{A}\right| = n$, and observes a stochastic reward of each arm in the selected set $s$. The goal of the online learner is to minimize the regret, induced by not selecting $s^*$ which maximizes the expected total reward. Specifically, we focus on a challenging setting where 1) the reward distribution of an arm depends on the set $s$ it is part of, and crucially 2) there is \textit{no total order} for the arms in $\mathcal{A}$. In this paper, we formally present a reward model that captures set-dependent reward distribution and assumes no total order for arms. Correspondingly, we propose an Upper Confidence Bound (UCB) algorithm that maintains UCB for each individual arm and selects the arms with top-$k$ UCB. We develop a novel regret analysis and show an $O\left(\frac{k^2 n \log T}ε\right)$ gap-dependent regret bound as well as an $O\left(k^2\sqrt{n T \log T}\right)$ gap-independent regret bound. We also provide a lower bound for the proposed reward model, which shows our proposed algorithm is near-optimal for any constant $k$. Empirical results on various reward models demonstrate the broad applicability of our algorithm.

cs.LG↗