arXiv subjects

Yang Li

Publications and source records attributed to Yang Li.

At least 1,225 records · Page 68Linked to original sources

arXiv2021

On convergence of numerical solutions for the compressible MHD system with weakly divergence-free magnetic field

We study a general convergence theory for the analysis of numerical solutions to the magnetohydrodynamic system describing the time evolution of compressible, viscous, electrically conducting fluids in space dimension d (= 2; 3). First, we introduce the concept of dissipative weak solutions and prove the weak-strong uniqueness property for dissipative weak solutions, meaning a dissipative weak solution coincides with a classical solution emanating from the same initial data on the lifespan of the latter. Next, we introduce the concept of consistent approximations and prove the convergence of consistent approximations towards the dissipative weak solution as well as the classical solution. Interpreting the consistent approximation as the energy stability and consistency of numerical solutions, we have built a nonlinear variant of the celebrated Lax-equivalence theorem. Finally, as an application of this theory, we show the convergence analysis for two numerical methods.

math.AP↗

arXiv2021

Distinct thermodynamic and dynamic effects produced by scale factors in conformally related Einstein-power-Yang-Mills black holes

We study the thermodynamics and dynamics of high-dimensional Einstein-power-Yang-Mills black holes in conformal gravity. Specifically, we investigate a class of conformally related black holes whose metrics differ by a scale factor. We show that a suitable scale factor cures the geodesic incompleteness and the divergence of Kretschmann scalars at the center of black holes. In the aspect of thermodynamics, we analyse the Hawking temperature, the entropy, and the specific heat, and verify the existence of second-order phase transitions. We find that the thermodynamics of this class of conformally related black holes is independent of scale factors. In the aspect of dynamics, we find that the quasinormal modes of minimally coupled scalar field perturbations are dependent on scale factors. Quite interesting is that the behavior of quasinormal mode frequencies also supports the independence of scale factors for the second-order phase transitions. Our results show that the scale factors produce distinct thermodynamic and dynamic effects in the conformally related Einstein-power-Yang-Mills black holes, which provides an interesting connection between thermodynamics and dynamics of black holes in conformal gravity.

gr-qc↗

arXiv2021

Rethinking Training from Scratch for Object Detection

The ImageNet pre-training initialization is the de-facto standard for object detection. He et al. found it is possible to train detector from scratch(random initialization) while needing a longer training schedule with proper normalization technique. In this paper, we explore to directly pre-training on target dataset for object detection. Under this situation, we discover that the widely adopted large resizing strategy e.g. resize image to (1333, 800) is important for fine-tuning but it's not necessary for pre-training. Specifically, we propose a new training pipeline for object detection that follows `pre-training and fine-tuning', utilizing low resolution images within target dataset to pre-training detector then load it to fine-tuning with high resolution images. With this strategy, we can use batch normalization(BN) with large bath size during pre-training, it's also memory efficient that we can apply it on machine with very limited GPU memory(11G). We call it direct detection pre-training, and also use direct pre-training for short. Experiment results show that direct pre-training accelerates the pre-training phase by more than 11x on COCO dataset while with even +1.8mAP compared to ImageNet pre-training. Besides, we found direct pre-training is also applicable to transformer based backbones e.g. Swin Transformer. Code will be available.

cs.CV↗

arXiv2021

Relational Learning with Gated and Attentive Neighbor Aggregator for Few-Shot Knowledge Graph Completion

Aiming at expanding few-shot relations' coverage in knowledge graphs (KGs), few-shot knowledge graph completion (FKGC) has recently gained more research interests. Some existing models employ a few-shot relation's multi-hop neighbor information to enhance its semantic representation. However, noise neighbor information might be amplified when the neighborhood is excessively sparse and no neighbor is available to represent the few-shot relation. Moreover, modeling and inferring complex relations of one-to-many (1-N), many-to-one (N-1), and many-to-many (N-N) by previous knowledge graph completion approaches requires high model complexity and a large amount of training instances. Thus, inferring complex relations in the few-shot scenario is difficult for FKGC models due to limited training instances. In this paper, we propose a few-shot relational learning with global-local framework to address the above issues. At the global stage, a novel gated and attentive neighbor aggregator is built for accurately integrating the semantics of a few-shot relation's neighborhood, which helps filtering the noise neighbors even if a KG contains extremely sparse neighborhoods. For the local stage, a meta-learning based TransH (MTransH) method is designed to model complex relations and train our model in a few-shot learning fashion. Extensive experiments show that our model outperforms the state-of-the-art FKGC approaches on the frequently-used benchmark datasets NELL-One and Wiki-One. Compared with the strong baseline model MetaR, our model achieves 5-shot FKGC performance improvements of 8.0% on NELL-One and 2.8% on Wiki-One by the metric Hits@10.

cs.AI↗

arXiv2021

NRTSI: Non-Recurrent Time Series Imputation

Time series imputation is a fundamental task for understanding time series with missing data. Existing methods either do not directly handle irregularly-sampled data or degrade severely with sparsely observed data. In this work, we reformulate time series as permutation-equivariant sets and propose a novel imputation model NRTSI that does not impose any recurrent structures. Taking advantage of the permutation equivariant formulation, we design a principled and efficient hierarchical imputation procedure. In addition, NRTSI can directly handle irregularly-sampled time series, perform multiple-mode stochastic imputation, and handle data with partially observed dimensions. Empirically, we show that NRTSI achieves state-of-the-art performance across a wide range of time series imputation benchmarks.

cs.LG↗

arXiv2021

BSNN: Towards Faster and Better Conversion of Artificial Neural Networks to Spiking Neural Networks with Bistable Neurons

The spiking neural network (SNN) computes and communicates information through discrete binary events. It is considered more biologically plausible and more energy-efficient than artificial neural networks (ANN) in emerging neuromorphic hardware. However, due to the discontinuous and non-differentiable characteristics, training SNN is a relatively challenging task. Recent work has achieved essential progress on an excellent performance by converting ANN to SNN. Due to the difference in information processing, the converted deep SNN usually suffers serious performance loss and large time delay. In this paper, we analyze the reasons for the performance loss and propose a novel bistable spiking neural network (BSNN) that addresses the problem of spikes of inactivated neurons (SIN) caused by the phase lead and phase lag. Also, when ResNet structure-based ANNs are converted, the information of output neurons is incomplete due to the rapid transmission of the shortcut path. We design synchronous neurons (SN) to help efficiently improve performance. Experimental results show that the proposed method only needs 1/4-1/10 of the time steps compared to previous work to achieve nearly lossless conversion. We demonstrate state-of-the-art ANN-SNN conversion for VGG16, ResNet20, and ResNet34 on challenging datasets including CIFAR-10 (95.16% top-1), CIFAR-100 (78.12% top-1), and ImageNet (72.64% top-1).

cs.NE↗

arXiv2021

BackEISNN: A Deep Spiking Neural Network with Adaptive Self-Feedback and Balanced Excitatory-Inhibitory Neurons

Spiking neural networks (SNNs) transmit information through discrete spikes, which performs well in processing spatial-temporal information. Due to the non-differentiable characteristic, there still exist difficulties in designing well-performed SNNs. Recently, SNNs trained with backpropagation have shown superior performance due to the proposal of the gradient approximation. However, the performance on complex tasks is still far away from the deep neural networks. Taking inspiration from the autapse in the brain which connects the spiking neurons with a self-feedback connection, we apply an adaptive time-delayed self-feedback on the membrane potential to regulate the spike precisions. As well as, we apply the balanced excitatory and inhibitory neurons mechanism to control the spiking neurons' output dynamically. With the combination of the two mechanisms, we propose a deep spiking neural network with adaptive self-feedback and balanced excitatory and inhibitory neurons (BackEISNN). The experimental results on several standard datasets have shown that the two modules not only accelerate the convergence of the network but also improve the accuracy. For the MNIST, FashionMNIST, and N-MNIST datasets, our model has achieved state-of-the-art performance. For the CIFAR10 dataset, our BackEISNN also gets remarkable performance on a relatively light structure that competes against state-of-the-art SNNs.

cs.NE↗

arXiv2021

A multi-grid Cellular Automaton model for simulating dendrite growth and its application in additive manufacturing

The dendrite growth in casting and additive manufacturing is rather important and related to the formation of some defects. However, quantitatively simulating the growth of dendrites with arbitrary crystallographic orientations in 3-dimension(3D) is still very challenging. In the present work, we develop a multi-grid Cellular Automaton (CA) model for the dendrite growth. In this model, the interfacial area is further discretized into a child grid, on which the decentered octahedron growth algorithm is performed. The model is comprehensively and quantitatively verified by comparing with the prediction of analytical models and a published x-ray imaging observation result, proving that the model is quantitatively and morphologically accurate. After that, with the temperature gradient and cooling rate extracted from a finite-volume-method(FVM)-based thermal-fluid model, the model was applied in reproducing the dendrite growth process of nickel-based superalloy during a single-track electron beam melting process. The simulation results agree fairly well with the experimental observation, demonstrating the feasibility and effectiveness of using the model in additive manufacturing.

cond-mat.mtrl-sci↗

arXiv2021

2D MXenes: Visible Black but Infrared White Materials

Black materials with low infrared absorption/emission (or IR white) are rare in nature but highly desired in numerous areas, such as solar-thermal energy harvesting, multispectral camouflage, thermal insulation, and anti-counterfeiting. Due to the lack of spectral selectivity in intrinsic materials, such counter-intuitive properties are generally realized by constructing complicated subwavelength metamaterials with costly nanofabrication techniques. Here we report the low mid-IR emissivity (down to 10%) of 2D Ti3C2Tx MXenes. Associated with a high solar absorptance (up to 90%), they embrace the best spectral selectivity among the reported intrinsic black solar absorbing materials. Their appealing potentials in several aforementioned areas are experimentally demonstrated. First-principles calculations reveal that the IR emissivity of MXenes relies on both the nanoflake orientations and terminal groups, indicating great tunability. The calculations also suggest that more MXenes including Ti2CTx, Nb2CTx, and V2CTx are also potential low-emissivity materials. This work opens the avenue to further exploration of a family of intrinsically low-emissivity materials with over 70 members.

cond-mat.mtrl-sci↗

arXiv2021

A discrete dislocation dynamics study of precipitate bypass mechanisms in nickel-based superalloys

Order strengthening in nickel-based superalloys is associated with the extra stress required for dislocations to bypass the $γ'$ precipitates distributed in the $γ$ matrix. A rich variety of bypass mechanism has been identified, with various shearing and Orowan looping processes giving way to climb bypass as the operating conditions change from the low/intermediate temperatures and high stress regime, to the high temperature and low stress regime. When anti phase boundary (APB) shearing and Orowan looping mechanisms operate, the bypass mechanism changes from shearing to looping with increased particle size and within a broad coexistence size window. Another possibility, supported by indirect experimental evidence, is that a third "hybrid" transition mechanism may operate. In this paper we use discrete dislocation dynamics (DDD) simulations to study dislocation bypass mechanisms in Ni-based superalloys. We develop a new method to compute generalized stacking fault forces in DDD simulations. We use this method to study the mechanisms of bypass of a square lattice of spherical $γ'$ precipitates by $a/2\langle110\rangle\{111\}$ edge dislocations, as a function of the precipitates volume fraction and size. We show that the hybrid mechanism is possible and it operates as a transition mechanism between the shearing and looping regimes over a large range of precipitates volume fraction and radii. We also consider the effects of a $γ/γ'$ lattice misfit on the bypass mechanisms, which we approximate by an additional precipitate stress computed according to Eshelby's inclusion theory. We show that in the shearing and hybrid looping-shearing regimes, a lattice misfit generally results in an increased bypass stress. For sufficiently high lattice misfit, the bypass stress is controlled by the pinning of the trailing dislocation on the exit side of the precipitates.

cond-mat.mtrl-sci↗

arXiv2021

Anisotropic magnon-magnon coupling in synthetic antiferromagnets

The magnon-magnon coupling in synthetic antiferromagnets advances it as hybrid magnonic systems to explore the quantum information technologies. To induce the magnon-magnon coupling, the parity symmetry between two magnetization needs to be broken. Here we experimentally demonstrate a convenient method to break the parity symmetry by the asymmetric thickness of two magnetic layers and thus introduce a magnon-magnon coupling in Ir-based synthetic antiferromagnets CoFeB(10 nm)/Ir(tIr=0.6 nm, 1.2 nm)/CoFeB(13 nm). Remarkably, we find that the weakly uniaxial anisotropy field (~ 20 Oe) makes the magnon-magnon coupling anisotropic. The coupling strength presented by a characteristic anticrossing gap varies in the range between 0.54 GHz and 0.90 GHz for tIr =0.6 nm, and between nearly zero to 1.4 GHz for tIr = 1.2 nm, respectively. Our results demonstrate a feasible way to induce the magnon-magnon coupling by an asymmetric structure and tune the coupling strength by varying the direction of in-plane magnetic field. The magnon-magnon coupling in this highly tunable material system could open exciting perspectives for exploring quantum-mechanical coupling phenomena.

cond-mat.mtrl-sci↗

arXiv2021

Joint Text and Label Generation for Spoken Language Understanding

Generalization is a central problem in machine learning, especially when data is limited. Using prior information to enforce constraints is the principled way of encouraging generalization. In this work, we propose to leverage the prior information embedded in pretrained language models (LM) to improve generalization for intent classification and slot labeling tasks with limited training data. Specifically, we extract prior knowledge from pretrained LM in the form of synthetic data, which encode the prior implicitly. We fine-tune the LM to generate an augmented language, which contains not only text but also encodes both intent labels and slot labels. The generated synthetic data can be used to train a classifier later. Since the generated data may contain noise, we rephrase the learning from generated data as learning with noisy labels. We then utilize the mixout regularization for the classifier and prove its effectiveness to resist label noise in generated data. Empirically, our method demonstrates superior performance and outperforms the baseline by a large margin.

cs.CL↗

arXiv2021

4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface

Tracking non-rigidly deforming scenes using range sensors has numerous applications including computer vision, AR/VR, and robotics. However, due to occlusions and physical limitations of range sensors, existing methods only handle the visible surface, thus causing discontinuities and incompleteness in the motion field. To this end, we introduce 4DComplete, a novel data-driven approach that estimates the non-rigid motion for the unobserved geometry. 4DComplete takes as input a partial shape and motion observation, extracts 4D time-space embedding, and jointly infers the missing geometry and motion field using a sparse fully-convolutional network. For network training, we constructed a large-scale synthetic dataset called DeformingThings4D, which consists of 1972 animation sequences spanning 31 different animals or humanoid categories with dense 4D annotation. Experiments show that 4DComplete 1) reconstructs high-resolution volumetric shape and motion field from a partial observation, 2) learns an entangled 4D feature representation that benefits both shape and motion estimation, 3) yields more accurate and natural deformation than classic non-rigid priors such as As-Rigid-As-Possible (ARAP) deformation, and 4) generalizes well to unseen objects in real-world sequences.

cs.CV↗

arXiv2021

Neural Plasticity Networks

Neural plasticity is an important functionality of human brain, in which number of neurons and synapses can shrink or expand in response to stimuli throughout the span of life. We model this dynamic learning process as an $L_0$-norm regularized binary optimization problem, in which each unit of a neural network (e.g., weight, neuron or channel, etc.) is attached with a stochastic binary gate, whose parameters determine the level of activity of a unit in the network. At the beginning, only a small portion of binary gates (therefore the corresponding neurons) are activated, while the remaining neurons are in a hibernation mode. As the learning proceeds, some neurons might be activated or deactivated if doing so can be justified by the cost-benefit tradeoff measured by the $L_0$-norm regularized objective. As the training gets mature, the probability of transition between activation and deactivation will diminish until a final hardening stage. We demonstrate that all of these learning dynamics can be modulated by a single parameter $k$ seamlessly. Our neural plasticity network (NPN) can prune or expand a network depending on the initial capacity of network provided by the user; it also unifies dropout (when $k=0$), traditional training of DNNs (when $k=\infty$) and interpolates between these two. To the best of our knowledge, this is the first learning framework that unifies network sparsification and network expansion in an end-to-end training pipeline. Extensive experiments on synthetic dataset and multiple image classification benchmarks demonstrate the superior performance of NPN. We show that both network sparsification and network expansion can yield compact models of similar architectures, while retaining competitive accuracies of the original networks.

cs.NE↗

arXiv2021

Robust Deep Multi-Modal Sensor Fusion using Fusion Weight Regularization and Target Learning

Sensor fusion has wide applications in many domains including health care and autonomous systems. While the advent of deep learning has enabled promising multi-modal fusion of high-level features and end-to-end sensor fusion solutions, existing deep learning based sensor fusion techniques including deep gating architectures are not always resilient, leading to the issue of fusion weight inconsistency. We propose deep multi-modal sensor fusion architectures with enhanced robustness particularly under the presence of sensor failures. At the core of our gating architectures are fusion weight regularization and fusion target learning operating on auxiliary unimodal sensing networks appended to the main fusion model. The proposed regularized gating architectures outperform the existing deep learning architectures with and without gating under both clean and corrupted sensory inputs resulted from sensor failures. The demonstrated improvements are particularly pronounced when one or more multiple sensory modalities are corrupted.

cs.LG↗

arXiv2021

GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing

In recent studies, neural message passing has proved to be an effective way to design graph neural networks (GNNs), which have achieved state-of-the-art performance in many graph-based tasks. However, current neural-message passing architectures typically need to perform an expensive recursive neighborhood expansion in multiple rounds and consequently suffer from a scalability issue. Moreover, most existing neural-message passing schemes are inflexible since they are restricted to fixed-hop neighborhoods and insensitive to the actual demands of different nodes. We circumvent these limitations by a novel feature-message passing framework, called Graph Multi-layer Perceptron (GMLP), which separates the neural update from the message passing. With such separation, GMLP significantly improves the scalability and efficiency by performing the message passing procedure in a pre-compute manner, and is flexible and adaptive in leveraging node feature messages over various levels of localities. We further derive novel variants of scalable GNNs under this framework to achieve the best of both worlds in terms of performance and efficiency. We conduct extensive evaluations on 11 benchmark datasets, including large-scale datasets like ogbn-products and an industrial dataset, demonstrating that GMLP achieves not only the state-of-art performance, but also high training scalability and efficiency.

cs.LG↗

arXiv2021

Joint Optimization of Preamble Selection and Access Barring for MTC with Correlated Device Activities

Most existing works on random access for machine-type communication (MTC) assume independent device activities. However, in several Internet-of-Things (IoT) applications, device activities are driven by events and hence may be correlated. This paper investigates the joint optimization of preamble selection and access barring for correlated device activities. We adopt a random access scheme with general random preamble selection parameterized by the preamble selection distributions of all devices and an access barring scheme parameterized by the access barring factor, to maximally exploit correlated device activities for improving the average throughput. First, we formulate the average throughput maximization problem with respect to the preamble selection distributions and the access barring factor. It is a challenging nonconvex problem. We characterize an optimality property of the problem. Then, we develop two iterative algorithms to obtain a stationary point and a low-complexity solution respectively by using the block coordinate descend (BCD) method. Numerical results show that the two proposed solutions achieve significant gains over existing schemes, demonstrating the significance of exploiting correlation of device activities in improving the average throughput. Numerical results also show that compared to the stationary point, the low-complexity solution achieves a similar average throughput with much lower computational complexity, demonstrating the effectiveness of the low-complexity solution.

cs.IT↗

arXiv2021

SSIM-Based CTU-Level Joint Optimal Bit Allocation and Rate Distortion Optimization

Structural similarity (SSIM)-based distortion $D_\text{SSIM}$ is more consistent with human perception than the traditional mean squared error $D_\text{MSE}$. To achieve better video quality, many studies on optimal bit allocation (OBA) and rate-distortion optimization (RDO) used $D_\text{SSIM}$ as the distortion metric. However, many of them failed to optimize OBA and RDO jointly based on SSIM, thus causing a non-optimal R-$D_\text{SSIM}$ performance. This problem is due to the lack of an accurate R-$D_\text{SSIM}$ model that can be used uniformly in both OBA and RDO. To solve this problem, we propose a $D_\text{SSIM}$-$D_\text{MSE}$ model first. Based on this model, the complex R-$D_\text{SSIM}$ cost in RDO can be calculated as simpler R-$D_\text{MSE}$ cost with a new SSIM-related Lagrange multiplier. This not only reduces the computation burden of SSIM-based RDO, but also enables the R-$D_\text{SSIM}$ model to be uniformly used in OBA and RDO. Moreover, with the new SSIM-related Lagrange multiplier in hand, the joint relationship of R-$D_\text{SSIM}$-$λ_\text{SSIM}$ (the negative derivative of R-$D_\text{SSIM}$) can be built, based on which the R-$D_\text{SSIM}$ model parameters can be calculated accurately. With accurate and unified R-$D_\text{SSIM}$ model, SSIM-based OBA and SSIM-based RDO are unified together in our scheme, called SOSR. Compared with the HEVC reference encoder HM16.20, SOSR saves 4%, 10%, and 14% bitrate under the same SSIM in all-intra, hierarchical and non-hierarchical low-delay-B configurations, which is superior to other state-of-the-art schemes.

cs.MM↗