arXiv subjects

Tao Luo

Publications and source records attributed to Tao Luo.

At least 145 records · Page 8Linked to original sources

arXiv2022

Beyond Codebook-Based Analog Beamforming at mmWave: Compressed Sensing and Machine Learning Methods

Analog beamforming is the predominant approach for millimeter wave (mmWave) communication given its favorable characteristics for limited-resource devices. In this work, we aim at reducing the spectral efficiency gap between analog and digital beamforming methods. We propose a method for refined beam selection based on the estimated raw channel. The channel estimation, an underdetermined problem, is solved using compressed sensing (CS) methods leveraging angular domain sparsity of the channel. To reduce the complexity of CS methods, we propose dictionary learning iterative soft-thresholding algorithm, which jointly learns the sparsifying dictionary and signal reconstruction. We evaluate the proposed method on a realistic mmWave setup and show considerable performance improvement with respect to code-book based analog beamforming approaches.

eess.SP↗

arXiv2022

Towards Understanding the Condensation of Neural Networks at Initial Training

Empirical works show that for ReLU neural networks (NNs) with small initialization, input weights of hidden neurons (the input weight of a hidden neuron consists of the weight from its input layer to the hidden neuron and its bias term) condense on isolated orientations. The condensation dynamics implies that the training implicitly regularizes a NN towards one with a much smaller effective size. In this work, we illustrate the formation of the condensation in multi-layer fully connected NNs and show that the maximal number of condensed orientations in the initial training stage is twice the multiplicity of the activation function, where "multiplicity" indicates the multiple roots of activation function at origin. Our theoretical analysis confirms experiments for two cases, one is for the activation function of multiplicity one with arbitrary dimension input, which contains many common activation functions, and the other is for the layer with one-dimensional input and arbitrary multiplicity. This work makes a step towards understanding how small initialization leads NNs to condensation at the initial training stage.

cs.LG↗

arXiv2022

Nonlinear Asymptotic Stability of Gravitational Hydrostatic Equilibrium for Viscous White Dwarfs with Symmetric Perturbations

We prove the nonlinear asymptotic stability of the gravitational hydrostatic equilibrium for the general equation of state of pressure-density relation in the framework of vacuum free boundary problem of spherically symmetric compressible Navier-Stokes-Poisson equations in three dimensions.The results apply to white dwarfs and polytropes with $γ\ge 2$, for which even the linearized asymptotic stability results are not available in literature, to the best of the authors' knowledge. Detailed decay rates of perturbations are given.

math.AP↗

arXiv2022

Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width

Substantial work indicates that the dynamics of neural networks (NNs) is closely related to their initialization of parameters. Inspired by the phase diagram for two-layer ReLU NNs with infinite width (Luo et al., 2021), we make a step towards drawing a phase diagram for three-layer ReLU NNs with infinite width. First, we derive a normalized gradient flow for three-layer ReLU NNs and obtain two key independent quantities to distinguish different dynamical regimes for common initialization methods. With carefully designed experiments and a large computation cost, for both synthetic datasets and real datasets, we find that the dynamics of each layer also could be divided into a linear regime and a condensed regime, separated by a critical regime. The criteria is the relative change of input weights (the input weight of a hidden neuron consists of the weight from its input layer to the hidden neuron and its bias term) as the width approaches infinity during the training, which tends to $0$, $+\infty$ and $O(1)$, respectively. In addition, we also demonstrate that different layers can lie in different dynamical regimes in a training process within a deep NN. In the condensed regime, we also observe the condensation of weights in isolated orientations with low complexity. Through experiments under three-layer condition, our phase diagram suggests a complicated dynamical regimes consisting of three possible regimes, together with their mixture, for deep NNs and provides a guidance for studying deep NNs in different initialization regimes, which reveals the possibility of completely different dynamics emerging within a deep NN for its different layers.

cs.LG↗

arXiv2022

Statistical Modeling of Soft Error Influence on Neural Networks

Soft errors in large VLSI circuits pose dramatic influence on computing- and memory-intensive neural network (NN) processing. Understanding the influence of soft errors on NNs is critical to protect against soft errors for reliable NN processing. Prior work mainly rely on fault simulation to analyze the influence of soft errors on NN processing. They are accurate but usually specific to limited configurations of errors and NN models due to the prohibitively slow simulation speed especially for large NN models and datasets. With the observation that the influence of soft errors propagates across a large number of neurons and accumulates as well, we propose to characterize the soft error induced data disturbance on each neuron with normal distribution model according to central limit theorem and develop a series of statistical models to analyze the behavior of NN models under soft errors in general. The statistical models reveal not only the correlation between soft errors and NN model accuracy, but also how NN parameters such as quantization and architecture affect the reliability of NNs. The proposed models are compared with fault simulation and verified comprehensively. In addition, we observe that the statistical models that characterize the soft error influence can also be utilized to predict fault simulation results in many cases and we explore the use of the proposed statistical models to accelerate fault simulations of NNs. According to our experiments, the accelerated fault simulation shows almost two orders of magnitude speedup with negligible simulation accuracy loss over the baseline fault simulations.

cs.LG↗

arXiv2022

Image Segmentation Semantic Communication over Internet of Vehicles

In this paper, the problem of semantic-based efficient image transmission is studied over the Internet of Vehicles (IoV). In the considered model, a vehicle shares massive amount of visual data perceived by its visual sensors to assist other vehicles in making driving decisions. However, it is hard to maintain a high reliable visual data transmission due to the limited spectrum resources. To tackle this problem, a semantic communication approach is introduced to reduce the transmission data amount while ensuring the semantic-level accuracy. Particularly, an image segmentation semantic communication (ISSC) system is proposed, which can extract the semantic features from the perceived images and transmit the features to the receiving vehicle that reconstructs the image segmentations. The ISSC system consists of an encoder and a decoder at the transmitter and the receiver, respectively. To accurately extract the image semantic features, the ISSC system encoder employs a Swin Transformer based multi-scale semantic feature extractor. Then, to resist the wireless noise and reconstruct the image segmentation, a semantic feature decoder and a reconstructor are designed at the receiver. Simulation results show that the proposed ISSC system can reconstruct the image segmentation accurately with a high compression ratio, and can achieve robust transmission performance against channel noise, especially at the low signal-to-noise ratio (SNR). In terms of mean Intersection over Union (mIoU), the ISSC system can achieve an increase by 75%, compared to the baselines using traditional coding method

cs.NI↗

arXiv2022

Benchmarking Quantum(-inspired) Annealing Hardware on Practical Use Cases

Quantum(-inspired) annealers show promise in solving combinatorial optimisation problems in practice. There has been extensive researches demonstrating the utility of D-Wave quantum annealer and quantum-inspired annealer, i.e., Fujitsu Digital Annealer on various applications, but few works are comparing these platforms. In this paper, we benchmark quantum(-inspired) annealers with three combinatorial optimisation problems ranging from generic scientific problems to complex problems in practical use. In the case where the problem size goes beyond the capacity of a quantum(-inspired) computer, we evaluate them in the context of decomposition. Experiments suggest that both annealers are effective on problems with small size and simple settings, but lose their utility when facing problems in practical size and settings. Decomposition methods extend the scalability of annealers, but they are still far away from practical use. Based on the experiments and comparison, we discuss the advantages and limitations of quantum(-inspired) annealers, as well as the research directions that may improve the utility and scalability of the these emerging computing technologies.

quant-ph↗

arXiv2022

An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network

Deep neural network (DNN) usually learns the target function from low to high frequency, which is called frequency principle or spectral bias. This frequency principle sheds light on a high-frequency curse of DNNs -- difficult to learn high-frequency information. Inspired by the frequency principle, a series of works are devoted to develop algorithms for overcoming the high-frequency curse. A natural question arises: what is the upper limit of the decaying rate w.r.t. frequency when one trains a DNN? In this work, our theory, confirmed by numerical experiments, suggests that there is a critical decaying rate w.r.t. frequency in DNN training. Below the upper limit of the decaying rate, the DNN interpolates the training data by a function with a certain regularity. However, above the upper limit, the DNN interpolates the training data by a trivial function, i.e., a function is only non-zero at training data points. Our results indicate a better way to overcome the high-frequency curse is to design a proper pre-condition approach to shift high-frequency information to low-frequency one, which coincides with several previous developed algorithms for fast learning high-frequency information. More importantly, this work rigorously proves that the high-frequency curse is an intrinsic difficulty of DNNs.

cs.LG↗

arXiv2022

Performance Optimization for Semantic Communications: An Attention-based Reinforcement Learning Approach

In this paper, a semantic communication framework is proposed for textual data transmission. In the studied model, a base station (BS) extracts the semantic information from textual data, and transmits it to each user. The semantic information is modeled by a knowledge graph (KG) that consists of a set of semantic triples. After receiving the semantic information, each user recovers the original text using a graph-to-text generation model. To measure the performance of the considered semantic communication framework, a metric of semantic similarity (MSS) that jointly captures the semantic accuracy and completeness of the recovered text is proposed. Due to wireless resource limitations, the BS may not be able to transmit the entire semantic information to each user and satisfy the transmission delay constraint. Hence, the BS must select an appropriate resource block for each user as well as determine and transmit part of the semantic information to the users. As such, we formulate an optimization problem whose goal is to maximize the total MSS by jointly optimizing the resource allocation policy and determining the partial semantic information to be transmitted. To solve this problem, a proximal-policy-optimization-based reinforcement learning (RL) algorithm integrated with an attention network is proposed. The proposed algorithm can evaluate the importance of each triple in the semantic information using an attention network and then, build a relationship between the importance distribution of the triples in the semantic information and the total MSS. Compared to traditional RL algorithms, the proposed algorithm can dynamically adjust its learning rate thus ensuring convergence to a locally optimal solution.

cs.IT↗

arXiv2022

Synthesis and Superconductivity in Yttrium-Cerium Hydrides at Moderate Pressures

Inspired by the high critical temperature in yttrium superhydride and the low stabilized pressure in superconducting cerium superhydride, we carry out four independent runs to synthesize yttrium-cerium alloy hydrides. The phases examined by the Raman scattering and x-ray diffraction measurements. The superconductivity is detected with the zero-resistance state at the critical temperature in the range of 97-140 K at pressures ranging from 114 GPa to 120$\pm$4 GPa. The maximum critical temperature of the synthesized hydrides is larger than those reported for cerium hydrides, while the corresponding stabilized pressure is much lower than those for superconducting yttrium hydrides. The structural analysis and theoretical calculations suggest that the phase of Y$_{0.5}$Ce$_{0.5}$H$_9$ has the space group $P6_3/mmc$ with the calculated critical temperature of 119 K, in fair agreement with the experiments. These results indicate that alloying superhydrides indeed can maintain relatively high critical temperature at modest pressures accessible by many laboratories.

cond-mat.supr-con↗

arXiv2022

Synthesis of Superconducting Phase of La$_{0.5}$Ce$_{0.5}$H$_{10}$ at High Pressures

Clathrate hydride \emph{Fm}\={3}\emph{m}-LaH$_{10}$ has been proven as the most extraordinary superconductor with the critical temperature $T_c$ above 250 K upon compression of hundreds of GPa in recent years. A general hope is to reduce the stabilization pressure and maintain the high $T_c$ value of the specific phase in LaH$_{10}$. However, strong structural instability distorts \emph{Fm}\={3}\emph{m} structure and leads to a rapid decrease of $T_c$ at low pressures. Here, we investigate the phase stability and superconducting behaviors of \emph{Fm}\={3}\emph{m}-LaH$_{10}$ with enhanced chemical pre-compression through partly replacing La by Ce atoms from both experiments and calculations. For explicitly characterizing the synthesized hydride, we choose lanthanum-cerium alloy with stoichiometry composition of 1:1. X-ray diffraction and Raman scattering measurements reveal the stabilization of \emph{Fm}\={3}\emph{m}-La$_{0.5}$Ce$_{0.5}$H$_{10}$ in the pressure range of 140-160 GPa. Superconductivity with $T_c$ of 175$\pm$2 K at 155 GPa is confirmed with the observation of the zero-resistivity state and supported by the theoretical calculations. These findings provide applicability in the future explorations for a large variety of hydrogen-rich hydrides.

cond-mat.supr-con↗

arXiv2022

An Insight-HXMT view of the mHz quasi-regular modulation phenomenon in the black hole X-ray binary 4U 1630-47

Here we report the spectral-timing results of the black hole X-ray binary 4U 1630-47 during its 2021 outburst using observations from the Hard X-ray Modulation Telescope. Type-C quasi-periodic oscillations (QPOs) in 1.6--4.2 Hz and quasi-regular modulation (QRM) near 60 mHz are detected during the outburst. The mHz QRM has a fractional rms of 10%--16% in the 8--35 keV energy band with a Q factor (frequency/width) of 2--4. Benefiting from the broad energy band of hxmt, we study the energy dependence of the 60 mHz QRM in 1--100 keV for the first time. We find that the fractional rms of the mHz QRM increases with photon energy, while the time lags of the mHz QRM are soft and decrease with photon energy. Fast recurrence of the mHz QRM, in a timescale of less than one hour, has been observed during the outburst. During this period, the corresponding energy spectra moderately change when the source transitions from the QRM state to the non-QRM state. The QRM phenomena also shows a dependence with the accretion rate. We suggest that the QRM could be caused by an unknown accretion instability aroused from the corona.

astro-ph.HE↗

arXiv2022

Existence, uniqueness, and energy scaling of 2+1 dimensional continuum model for stepped epitaxial surfaces with elastic effects

We study the 2+1 dimensional continuum model for the evolution of stepped epitaxial surface under long-range elastic interaction proposed by Xu and Xiang (SIAM J. Appl. Math. 69, 1393-1414, 2009). The long-range interaction term and the two length scales in this model makes PDE analysis challenging. Moreover, unlike in the 1+1 dimensional case, there is a nonconvexity contribution in the total energy in the 2+1 dimensional case, and it is not easy to prove that the solution is always in the well-posed regime during the evolution. In this paper, we propose a modified 2+1 dimensional continuum model based on the underlying physics. This modification fixes the problem of possible illposedness due to the nonconvexity of the energy functional. We prove the existence and uniqueness of both the static and dynamic solutions and derive a minimum energy scaling law for them. We show that the minimum energy surface profile is mainly attained by surfaces with step meandering instability. This is essentially different from the energy scaling law for the 1+1 dimensional epitaxial surfaces under elastic effects attained by step bunching surface profiles. We also discuss the transition from the step bunching instability to the step meandering instability in 2+1 dimensions.

math.AP↗

arXiv2022

Singular Limits for the Navier-Stokes-Poisson Equations of Viscous Plasma with Strong Density Boundary Layer

The quasi-neutral limit of the Navier-Stokes-Poisson system modeling a viscous plasma with vanishing viscosity coefficients in the half-space $\mathbb{R}^{3}_{+}$ is rigorously proved under a Navier-slip boundary condition for velocity and the Dirichlet boundary condition for electric potential. This is achieved by establishing the nonlinear stability of the approximation solutions involving the strong boundary layer in density and electric potential, which comes from the break-down of the quasi-neutrality near the boundary, and dealing with the difficulty of the interaction of this strong boundary layer with the weak boundary layer of the velocity field.

math.AP↗

arXiv2022

From Mean Field Games To Navier-Stokes Equations

This work establishes the equivalence between Mean Field Game and a class of PDE systems closely related to compressible Navier-Stokes equations. The solvability of the PDE system via the existence of the Nash Equilibrium of the Mean Field Game is provided under a set of conditions.

math.AP↗

arXiv2022

A Resource-efficient Spiking Neural Network Accelerator Supporting Emerging Neural Encoding

Spiking neural networks (SNNs) recently gained momentum due to their low-power multiplication-free computing and the closer resemblance of biological processes in the nervous system of humans. However, SNNs require very long spike trains (up to 1000) to reach an accuracy similar to their artificial neural network (ANN) counterparts for large models, which offsets efficiency and inhibits its application to low-power systems for real-world use cases. To alleviate this problem, emerging neural encoding schemes are proposed to shorten the spike train while maintaining the high accuracy. However, current accelerators for SNN cannot well support the emerging encoding schemes. In this work, we present a novel hardware architecture that can efficiently support SNN with emerging neural encoding. Our implementation features energy and area efficient processing units with increased parallelism and reduced memory accesses. We verified the accelerator on FPGA and achieve 25% and 90% improvement over previous work in power consumption and latency, respectively. At the same time, high area efficiency allows us to scale for large neural network models. To the best of our knowledge, this is the first work to deploy the large neural network model VGG on physical FPGA-based neuromorphic hardware.

cs.AR↗

arXiv2022

Bunching instability and asymptotic properties in epitaxial growth with elasticity effects: continuum model

We study the continuum epitaxial model for elastic interacting atomic steps on vicinal surfaces proposed by Xiang and E (Xiang, SIAM J. Appl. Math. 63:241-258, 2002; Xiang and E, Phys. Rev. B 69:035409, 2004). The non-local term and the singularity complicate the analysis of its PDE. In this paper, we first generalize this model to the Lennard-Jones (m,n) interaction between steps. Based on several important formulations of the non-local energy, we prove the existence, symmetry, unimodality, and regularity of the energy minimizer in the periodic setting. In particular, the symmetry and unimodality of the minimizer implies that it has a bunching profile. Furthermore, we derive the minimum energy scaling law for the original continnum model. All results are consistent with the corresponding results proved for discrete models by Luo et al. (Luo et al., Multiscale Model. Simul. 14:737 - 771, 2016).

math.AP↗

arXiv2022

Feasibility Study of $D_s^+ \to τ^+ ν_τ$ Decay and Test of Lepton Flavor Universality with Leptonic $D_s^+$ Decays at STCF

We report a sensitive study of $D_s^+ \to τ^+ ν_τ$ decay via $τ^+ \to e^+ ν_e \barν_τ$ with an integrated luminosity of 1 ab$^{-1}$ at the center-of-mass energy of $4.009$ GeV at a future Super Tau Charm Facility (STCF). Under the help of the fast simulation software package, the statistical sensitivity for the absolute branching fraction of $D_s^+ \to τ^+ ν_τ$ is determined to be $2\times10^{-4}$. Combining with our previous prospect of $D_s^+ \to μ^+ ν_μ$, the ratio of the branching fractions for $D_s^+ \to τ^+ ν_τ$ over $D_s^+ \to μ^+ ν_μ$ can achieve a relative statistical precision of 0.5%, which will provide the most stringent test of the $τ$-$μ$ lepton flavor universality in heavy quark decays. Taking the decay constant $f_{D_s^+}$ from lattice QCD calculations or the CKM matrix element $|V_{cs}|$ from the CKMfitter group as an input, the relative statistical uncertainties for $|V_{cs}|$ and $f_{D_s^+}$ are estimated to be 0.3% and 0.2%, respectively.

hep-ex↗