arXiv subjects

Lei Wang

Publications and source records attributed to Lei Wang.

At least 757 records · Page 42Linked to original sources

arXiv2023

CMLCompiler: A Unified Compiler for Classical Machine Learning

Classical machine learning (CML) occupies nearly half of machine learning pipelines in production applications. Unfortunately, it fails to utilize the state-of-the-practice devices fully and performs poorly. Without a unified framework, the hybrid deployments of deep learning (DL) and CML also suffer from severe performance and portability issues. This paper presents the design of a unified compiler, called CMLCompiler, for CML inference. We propose two unified abstractions: operator representations and extended computational graphs. The CMLCompiler framework performs the conversion and graph optimization based on two unified abstractions, then outputs an optimized computational graph to DL compilers or frameworks. We implement CMLCompiler on TVM. The evaluation shows CMLCompiler's portability and superior performance. It achieves up to 4.38$\times$ speedup on CPU, 3.31$\times$ speedup on GPU, and 5.09$\times$ speedup on IoT devices, compared to the state-of-the-art solutions -- scikit-learn, intel sklearn, and hummingbird. Our performance of CML and DL mixed pipelines achieves up to 3.04x speedup compared with cross-framework implementations. The project documents and source code are available at https://www.computercouncil.org/cmlcompiler.

cs.LG↗

arXiv2023

Learning Partial Correlation based Deep Visual Representation for Image Classification

Visual representation based on covariance matrix has demonstrates its efficacy for image classification by characterising the pairwise correlation of different channels in convolutional feature maps. However, pairwise correlation will become misleading once there is another channel correlating with both channels of interest, resulting in the ``confounding'' effect. For this case, ``partial correlation'' which removes the confounding effect shall be estimated instead. Nevertheless, reliably estimating partial correlation requires to solve a symmetric positive definite matrix optimisation, known as sparse inverse covariance estimation (SICE). How to incorporate this process into CNN remains an open issue. In this work, we formulate SICE as a novel structured layer of CNN. To ensure end-to-end trainability, we develop an iterative method to solve the above matrix optimisation during forward and backward propagation steps. Our work obtains a partial correlation based deep visual representation and mitigates the small sample problem often encountered by covariance matrix estimation in CNN. Computationally, our model can be effectively trained with GPU and works well with a large number of channels of advanced CNNs. Experiments show the efficacy and superior classification performance of our deep visual representation compared to covariance matrix based counterparts.

cs.CV↗

arXiv2023

Physical cool-core condensation radius in massive galaxy clusters

We investigate the properties of cool cores in an optimally selected sample of 37 massive and X-ray-bright galaxy clusters, with regular morphologies, observed with Chandra. We measured the density, temperature, and abundance radial profiles of their intracluster medium (ICM). From these independent quantities, we computed the cooling (tcool) free-fall (tff), and turbulence (teddy) timescales as a function of radius. By requiring the profile-crossing condition, tcool=teddy=1, we measured the cool-core condensation radius Rccc, within which the balancing feeding and feedback processes generate the turbulent condensation rain and related chaotic cold accretion (CCA). We also constrained the complementary (quenched) cooling flow radius Rqcf, obtained via the condition tcool=25Xtff, that encompasses the region of thermally unstable cooling. We find that in our cluster sample and in the limited redshift range considered (1.3E14<M500<16.6E14 Msun, 0.03<z<0.29), the distribution of Rccc peaks at 0.01r500 and the entire range remains below 0.07r500, with a very weak increase with redshift and no dependence on the cluster mass. We find that Rqcf is typically 3 times larger than Rccc, with a wider distribution, and growing more slowly along Rccc, according to an average relation Rqcf~Rccc^(0.46), with a large intrinsic scatter. We suggest that this sublinear relation can be understood as an effect of the micro rain of pockets of cooled gas flickering in the turbulent ICM, whose dynamical and thermodynamical properties are referred to as "macro weather". Substituting the classical cool-core radius R(7.7Gyr), we propose that Rqcf is an indicator of the size of the global cores tied to the long-term macro weather, with the inner Rccc closely tracing the effective condensation rain and chaotic cold accretion (CCA) zone that feeds the central supermassive black hole.

astro-ph.GA↗

arXiv2023

Universal scaling of Klein bottle entropy near conformal critical points

We show that the Klein bottle entropy [Phys. Rev. Lett. 119, 261603 (2017)] for conformal field theories (CFTs) perturbed by a relevant operator is a universal function of the dimensionless coupling constant. The universal scaling of the Klein bottle entropy near criticality provides an efficient approach to extract the scaling dimension of lattice operators via data collapse. As paradigmatic examples, we validate the universal scaling of the Klein bottle entropy for Ising and Z3 parafermion CFTs with various perturbations using numerical simulation with continuous matrix product operator approach.

cond-mat.str-el↗

arXiv2023

A concise review on some Higgs-related new physics models in light of current experiments

Higgs boson may serve as a portal to new physics beyond the standard model (BSM) which is implied by theoretical naturalness or experimental anomalies. In this note we briefly survey some Higgs-related BSM physics models, including the low energy SUSY (focusing on the minimal SUSY model), the little Higgs models, the two-Higgs-doublet models and the simplest singlet extensions of the Higgs sector. For each illustrated BSM model, we emphatically elucidate its phenomenological power in explaining current measurements of the muon g-2, the W-boson mass and the dark matter. For the singlet extensions like the xSM and 2HDM+S, we discuss the induced cosmic phase transition and the dark matter relic density as well as the vacuum stability. Finally, we give an outlook.

hep-ph↗

arXiv2023

Inflation, electroweak phase transition, and Higgs searches at the LHC in the two-Higgs-doublet model

Combining the electroweak phase transition and the Higgs searches at the LHC, we study the Higgs inflation in the type-I and type-II two-Higgs-doublet models with non-minimally couplings to gravity. Considering relevant theoretical and experimental constraints, we find that the Higgs inflation imposes stringent constraints on the mass splitting between $A$, $H^\pm$ and $H$, and they tend to be nearly degenerate in mass with increasing of their masses. The direct searches for Higgs at the LHC can exclude many points achieving Higgs inflation in the region of $m_H~(m_A)<$ 450 GeV in the type-I model, and impose a lower bound on $\tanβ$ for the type-II model. The Higgs inflation disfavors the wrong sign Yukawa coupling region of type-II model. In the parameter space achieving the Higgs inflation, the type-I and type-II models can produce a first order electroweak phase transition, but $v_c/T_c$ is much smaller than 1.0.

hep-ph↗

arXiv2023

Two-Higgs-doublet models in light of current experiments: a brief review

We briefly survey several typical CP-conserving two-Higgs-doublet models (2HDMs) in light of current experiments. First we derive the masses and couplings of the mass eigenstates from the Lagrangians. Then we analyze the constraints from theory and oblique electroweak parameters. Finally, we delineate the status of 2HDM in light of the LHC searches, the dark matter detections and the muon $g-2$ measurement.

hep-ph↗

arXiv2023

Zero-Shot Next-Item Recommendation using Large Pretrained Language Models

Large language models (LLMs) have achieved impressive zero-shot performance in various natural language processing (NLP) tasks, demonstrating their capabilities for inference without training examples. Despite their success, no research has yet explored the potential of LLMs to perform next-item recommendations in the zero-shot setting. We have identified two major challenges that must be addressed to enable LLMs to act effectively as recommenders. First, the recommendation space can be extremely large for LLMs, and LLMs do not know about the target user's past interacted items and preferences. To address this gap, we propose a prompting strategy called Zero-Shot Next-Item Recommendation (NIR) prompting that directs LLMs to make next-item recommendations. Specifically, the NIR-based strategy involves using an external module to generate candidate items based on user-filtering or item-filtering. Our strategy incorporates a 3-step prompting that guides GPT-3 to carry subtasks that capture the user's preferences, select representative previously watched movies, and recommend a ranked list of 10 movies. We evaluate the proposed approach using GPT-3 on MovieLens 100K dataset and show that it achieves strong zero-shot performance, even outperforming some strong sequential recommendation models trained on the entire training dataset. These promising results highlight the ample research opportunities to use LLMs as recommenders. The code can be found at https://github.com/AGI-Edgerunners/LLM-Next-Item-Rec.

cs.IR↗

arXiv2023

METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens

In clinical scenarios, multi-specialist consultation could significantly benefit the diagnosis, especially for intricate cases. This inspires us to explore a "multi-expert joint diagnosis" mechanism to upgrade the existing "single expert" framework commonly seen in the current literature. To this end, we propose METransformer, a method to realize this idea with a transformer-based backbone. The key design of our method is the introduction of multiple learnable "expert" tokens into both the transformer encoder and decoder. In the encoder, each expert token interacts with both vision tokens and other expert tokens to learn to attend different image regions for image representation. These expert tokens are encouraged to capture complementary information by an orthogonal loss that minimizes their overlap. In the decoder, each attended expert token guides the cross-attention between input words and visual tokens, thus influencing the generated report. A metrics-based expert voting strategy is further developed to generate the final report. By the multi-experts concept, our model enjoys the merits of an ensemble-based approach but through a manner that is computationally more efficient and supports more sophisticated interactions among experts. Experimental results demonstrate the promising performance of our proposed model on two widely used benchmarks. Last but not least, the framework-level innovation makes our work ready to incorporate advances on existing "single-expert" models to further improve its performance.

cs.CV↗

arXiv2023

Phase Field Model for Non-equilibrium Interface Conditions

This article presents a new phase-field formulation for non-equilibrium interface conditions in rapid phase transformations. With a particular way of defining concentration fields, the classical sharp and diffuse (thick) interface theories are unified into the present phase-field model. Each point inside the diffuse interface is comparable to the classical sharp interface model, governed by phase boundary migration and short-range atomic exchanges and connected by long-range diffusion. Their thermodynamic forces and fluxes are following the Onsager reciprocal relation and consistent with the classical irreversible thermodynamics. Furthermore, we establish the balance of driving forces and energy dissipations for both the diffuse interface and the single point inside it. The model is then validated by rapid solidification of Al-Cu alloys. With an effective mobility of considering non-equilibrium long-range diffusion, the model represents the main characteristics of non-equilibrium interface kinetics, i.e., solute trapping and solute drag. Especially, we reproduce the complete trapping at the high-velocity regime and provide a thermodynamic consistent description of the partial drag phenomena at the low-velocity region. Moreover, the dissipation analysis indicates that even the far-from-equilibrium diffuse interface is composed of numerous representative volume elements near the equilibrium, providing a new understanding of non-equilibrium interfaces.

cond-mat.mtrl-sci↗

arXiv2023

Smoothing Gradient Tracking for Decentralized Optimization over the Stiefel Manifold with Non-smooth Regularizers

Recently, decentralized optimization over the Stiefel manifold has attacked tremendous attentions due to its wide range of applications in various fields. Existing methods rely on the gradients to update variables, which are not applicable to the objective functions with non-smooth regularizers, such as sparse PCA. In this paper, to the best of our knowledge, we propose the first decentralized algorithm for non-smooth optimization over Stiefel manifolds. Our algorithm approximates the non-smooth part of objective function by its Moreau envelope, and then existing algorithms for smooth optimization can be deployed. We establish the convergence guarantee with the iteration complexity of $\mathcal{O} (ε^{-4})$. Numerical experiments conducted under the decentralized setting demonstrate the effectiveness and efficiency of our algorithm.

math.OC↗

arXiv2023

3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition

Many skeletal action recognition models use GCNs to represent the human body by 3D body joints connected body parts. GCNs aggregate one- or few-hop graph neighbourhoods, and ignore the dependency between not linked body joints. We propose to form hypergraph to model hyper-edges between graph nodes (e.g., third- and fourth-order hyper-edges capture three and four nodes) which help capture higher-order motion patterns of groups of body joints. We split action sequences into temporal blocks, Higher-order Transformer (HoT) produces embeddings of each temporal block based on (i) the body joints, (ii) pairwise links of body joints and (iii) higher-order hyper-edges of skeleton body joints. We combine such HoT embeddings of hyper-edges of orders 1, ..., r by a novel Multi-order Multi-mode Transformer (3Mformer) with two modules whose order can be exchanged to achieve coupled-mode attention on coupled-mode tokens based on 'channel-temporal block', 'order-channel-body joint', 'channel-hyper-edge (any order)' and 'channel-only' pairs. The first module, called Multi-order Pooling (MP), additionally learns weighted aggregation along the hyper-edge mode, whereas the second module, Temporal block Pooling (TP), aggregates along the temporal block mode. Our end-to-end trainable network yields state-of-the-art results compared to GCN-, transformer- and hypergraph-based counterparts.

cs.CV↗

arXiv2023

Projected d-wave superconducting state: a fermionic projected entangled pair state study

We investigate the physics of projected d-wave pairing states using their fermionic projected entangled pair state (fPEPS) representation. First, we approximate a d-wave Bardeen-Cooper-Schrieffer state using the Gaussian fPEPS. Next, we translate the resulting state into fPEPS tensors and implement the Gutzwiller projection which removes double occupancy by modifying the local tensor elements. The tensor network representation of the projected d-wave pairing state allows us to evaluate physical quantities in the thermodynamic limit without employing the Gutzwiller approximation. Despite having very few variational parameters, such physically motivated tensor network states are shown to exhibit competitive energies for the doped t-J model. We expect that such construction offers useful initial states and guidance for variational tensor network calculations.

cond-mat.str-el↗

arXiv2023

Robust Eigenvectors of Regular Simplex Tensors: Conjecture Proof

The concept of tensor eigenpairs has received more researches in past decades. Recent works have paid attentions to a special class of symmetric tensors termed regular simplex tensors, which is constructed by equiangular tight frame of n + 1 vectors in n-dimensional space, and the robustness of eigenpairs was investigated. In the end of the literature, a conjecture was claimed that the robust eigenvectors of a regular simplex tensor are precisely the vectors in the frame. One later work theoretically proved that the case of n = 2 was true. In this paper, we proceed further and complete the proof for the above conjecture. Some promising directions are discussed in the end for future works.

math.SP↗

arXiv2023

Unsupervised Domain Generalization for Person Re-identification: A Domain-specific Adaptive Framework

Domain generalization (DG) has attracted much attention in person re-identification (ReID) recently. It aims to make a model trained on multiple source domains generalize to an unseen target domain. Although achieving promising progress, existing methods usually need the source domains to be labeled, which could be a significant burden for practical ReID tasks. In this paper, we turn to investigate unsupervised domain generalization for ReID, by assuming that no label is available for any source domains. To address this challenging setting, we propose a simple and efficient domain-specific adaptive framework, and realize it with an adaptive normalization module designed upon the batch and instance normalization techniques. In doing so, we successfully yield reliable pseudo-labels to implement training and also enhance the domain generalization capability of the model as required. In addition, we show that our framework can even be applied to improve person ReID under the settings of supervised domain generalization and unsupervised domain adaptation, demonstrating competitive performance with respect to relevant methods. Extensive experimental study on benchmark datasets is conducted to validate the proposed framework. A significance of our work lies in that it shows the potential of unsupervised domain generalization for person ReID and sets a strong baseline for the further research on this topic.

cs.CV↗

arXiv2023

Particle production at large $p_T$ in Xe+Xe collisions with jet quenching using the higher twist approach

The production of $π^0$, $η$, and $ϕ$ in the most central (0%-5%) Xe+Xe collisions at $\sqrt{s_{NN}}$ = 5.44 TeV is investigated in the framework of the perturbative QCD (pQCD) improved parton model at an accuracy of next-to-leading order (NLO). The jet quenching effect is effectively incorporated by medium-modified fragmentation functions via the higher-twist approach. Predictions of the nuclear modification factors of $π^0$, $η$, and $ϕ$ as functions of the transverse momentum $p_{\rm T}$ are made with the jet transport parameter $\hat{q}_0$, which is extracted from the available experimental data of charged hadrons provided by ALICE and CMS. The particle ratios $η/π^0,\ ϕ/π^0$ as functions of $p_{\rm T}$ in Xe+Xe collisions at $\sqrt{s_{NN}}$ = 5.44 TeV as well as in 0%-5% Pb+Pb collisions at $\sqrt{s_{NN}}$ = 5.02 TeV are also presented. The numerical simulations of the scaled ratios of charged hadron production in the Xe+Xe 5.44 TeV system over those in the Pb+Pb 5.02 TeV system give a good description of the CMS data, and the scaled ratios of $π^0$, $η$, and $ϕ$ production coincide with the curve of charged hadron production.

hep-ph↗

arXiv2023

Thermodynamics of Non-equilibrium Diffuse-Interfaces in Mesoscale Phase Transformations

We present a new phase-field formulation for the non-equilibrium interface kinetics. The diffuse interface is considered an integral of numerous representative volume elements (RVEs), in which there is a two-phase mixture with two conserved and two non-conserved concentration fields. This particular way of separating concentration fields leads to two distinct dissipative processes besides the phase boundary migration, i.e., trans-sharp-interface diffusion between different phases within a single RVE and trans-diffuse-interface diffusion between different RVEs within a single phase. A two-part mechanism is proposed to balance the driving forces and energy dissipation for diffusionless and diffusional processes in RVEs. Moreover, the classical interface friction and solute drag can be recovered by integrating the diffuse interface. Compared with the sharp interface theory, the present diffuse interface can reproduce the atomic simulated partial-drag self-consistently in thermodynamics.

cond-mat.mtrl-sci↗

arXiv2023

High-level semantic feature matters few-shot unsupervised domain adaptation

In few-shot unsupervised domain adaptation (FS-UDA), most existing methods followed the few-shot learning (FSL) methods to leverage the low-level local features (learned from conventional convolutional models, e.g., ResNet) for classification. However, the goal of FS-UDA and FSL are relevant yet distinct, since FS-UDA aims to classify the samples in target domain rather than source domain. We found that the local features are insufficient to FS-UDA, which could introduce noise or bias against classification, and not be used to effectively align the domains. To address the above issues, we aim to refine the local features to be more discriminative and relevant to classification. Thus, we propose a novel task-specific semantic feature learning method (TSECS) for FS-UDA. TSECS learns high-level semantic features for image-to-class similarity measurement. Based on the high-level features, we design a cross-domain self-training strategy to leverage the few labeled samples in source domain to build the classifier in target domain. In addition, we minimize the KL divergence of the high-level feature distributions between source and target domains to shorten the distance of the samples between the two domains. Extensive experiments on DomainNet show that the proposed method significantly outperforms SOTA methods in FS-UDA by a large margin (i.e., 10%).

cs.CV↗