arXiv subjects

Xiang Liu

Publications and source records attributed to Xiang Liu.

At least 199 records · Page 11Linked to original sources

arXiv2024

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning

The machine learning community has witnessed impressive advancements since large language models (LLMs) first appeared. Yet, their massive memory consumption has become a significant roadblock to large-scale training. For instance, a 7B model typically requires at least 60 GB of GPU memory with full parameter training, which presents challenges for researchers without access to high-resource environments. Parameter Efficient Fine-Tuning techniques such as Low-Rank Adaptation (LoRA) have been proposed to alleviate this problem. However, in most large-scale fine-tuning settings, their performance does not reach the level of full parameter training because they confine the parameter search to a low-rank subspace. Attempting to complement this deficiency, we investigate the layerwise properties of LoRA on fine-tuning tasks and observe an unexpected but consistent skewness of weight norms across different layers. Utilizing this key observation, a surprisingly simple training strategy is discovered, which outperforms both LoRA and full parameter training in a wide range of settings with memory costs as low as LoRA. We name it Layerwise Importance Sampled AdamW (LISA), a promising alternative for LoRA, which applies the idea of importance sampling to different layers in LLMs and randomly freezes most middle layers during optimization. Experimental results show that with similar or less GPU memory consumption, LISA surpasses LoRA or even full parameter tuning in downstream fine-tuning tasks, where LISA consistently outperforms LoRA by over 10%-35% in terms of MT-Bench score while achieving on-par or better performance in MMLU, AGIEval and WinoGrande. On large models, specifically LLaMA-2-70B, LISA surpasses LoRA on MT-Bench, GSM8K, and PubMedQA, demonstrating its effectiveness across different domains.

cs.LG↗

arXiv2024

Production potential of hidden-strange molecular pentaquarks through the $π^-p\rightarrow K^{*}Σ$ process

In this study, applying the effective Lagrangian approach, we investigate the reaction $π^-p\rightarrow K^{*}Σ$ to explore the production of the hidden-strange molecular pentaquarks $P_{s\bar{s}}$. Specifically, we analyze two scenarios, where the $J^P$ quantum numbers of $P_{s\bar{s}}$ are $3/2^-$ and $1/2^-$. Our results show that the contribution from $s$-channel $P_{s\bar{s}}$ exchange dominates the total cross section, surpassing those from $t$-channel $K^{(*)}$ exchange and $u$-channel $Σ$ exchange. Additionally, we present predictions for the differential cross section of the $π^-p\rightarrow K^{*}Σ$ process. Finally, we extend the 2-to-2 scattering process to the 2-to-3 Dalitz process and provide theoretical predictions for this scenario. Our findings suggest that the cross section for the $π^-p \rightarrow K^{*}Σ\rightarrow K^+π^-Σ^0$ process can reach several tens of $μb$ near the threshold, making it highly favorable for experimental measurement. However, the current scarcity of experimental data for the $π^-p\rightarrow K^{*}Σ$ reaction at threshold energy limits our ability to determine the properties of the hidden-strange molecular pentaquarks $P_{s\bar{s}}$ and related physical quantities. Therefore, additional experimental measurements of the $π^-p \rightarrow K^{*}Σ$ reaction at threshold are strongly encouraged, and correlation studies could be pursued at facilities such as J-PARC, AMBER, and upcoming HIKE and HIAF meson beam experiments.

hep-ph↗

arXiv2024

WATER-GS: Toward Copyright Protection for 3D Gaussian Splatting via Universal Watermarking

3D Gaussian Splatting (3DGS) has emerged as a pivotal technique for 3D scene representation, providing rapid rendering speeds and high fidelity. As 3DGS gains prominence, safeguarding its intellectual property becomes increasingly crucial since 3DGS could be used to imitate unauthorized scene creations and raise copyright issues. Existing watermarking methods for implicit NeRFs cannot be directly applied to 3DGS due to its explicit representation and real-time rendering process, leaving watermarking for 3DGS largely unexplored. In response, we propose WATER-GS, a novel method designed to protect 3DGS copyrights through a universal watermarking strategy. First, we introduce a pre-trained watermark decoder, treating raw 3DGS generative modules as potential watermark encoders to ensure imperceptibility. Additionally, we implement novel 3D distortion layers to enhance the robustness of the embedded watermark against common real-world distortions of point cloud data. Comprehensive experiments and ablation studies demonstrate that WATER-GS effectively embeds imperceptible and robust watermarks into 3DGS without compromising rendering efficiency and quality. Our experiments indicate that the 3D distortion layers can yield up to a 20% improvement in accuracy rate. Notably, our method is adaptable to different 3DGS variants, including 3DGS compression frameworks and 2D Gaussian splatting.

cs.CR↗

arXiv2024

Construction of Isoscalar $2^{-+}$ Mesonic States Inspired by the $X(2600)$ Discovery

The BESIII observation of the $X(2600)$ suggests that one possible assignment for its spin-parity quantum numbers is $J^{PC}=2^{-+}$. This finding motivates us to systematically revisit isoscalar pseudotensor mesonic states through a comprehensive spectroscopic study. In this work, we employ a potential model to calculate the mass spectrum of isoscalar pseudotensor meson states with varying radial quantum numbers. Additionally, we analyze their strong decay behaviors, as allowed by the Okubo-Zweig-Iizuka (OZI) rule, using the quark pair creation model. Our findings yield valuable information about their resonant parameters and partial decay widths, which can aid in identifying the observed isoscalar pseudotensor states when combined with current experimental data. Importantly, we also predict several radial excitations of isoscalar pseudotensor mesons that remain unobserved experimentally. Furthermore, we exclude the $X(2600)$ from being classified as an isoscalar pseudotensor meson, which is critical for elucidating the properties of the $X(2600)$.

hep-ph↗

arXiv2024

IntComplex for high-order interactions

Graphs serve as powerful tools for modeling pairwise interactions in diverse fields such as biology, material science, and social networks. However, they inherently overlook interactions involving more than two entities. Simplicial complexes and hypergraphs have emerged as prominent frameworks for modeling many-body interactions; nevertheless, they exhibit limitations in capturing specific high-order interactions, particularly those involving transitions from $n$-interactions to $m$-interactions. Addressing this gap, we propose IntComplex as an innovative framework to characterize such high-order interactions comprehensively. Our framework leverages homology theory to provide a quantitative representation of the topological structure inherent in such interactions. IntComplex is defined as a collection of interactions, each of which can be equivalently represented by a binary tree. Drawing inspiration from GLMY homology, we introduce homology for the detailed analysis of structural patterns formed by interactions across adjacent dimensions, $p$-layer homology to elucidate loop structures within $p$-interactions in specific dimensions, and multilayer homology to analyze loop structures of interactions across multiple dimensions. Furthermore, we introduce persistent homology through a filtration process and establish its stability to ensure robust quantitative analysis of these complex interactions. The proposed IntComplex framework establishes a foundational paradigm for the analysis of topological properties in high-order interactions, presenting significant potential to drive forward the advancements in the domain of complex network analysis.

math.AT↗

arXiv2024

S3TU-Net: Structured Convolution and Superpixel Transformer for Lung Nodule Segmentation

The irregular and challenging characteristics of lung adenocarcinoma nodules in computed tomography (CT) images complicate staging diagnosis, making accurate segmentation critical for clinicians to extract detailed lesion information. In this study, we propose a segmentation model, S3TU-Net, which integrates multi-dimensional spatial connectors and a superpixel-based visual transformer. S3TU-Net is built on a multi-view CNN-Transformer hybrid architecture, incorporating superpixel algorithms, structured weighting, and spatial shifting techniques to achieve superior segmentation performance. The model leverages structured convolution blocks (DWF-Conv/D2BR-Conv) to extract multi-scale local features while mitigating overfitting. To enhance multi-scale feature fusion, we introduce the S2-MLP Link, integrating spatial shifting and attention mechanisms at the skip connections. Additionally, the residual-based superpixel visual transformer (RM-SViT) effectively merges global and local features by employing sparse correlation learning and multi-branch attention to capture long-range dependencies, with residual connections enhancing stability and computational efficiency. Experimental results on the LIDC-IDRI dataset demonstrate that S3TU-Net achieves a DSC, precision, and IoU of 89.04%, 90.73%, and 90.70%, respectively. Compared to recent methods, S3TU-Net improves DSC by 4.52% and sensitivity by 3.16%, with other metrics showing an approximate 2% increase. In addition to comparison and ablation studies, we validated the generalization ability of our model on the EPDB private dataset, achieving a DSC of 86.40%.

eess.IV↗

arXiv2024

Refining radiative decay studies in singly heavy baryons

In this work, we systematically study the radiative decays of singly heavy baryons, a crucial aspect of their spectroscopic behavior. To enhance the accuracy of our calculations, we utilize numerical spatial wave functions for the singly heavy baryons obtained through the Gaussian expansion method, which also yields their mass spectrum. As hadron spectroscopy enters an era of high precision, we believe our study of the radiative decays of singly heavy baryons will provide valuable insights for further exploration of these particles.

hep-ph↗

arXiv2024

LongGenBench: Long-context Generation Benchmark

Current long-context benchmarks primarily focus on retrieval-based tests, requiring Large Language Models (LLMs) to locate specific information within extensive input contexts, such as the needle-in-a-haystack (NIAH) benchmark. Long-context generation refers to the ability of a language model to generate coherent and contextually accurate text that spans across lengthy passages or documents. While recent studies show strong performance on NIAH and other retrieval-based long-context benchmarks, there is a significant lack of benchmarks for evaluating long-context generation capabilities. To bridge this gap and offer a comprehensive assessment, we introduce a synthetic benchmark, LongGenBench, which allows for flexible configurations of customized generation context lengths. LongGenBench advances beyond traditional benchmarks by redesigning the format of questions and necessitating that LLMs respond with a single, cohesive long-context answer. Upon extensive evaluation using LongGenBench, we observe that: (1) both API accessed and open source models exhibit performance degradation in long-context generation scenarios, ranging from 1.2% to 47.1%; (2) different series of LLMs exhibit varying trends of performance degradation, with the Gemini-1.5-Flash model showing the least degradation among API accessed models, and the Qwen2 series exhibiting the least degradation in LongGenBench among open source models.

cs.CL↗

arXiv2024

Should We Really Edit Language Models? On the Evaluation of Edited Language Models

Model editing has become an increasingly popular alternative for efficiently updating knowledge within language models. Current methods mainly focus on reliability, generalization, and locality, with many methods excelling across these criteria. Some recent works disclose the pitfalls of these editing methods such as knowledge distortion or conflict. However, the general abilities of post-edited language models remain unexplored. In this paper, we perform a comprehensive evaluation on various editing methods and different language models, and have following findings. (1) Existing editing methods lead to inevitable performance deterioration on general benchmarks, indicating that existing editing methods maintain the general abilities of the model within only a few dozen edits. When the number of edits is slightly large, the intrinsic knowledge structure of the model is disrupted or even completely damaged. (2) Instruction-tuned models are more robust to editing, showing less performance drop on general knowledge after editing. (3) Language model with large scale is more resistant to editing compared to small model. (4) The safety of the edited model, is significantly weakened, even for those safety-aligned models. Our findings indicate that current editing methods are only suitable for small-scale knowledge updates within language models, which motivates further research on more practical and reliable editing methods. The details of code and reproduction can be found in https://github.com/lqinfdim/EditingEvaluation.

cs.AI↗

arXiv2024

FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation

Efficiently aggregating trained neural networks from local clients into a global model on a server is a widely researched topic in federated learning. Recently, motivated by diminishing privacy concerns, mitigating potential attacks, and reducing communication overhead, one-shot federated learning (i.e., limiting client-server communication into a single round) has gained popularity among researchers. However, the one-shot aggregation performances are sensitively affected by the non-identical training data distribution, which exhibits high statistical heterogeneity in some real-world scenarios. To address this issue, we propose a novel one-shot aggregation method with layer-wise posterior aggregation, named FedLPA. FedLPA aggregates local models to obtain a more accurate global model without requiring extra auxiliary datasets or exposing any private label information, e.g., label distributions. To effectively capture the statistics maintained in the biased local datasets in the practical non-IID scenario, we efficiently infer the posteriors of each layer in each local model using layer-wise Laplace approximation and aggregate them to train the global parameters. Extensive experimental results demonstrate that FedLPA significantly improves learning performance over state-of-the-art methods across several metrics.

cs.LG↗

arXiv2024

Gaussian Primitives for Deformable Image Registration

Deformable Image Registration (DIR) is essential for aligning medical images that exhibit anatomical variations, facilitating applications such as disease tracking and radiotherapy planning. While classical iterative methods and deep learning approaches have achieved success in DIR, they are often hindered by computational inefficiency or poor generalization. In this paper, we introduce GaussianDIR, a novel, case-specific optimization DIR method inspired by 3D Gaussian splatting. In general, GaussianDIR represents image deformations using a sparse set of mobile and flexible Gaussian primitives, each defined by a center position, covariance, and local rigid transformation. This compact and explicit representation reduces noise and computational overhead while improving interpretability. Furthermore, the movement of individual voxel is derived via blending the local rigid transformation of the neighboring Gaussian primitives. By this, GaussianDIR captures both global smoothness and local rigidity as well as reduces the computational burden. To address varying levels of deformation complexity, GaussianDIR also integrates an adaptive density control mechanism that dynamically adjusts the density of Gaussian primitives. Additionally, we employ multi-scale Gaussian primitives to capture both coarse and fine deformations, reducing optimization to local minima. Experimental results on brain MRI, lung CT, and cardiac MRI datasets demonstrate that GaussianDIR outperforms existing DIR methods in both accuracy and efficiency, highlighting its potential for clinical applications. Finally, as a training-free approach, it challenges the stereotype that iterative methods are inherently slow and transcend the limitations of poor generalization.

cs.CV↗

arXiv2024

LPZero: Language Model Zero-cost Proxy Search from Zero

In spite of the outstanding performance, Neural Architecture Search (NAS) is criticized for massive computation. Recently, Zero-shot NAS has emerged as a promising approach by exploiting Zero-cost (ZC) proxies, which markedly reduce computational demands. Despite this, existing ZC proxies heavily rely on expert knowledge and incur significant trial-and-error costs. Particularly in NLP tasks, most existing ZC proxies fail to surpass the performance of the naive baseline. To address these challenges, we introduce a novel framework, \textbf{LPZero}, which is the first to automatically design ZC proxies for various tasks, achieving higher ranking consistency than human-designed proxies. Specifically, we model the ZC proxy as a symbolic equation and incorporate a unified proxy search space that encompasses existing ZC proxies, which are composed of a predefined set of mathematical symbols. To heuristically search for the best ZC proxy, LPZero incorporates genetic programming to find the optimal symbolic composition. We propose a \textit{Rule-based Pruning Strategy (RPS),} which preemptively eliminates unpromising proxies, thereby mitigating the risk of proxy degradation. Extensive experiments on FlexiBERT, GPT-2, and LLaMA-7B demonstrate LPZero's superior ranking ability and performance on downstream tasks compared to current approaches.

cs.CL↗

arXiv2024

Likelihood analysis of the newly observed $f_0(2020)$, $f_0(2330)$, and $f_0(2470)$ in $J/ψ\to γη^\primeη^\prime$ as high-lying unflavored scalar mesons

Light mesons have always been an important part of the whole hadron family, once inspiring the SU(3) classification of hadrons as a typical example~\cite{Gell-Mann:1964ewy}. Until now, the search for light mesons has been an ongoing task in hadron physics. In particular, with the accumulation of experimental data, more and more light flavor mesons with the masses around 2.2 GeV were found, including the $Y(2175)$ observed in $J/ψ\to ηϕf_0(980)$~\cite{BES:2007sqy} and the $X(2370)$ reported in the decay of $J/ψ\to γη^\prime π^+π^-$~\cite{BESIII:2010gmv,BESIII:2019wkp}. These progresses show that there is a chance to construct the light meson family, which is also an effective approach to deepen our understanding of non-perturbative behavior of the strong interaction.

hep-ph↗

arXiv2024

Predicting possible molecular states of nucleons with $Ξ_c$, $Ξ_c^{*}$, and $Ξ_c^{\prime}$

In the framework of a one-boson-exchange model, we carry out a comprehensive investigation of the $Ξ_cN/Λ_cΣ/Ξ_c^{\prime}N/Σ_cΛ/Ξ_c^*N/Σ_c^*Λ/Σ_cΣ/Σ_c^*Σ$ interactions. We consider the $S$-$D$-wave mixing effects and the coupled-channel effects to derive the relevant effective potentials. Our results can predict several possible charm-strange deuteronlike $Ξ_c^{(',*)}N$ hexaquarks, the $Ξ_c^{\prime}N$ molecules with $I(I^P)=0(0^+)$, $0(1^+)$, $1(1^+)$, the $Ξ_c^*N$ molecules with $0(1^+)$, $0(2^+)$, $1(2^+)$, and the coupled $Ξ_cN/Ξ_c^{\prime}N/Ξ_c^*N/Σ_cΣ/Σ_c^*Σ$ molecule with $0(1^+)$. We expect the experiments to search for our predictions of the $Ξ_c^{(\prime,*)}N$ bound states.

hep-ph↗

arXiv2024

Exploring the mass spectrum and the electromagnetic properties of the possible $Ξ_{cc}K^{()}$ and $Ξ_{cc}\bar{K}^{()}$ molecules

Using the one-boson-exchange model, we investigate the interactions between the doubly charmed baryon $Ξ_{cc}(3621)$ and the $S-$wave (anti-)kaon accounting for the $S-D$ wave mixing and coupled-channel effects. We find the coupled $Ξ_{cc}K/Ξ_{cc}K^*$ state with $I(J^P)=0(1/2^-)$, the $Ξ_{cc}K^*$ state with $0(1/2^-)$, the $Ξ_{cc}\bar{K}$ state with $0(1/2^-)$, and the $Ξ_{cc}\bar{K}^*$ states with $0(1/2^-,3/2^-)$ can be recommended as good doubly charmed molecular candidates with strangeness $|S|=1$. We further examine their M1 radiative decay behaviors and magnetic moments within the constituent quark model framework. This information can enhance our understanding of their inner structures, including the distribution of electric charge and the orientation of the constituent quarks' spins.

hep-ph↗

arXiv2024

DIIT: A Domain-Invariant Information Transfer Method for Industrial Cross-Domain Recommendation

Cross-Domain Recommendation (CDR) have received widespread attention due to their ability to utilize rich information across domains. However, most existing CDR methods assume an ideal static condition that is not practical in industrial recommendation systems (RS). Therefore, simply applying existing CDR methods in the industrial RS environment may lead to low effectiveness and efficiency. To fill this gap, we propose DIIT, an end-to-end Domain-Invariant Information Transfer method for industrial cross-domain recommendation. Specifically, We first simulate the industrial RS environment that maintains respective models in multiple domains, each of them is trained in the incremental mode. Then, for improving the effectiveness, we design two extractors to fully extract domain-invariant information from the latest source domain models at the domain level and the representation level respectively. Finally, for improving the efficiency, we design a migrator to transfer the extracted information to the latest target domain model, which only need the target domain model for inference. Experiments conducted on one production dataset and two public datasets verify the effectiveness and efficiency of DIIT.

cs.IR↗

arXiv2024

Possible molecules of triple-heavy pentaquarks within the extended local hidden gauge formalism

In this study, we explore the interactions between mesons and baryons in the open heavy sectors to identify potential triple-heavy molecular pentaquarks. We derive the meson-baryon interaction potentials using the vector meson exchange mechanism within the extended local hidden gauge formalism. The scattering amplitudes are computed by solving the coupled-channel Bethe-Salpeter equation, revealing several bound systems. By analyzing the poles of these amplitudes in the complex plane, we determine the masses and widths of these bound states. Additionally, we evaluate the couplings and compositeness of different channels within each bound system to assess their molecular characteristics. Our predictions include four $Ω_{ccc}$-like states, four $Ω_{bbb}$-like states, fourteen $Ω_{bcc}$-like states, and ten $Ω_{bbc}$-like states, which could be targets for future experimental investigations.

hep-ph↗

arXiv2024

Probing exotic $J^{PC}$ resonances from deeply bound charmoniumlike molecules: Insights for identifying exotic hadrons

We investigate the resonance phenomenon in deeply bound charmoniumlike states. By interpreting the $Y(4220)$, $Y(4360)$, and $ψ(4415)$ states as deeply $S$-wave bound $D \bar D_1$, $D^* \bar D_1$, and $D^* \bar D_2^*$ molecules, respectively, we robustly predict the existence of $4$ $S$-wave deeply bound $C$-parity partner states and $13$ $P$-wave charmoniumlike molecular resonances. These states, formed by exchanging the light mesons and pinpointed through the complex scaling method, include $3$ $S$-wave bound states with exotic quantum numbers $I(J^{PC})$= $0(1^{-+})$ and $5$ $P$-wave resonances with exotic $0(0^{-+})$ and $0(2^{+-})$. We expect future experiments to search for such exotic molecular resonances, which will not only enrich the ongoing construction of ``Particle Zoo 2.0" but also help to scrutinize the hadronic molecular assignments of the $Y$-family particles including the $Y(4220)$, $Y(4360)$, and $ψ(4415)$ states.

hep-ph↗