arXiv subjects

Lei Wang

Publications and source records attributed to Lei Wang.

At least 307 records · Page 17Linked to original sources

arXiv2025

FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA

Fine-tuning large language models (LLMs) in federated settings enables privacy-preserving adaptation but suffers from cross-client interference due to model aggregation. Existing federated LoRA fine-tuning methods, primarily based on FedAvg, struggle with data heterogeneity, leading to harmful cross-client interference and suboptimal personalization. In this work, we propose \textbf{FedALT}, a novel personalized federated LoRA fine-tuning algorithm that fundamentally departs from FedAvg. Instead of using an aggregated model to initialize local training, each client continues training its individual LoRA while incorporating shared knowledge through a separate Rest-of-World (RoW) LoRA component. To effectively balance local adaptation and global information, FedALT introduces an adaptive mixer that dynamically learns input-specific weightings between the individual and RoW LoRA components, drawing conceptual foundations from the Mixture-of-Experts (MoE) paradigm. Through extensive experiments on NLP benchmarks, we demonstrate that FedALT significantly outperforms state-of-the-art personalized federated LoRA fine-tuning methods, achieving superior local adaptation without sacrificing computational efficiency.

cs.LG↗

arXiv2025

Scalable data-driven modeling of microstructure evolution by learning local dependency and spatiotemporal translation invariance rules in phase field simulation

Phase-field (PF) simulation provides a powerful framework for predicting microstructural evolution but suffers from prohibitive computational costs that severely limit accessible spatiotemporal scales in practical applications. While data-driven methods have emerged as promising approaches for accelerating PF simulations, existing methods require extensive training data from numerous evolution trajectories, and their inherent black-box nature raises concerns about long-term prediction reliability. This work demonstrates, through examples of grain growth and spinodal decomposition, that a minimalist Convolutional Neural Network (CNN) trained with a remarkably small dataset even from a single small-scale simulation can achieve seamless scalability to larger systems and reliable long-term predictions far beyond the temporal range of the training data. The key insight of this work lies in revealing that the success of CNN-based models stems from the alignment between their inductive biases and the physical priors of phase-field simulations specifically, locality and spatiotemporal translation invariance. Through effective receptive field analysis, we verify that the model captures these essential properties during training. Therefore, from a reductionist perspective, the surrogate model essentially establishes a spatiotemporally invariant regression mapping between a grid point's local environment and its subsequent state. Further analysis of the model's feature space demonstrates that microstructural evolution effectively represents a continuous redistribution of a finite set of local environments. When the model has already encountered nearly all possible local environments in the early-stage training data, it can reliably generalize to much longer evolution timescales, regardless of the dramatic changes in global microstructural morphology.

cond-mat.mtrl-sci↗

arXiv2025

Mock Observations for the CSST Mission: Main Surveys -- the Mock Catalogue

The Chinese Space Station Survey Telescope (CSST) is a flagship space mission, designed to carry out a large-area sky survey to explore the nature of dark matter and dark energy in the Universe. The onboard multi-band imaging and slitless spectroscopic modules will enable us to obtain photometric data for billions of galaxies and stars, as well as hundreds of millions of spectroscopic measurements, advancing various scientific analyses such as galaxy clustering and weak gravitational lensing. To support the image simulations for the main survey of the CSST mission, we present a mock catalogue of stars and galaxies. For stars, the mock catalogue is generated using either Galaxia or TRILEGAL, both of which provide a range of stellar properties to meet the requirements of CSST image simulations. For galaxies, we built a mock light-cone up to redshift z~3.5 from the cosmological Nbody simulation and populated the mock galaxy catalogue from the dark mater haloes using a semi-analytical galaxy formation model. We then performed a full-sky ray-tracing simulation of weak gravitational lensing to obtain lensing shear at the position of each galaxy in the light-cone. To support both multi-band imaging and slitless spectroscopic simulations, we computed the spectral energy distribution (SED) for each galaxy based on its star formation history using a supervised deep-learning model and determined the magnitudes in each band using the CSST throughputs. Finally, the properties of our mock galaxies include positions, redshifts, stellar masses, shapes, sizes, SEDs, lensing shears and magnifications. We have validated our mock catalogue against observational data and theoretical models, with results showing good overall agreement. The catalogue provides a flexible dataset for the development of CSST image processing and can support a wide range of cosmological analyses within the CSST mission.

astro-ph.IM↗

arXiv2025

fiDrizzle-MU: A Fast Iterative Drizzle with Multiplicative Updates

We propose fiDrizzleMU, an algorithm for co-adding exposures via iterative multiplicative updates, replacing the additive correction framework. This method achieves superior anti-aliasing and noise reduction in stacked images. When applied to James Webb Space Telescope data, the fiDrizzleMU algorithm reconstructs a gravitational lensing candidate that was significantly blurred by the pipeline's resampling process. This enables the accurate recovery of faint and extended structures in high-resolution astronomical imaging.

astro-ph.IM↗

arXiv2025

CometNet: Contextual Motif-guided Long-term Time Series Forecasting

Long-term Time Series Forecasting is crucial across numerous critical domains, yet its accuracy remains fundamentally constrained by the receptive field bottleneck in existing models. Mainstream Transformer- and Multi-layer Perceptron (MLP)-based methods mainly rely on finite look-back windows, limiting their ability to model long-term dependencies and hurting forecasting performance. Naively extending the look-back window proves ineffective, as it not only introduces prohibitive computational complexity, but also drowns vital long-term dependencies in historical noise. To address these challenges, we propose CometNet, a novel Contextual Motif-guided Long-term Time Series Forecasting framework. CometNet first introduces a Contextual Motif Extraction module that identifies recurrent, dominant contextual motifs from complex historical sequences, providing extensive temporal dependencies far exceeding limited look-back windows; Subsequently, a Motif-guided Forecasting module is proposed, which integrates the extracted dominant motifs into forecasting. By dynamically mapping the look-back window to its relevant motifs, CometNet effectively harnesses their contextual information to strengthen long-term forecasting capability. Extensive experimental results on eight real-world datasets have demonstrated that CometNet significantly outperforms current state-of-the-art (SOTA) methods, particularly on extended forecast horizons.

cs.CE↗

arXiv2025

Free-T2M: Robust Text-to-Motion Generation for Humanoid Robots via Frequency-Domain

Enabling humanoid robots to synthesize complex, physically coherent motions from natural language commands is a cornerstone of autonomous robotics and human-robot interaction. While diffusion models have shown promise in this text-to-motion (T2M) task, they often generate semantically flawed or unstable motions, limiting their applicability to real-world robots. This paper reframes the T2M problem from a frequency-domain perspective, revealing that the generative process mirrors a hierarchical control paradigm. We identify two critical phases: a semantic planning stage, where low-frequency components establish the global motion trajectory, and a fine-grained execution stage, where high-frequency details refine the movement. To address the distinct challenges of each phase, we introduce Frequency enhanced text-to-motion (Free-T2M), a framework incorporating stage-specific frequency-domain consistency alignment. We design a frequency-domain temporal-adaptive module to modulate the alignment effects of different frequency bands. These designs enforce robustness in the foundational semantic plan and enhance the accuracy of detailed execution. Extensive experiments show our method dramatically improves motion quality and semantic correctness. Notably, when applied to the StableMoFusion baseline, Free-T2M reduces the FID from 0.152 to 0.060, establishing a new state-of-the-art within diffusion architectures. These findings underscore the critical role of frequency-domain insights for generating robust and reliable motions, paving the way for more intuitive natural language control of robots.

cs.CV↗

arXiv2025

Mock Observations for the CSST Mission: Multi-Channel Imager--The Cluster Field

The Multi-Channel Imager (MCI), one of the instruments aboard the China Survey Space Telescope (CSST), is designed to simultaneously observe the sky in three filters, covering wavelengths from the near-ultraviolet (NUV) to the near-infrared (NIR). With its large field of view ($7.5^{\prime}\times7.5^{\prime}$), MCI is particularly well-suited for observing galaxy clusters, providing a powerful tool for investigating galaxy evolution, dark matter and dark energy through gravitational lensing. Here we present a comprehensive simulation framework of a strong lensing cluster as observed by MCI, aiming to fully exploit its capabilities in capturing lensing features. The framework simulates a strong lensing cluster from the CosmoDC2 catalog, calculating the gravitational potential and performing ray-tracing to derive the true positions, shapes and light distribution of galaxies within the cluster field. Additionally, the simulation incorporates intra-cluster light (ICL) and spectral energy distributions (SEDs), enabling further strong lensing analyses, such as ICL seperation from galaxy light and mass reconstruction combining strong and weak lensing measurements. This framework provides a critical benchmark for testing the MCI data pipeline and maximizing its potential in galaxy cluster research.

astro-ph.IM↗

arXiv2025

Mock Observations for the CSST Mission: Main Surveys--An Overview of Framework and Simulation Suite

The Chinese Space Station Survey Telescope (CSST) is a flagship space-based observatory. Its main survey camera is designed to conduct high spatial resolution near-ultraviolet to near-infrared imaging and low-resolution spectroscopic surveys. To maximize the scientific output of CSST, we have developed a comprehensive, high-fidelity simulation pipeline for reproducing both imaging and spectroscopic observations. This paper presents an overview of the simulation framework, detailing its implementation and components. Built upon the GalSim package and incorporating the latest CSST instrumental specifications, our pipeline generates pixel-level mock observations that closely replicate the expected instrumental and observational conditions. The simulation suite integrates realistic astrophysical object catalogs, instrumental effects, point spread function (PSF) modeling, and observational noises to produce accurate synthetic data. We describe the key processing stages of the simulation, from constructing the input object catalogs to modeling the telescope optics and detector responses. Furthermore, we introduce the most recent release of simulated datasets, which provide a crucial testbed for data processing pipeline developments, calibration strategies, and scientific analyses, ensuring that CSST will meet its stringent requirements. Our pipeline serves as a vital tool for optimizing CSST main survey strategies and ensuring robust cosmological measurements.

astro-ph.IM↗

arXiv2025

What Makes Reasoning Invalid: Echo Reflection Mitigation for Large Language Models

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of reasoning tasks. Recent methods have further improved LLM performance in complex mathematical reasoning. However, when extending these methods beyond the domain of mathematical reasoning to tasks involving complex domain-specific knowledge, we observe a consistent failure of LLMs to generate novel insights during the reflection stage. Instead of conducting genuine cognitive refinement, the model tends to mechanically reiterate earlier reasoning steps without introducing new information or perspectives, a phenomenon referred to as "Echo Reflection". We attribute this behavior to two key defects: (1) Uncontrollable information flow during response generation, which allows premature intermediate thoughts to propagate unchecked and distort final decisions; (2) Insufficient exploration of internal knowledge during reflection, leading to repeating earlier findings rather than generating new cognitive insights. Building on these findings, we proposed a novel reinforcement learning method termed Adaptive Entropy Policy Optimization (AEPO). Specifically, the AEPO framework consists of two major components: (1) Reflection-aware Information Filtration, which quantifies the cognitive information flow and prevents the final answer from being affected by earlier bad cognitive information; (2) Adaptive-Entropy Optimization, which dynamically balances exploration and exploitation across different reasoning stages, promoting both reflective diversity and answer correctness. Extensive experiments demonstrate that AEPO consistently achieves state-of-the-art performance over mainstream reinforcement learning baselines across diverse benchmarks.

cs.AI↗

arXiv2025

Input-Output Data-Driven Stabilization of Continuous-Time Linear MIMO Systems

In this paper, we address the problem of data-driven stabilization of continuous-time multi-input multi-output (MIMO) linear time-invariant systems using the input-output data collected from an experiment. Building on recent results for data-driven output-feedback control based on non-minimal realizations, we propose an approach that can be applied to a broad class of continuous-time MIMO systems without requiring a uniform observability index. The key idea is to show that Kreisselmeier's adaptive filter can be interpreted as an observer of a stabilizable non-minimal realization of the plant. Then, by postprocessing the input-output data with such a filter, we derive a linear matrix inequality that yields the feedback gain of a dynamic output-feedback stabilizer.

eess.SY↗

arXiv2025

Ferroelectrically Controlled Chirality Switching of Weyl Quasiparticles

Weyl quasiparticles, as gapless low-energy excitations with nontrivial chirality, have garnered extensive interest in recent years. However, archieving effective and reversible control over their chirality (topological charge) remains a major challeng due to topological protection. In this work, we propose a ferroelectric mechanism to switch the chirality of Weyl phonons, where the reversal of ferroelectric polarization is intrinsically coupled to a simultaneous reversal of the chirality of Weyl points. This enables electric-field-driven control over the topological properties of phonon excitations. Through a comprehensive symmetry analysis of polar space groups, we identify 27 groups capable of hosting symmetry-protected Weyl phonons with chiral charges $C = 1$, $2$, and $3$, whose chirality can be reversed via polarization switching. The first-principles calculations are performed to screen feasible material candidates for each type of chirality, yielding a set of prototypical ferroelectric compounds that realize the proposed mechanism. As a representative example, K$_2$ZnBr$_4$ hosts the minimal configuration of two pairs of Weyl phonons. Upon polarization reversal, the chirality of all Weyl points is inverted, accompanied by a reversal of associated topological features such as Berry curvature and surface arcs. These findings provide a viable pathway for dynamic, electrical control of topological band crossings and open new avenues for chirality-based phononic applications.

cond-mat.mtrl-sci↗

arXiv2025

Dynamic Forgetting and Spatio-Temporal Periodic Interest Modeling for Local-Life Service Recommendation

In the context of the booming digital economy, recommendation systems, as a key link connecting users and numerous services, face challenges in modeling user behavior sequences on local-life service platforms, including the sparsity of long sequences and strong spatio-temporal dependence. Such challenges can be addressed by drawing an analogy to the forgetting process in human memory. This is because users' responses to recommended content follow the recency effect and the cyclicality of memory. By exploring this, this paper introduces the forgetting curve and proposes Spatio-Temporal periodic Interest Modeling (STIM) with long sequences for local-life service recommendation. STIM integrates three key components: a dynamic masking module based on the forgetting curve, which is used to extract both recent spatiotemporal features and periodic spatiotemporal features; a query-based mixture of experts (MoE) approach that can adaptively activate expert networks under different dynamic masks, enabling the collaborative modeling of time, location, and items; and a hierarchical multi-interest network unit, which captures multi-interest representations by modeling the hierarchical interactions between the shallow and deep semantics of users' recent behaviors. By introducing the STIM method, we conducted online A/B tests and achieved a 1.54\% improvement in gross transaction volume (GTV). In addition, extended offline experiments also showed improvements. STIM has been deployed in a large-scale local-life service recommendation system, serving hundreds of millions of daily active users in core application scenarios.

cs.IR↗

arXiv2025

MLLM-Driven Semantic Identifier Generation for Generative Cross-Modal Retrieval

Generative cross-modal retrieval, which treats retrieval as a generation task, has emerged as a promising direction with the rise of Multimodal Large Language Models (MLLMs). In this setting, the model responds to a text query by generating an identifier corresponding to the target image. However, existing methods typically rely on manually crafted string IDs, clustering-based labels, or atomic identifiers requiring vocabulary expansion, all of which face challenges in semantic alignment or scalability.To address these limitations, we propose a vocabulary-efficient identifier generation framework that prompts MLLMs to generate Structured Semantic Identifiers from image-caption pairs. These identifiers are composed of concept-level tokens such as objects and actions, naturally aligning with the model's generation space without modifying the tokenizer. Additionally, we introduce a Rationale-Guided Supervision Strategy, prompting the model to produce a one-sentence explanation alongside each identifier serves as an auxiliary supervision signal that improves semantic grounding and reduces hallucinations during training.

cs.IR↗

arXiv2025

Unsupervised Monocular Depth Estimation Based on Hierarchical Feature-Guided Diffusion

Unsupervised monocular depth estimation has received widespread attention because of its capability to train without ground truth. In real-world scenarios, the images may be blurry or noisy due to the influence of weather conditions and inherent limitations of the camera. Therefore, it is particularly important to develop a robust depth estimation model. Benefiting from the training strategies of generative networks, generative-based methods often exhibit enhanced robustness. In light of this, we employ a well-converging diffusion model among generative networks for unsupervised monocular depth estimation. Additionally, we propose a hierarchical feature-guided denoising module. This model significantly enriches the model's capacity for learning and interpreting depth distribution by fully leveraging image features to guide the denoising process. Furthermore, we explore the implicit depth within reprojection and design an implicit depth consistency loss. This loss function serves to enhance the performance of the model and ensure the scale consistency of depth within a video sequence. We conduct experiments on the KITTI, Make3D, and our self-collected SIMIT datasets. The results indicate that our approach stands out among generative-based models, while also showcasing remarkable robustness.

cs.CV↗

arXiv2025

Pixel-level modelling of group-scale strong lens CASSOWARY 19

We present the first high-precision model for the group-scale strong lensing system CASSOWARY 19 (CSWA19), utilising images from the Hubble Space Telescope (HST). Sixteen member galaxies identified via the red-sequence method, and the main halo, all modelled as the dual Pseudo Isothermal Elliptical profile (dPIE), are incorporated into a parametric lens model alongside an external shear field. To model the system, we adopt the PyAutoLens software package, employing a progressive search chain strategy for realizing the transition of source model from multiple Sérsic profiles to a brightness-adaptive pixelization, which uses 1000 pixels in the source plane to reconstruct the background source corresponding to 177,144 image pixels in the image plane. Our results indicate that the total mass within the Einstein radius is $M_{θ_\mathrm{E}}$ $\approx 1.41\times10^{13}$M$_{\odot}$ and the average slope of the total mass density $ρ(r)\propto r^{-γ}$ is $\tildeγ=1.33$ within the effective radius. This slope is shallower than those measured in galaxies and groups but is closer to those of galaxy clusters. In addition, our approach successfully resolves the two merging galaxies in the background source and yields a total magnification of $μ=103.18^{+0.23}_{-0.19}$, which is significantly higher than the outcomes from previous studies of CSWA19. In summary, our research demonstrates the effectiveness of the brightness-adaptive pixelization source reconstruction technique for modelling group-scale strong lensing systems. It can serve as a technical reference for future investigations into pixel-level modelling of the group- and cluster-scale strong lensing systems.

astro-ph.GA↗

arXiv2025

MPCache: MPC-Friendly KV Cache Eviction for Efficient Private LLM Inference

Private large language model (LLM) inference based on secure multi-party computation (MPC) achieves formal data privacy protection but suffers from significant latency overhead, especially for long input sequences. While key-value (KV) cache eviction and sparse attention algorithms have been proposed for efficient LLM inference in plaintext, they are not designed for MPC and cannot benefit private LLM inference directly. In this paper, we propose an accurate and MPC-friendly KV cache eviction framework, dubbed MPCache, building on the observation that historical tokens in a long sequence may have different effects on the downstream decoding. Hence, MPCache combines a look-once static eviction algorithm to discard unimportant KV cache and a query-aware dynamic selection algorithm to activate only a small subset of KV cache for attention computation. MPCache further incorporates a series of optimizations for efficient dynamic KV cache selection, including MPC-friendly similarity approximation, hierarchical KV cache clustering, and cross-layer index-sharing strategy. Extensive experiments demonstrate that MPCache consistently outperforms prior-art KV cache eviction baselines across different generation tasks and achieves 1.8 ~ 2.01x and 3.39 ~ 8.37x decoding latency and communication reduction on different sequence lengths, respectively.

cs.CR↗

arXiv2025

MTmixAtt: Integrating Mixture-of-Experts with Multi-Mix Attention for Large-Scale Recommendation

Industrial recommender systems critically depend on high-quality ranking models. However, traditional pipelines still rely on manual feature engineering and scenario-specific architectures, which hinder cross-scenario transfer and large-scale deployment. To address these challenges, we propose \textbf{MTmixAtt}, a unified Mixture-of-Experts (MoE) architecture with Multi-Mix Attention, designed for large-scale recommendation tasks. MTmixAtt integrates two key components. The \textbf{AutoToken} module automatically clusters heterogeneous features into semantically coherent tokens, removing the need for human-defined feature groups. The \textbf{MTmixAttBlock} module enables efficient token interaction via a learnable mixing matrix, shared dense experts, and scenario-aware sparse experts, capturing both global patterns and scenario-specific behaviors within a single framework. Extensive experiments on the industrial TRec dataset from Meituan demonstrate that MTmixAtt consistently outperforms state-of-the-art baselines including Transformer-based models, WuKong, HiFormer, MLP-Mixer, and RankMixer. At comparable parameter scales, MTmixAtt achieves superior CTR and CTCVR metrics; scaling to MTmixAtt-1B yields further monotonic gains. Large-scale online A/B tests validate the real-world impact: in the \textit{Homepage} scenario, MTmixAtt increases Payment PV by \textbf{+3.62\%} and Actual Payment GTV by \textbf{+2.54\%}. Overall, MTmixAtt provides a unified and scalable solution for modeling arbitrary heterogeneous features across scenarios, significantly improving both user experience and commercial outcomes.

cs.IR↗

arXiv2025

Multi-level Reliable Guidance for Unpaired Multi-view Clustering

In this thesis, we address the challenging problem of unpaired multi-view clustering (UMC), which aims to achieve effective joint clustering using unpaired samples observed across multiple views. Traditional incomplete multi-view clustering (IMC) methods typically rely on paired samples to capture complementary information between views. However, such strategies become impractical in the UMC due to the absence of paired samples. Although some researchers have attempted to address this issue by preserving consistent cluster structures across views, effectively mining such consistency remains challenging when the cluster structures {with low confidence}. Therefore, we propose a novel method, Multi-level Reliable Guidance for UMC (MRG-UMC), which integrates multi-level clustering and reliable view guidance to learn consistent and confident cluster structures from three perspectives. Specifically, inner-view multi-level clustering exploits high-confidence sample pairs across different levels to reduce the impact of boundary samples, resulting in more confident cluster structures. Synthesized-view alignment leverages a synthesized-view to mitigate cross-view discrepancies and promote consistency. Cross-view guidance employs a reliable view guidance strategy to enhance the clustering confidence of poorly clustered views. These three modules are jointly optimized across multiple levels to achieve consistent and confident cluster structures. Furthermore, theoretical analyses verify the effectiveness of MRG-UMC in enhancing clustering confidence. Extensive experimental results show that MRG-UMC outperforms state-of-the-art UMC methods, achieving an average NMI improvement of 12.95\% on multi-view datasets. {The source code is available at: https://anonymous.4open.science/r/MRG-UMC-5E20.

cs.CV↗