arXiv subjects

Xinyuan Wei

Publications and source records attributed to Xinyuan Wei.

12 recordsLinked to original sources

arXiv2026

KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model

Recent advancements in Large Language Models (LLMs)-based text embedding models primarily focus on data scaling or synthesis, yet limited exploration of training techniques and data quality, thereby constraining performance. In this work, we propose KaLM-Embedding-V2 from the Lychee-KaLM team, a series of versatile and compact embedding models, systematically incentivizing advanced embedding capability in LLMs by superior training techniques and high-quality data. For model architecture, we implement the models in a 0.5B compact size with simple mean-pooling to produce fixed-length embeddings and remove the causal attention mask to enable fully bidirectional representation learning. For training techniques, we propose a progressive multi-stage training pipeline: pre-training on weakly supervised large-scale datasets, fine-tuning with supervised high-quality datasets, and contrastive distillation with fine-grained soft signals, integrated with focal-style reweighting and online hard-negative mixing to emphasize difficult samples and enrich hard negatives, respectively. For training data, we curate over 20 categories for pre-training and 100 categories for fine-tuning and contrastive distillation to improve both performance and generalization, leveraging task-specific instructions, hard-negative mining, and example-based multi-class labeling to ensure high quality. Combining these techniques, our KaLM-Embedding-V2 series achieves state-of-the-art performance on the Massive Text Embedding Benchmark, outperforming models of comparable size and rivaling models 3--26x larger, setting a new standard for versatile and compact embedding models under 1B parameters. The code, data, and models are available at https://kalm-embedding.github.io/.

cs.CL↗

arXiv2026

DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generation

Diffusion models have enabled high-quality video generation in recent years, but the high cost of iterative sampling hinders their practical deployment. Few-step distillation alleviates this cost, yet exposes a quality--diversity trade-off between its two dominant paradigms: trajectory-level distillation (e.g., sCM) favors diversity, whereas distribution-level distillation (e.g., DMD) favors quality. Targeting extreme two-step video generation, we introduce DUET, which reconciles the two paradigms through a noise-level duet of experts: an sCM expert takes the high-noise step to lay out diverse structure, and a DMD expert takes the low-noise step to refine appearance detail. Since the two experts are trained independently with their native objectives, DUET sidesteps the optimization difficulties of loss-level combinations and delivers quality and diversity jointly rather than trading one for the other. We further identify the relay interface and the high-noise stage as the remaining bottlenecks, and address them with RL-guided expert adaptation, yielding DUET+. With the Wan2.1-T2V-1.3B backbone, DUET lifts the two-step quality of sCM close to the level of DMD while retaining nearly all of its structural diversity---about twice that of DMD---and DUET+ further improves overall quality while preserving this diversity advantage. Together, these results establish noise-level expert specialization as a simple, effective paradigm for reconciling diversity and quality in two-step video generation.

cs.CV↗

arXiv2026

FlashMol: High-Quality Molecule Generation in as Few as Four Steps

Generating chemically valid 3D molecular conformations is critical for computational drug discovery. Classical diffusion-based models like GeoLDM perform well but require hundreds of steps, making large-scale in silico screening impractical. Recent efforts on few-step molecular generation have accelerated this process to 12-50 steps, but they often largely sacrifice sample stability. In this work, we present FlashMol, an ultra-fast molecule generative model producing high-quality molecular conformations in as few as 4 steps. To achieve this, we adapt distribution matching distillation (DMD) - a reverse KL-divergence minimization objective - to the molecular domain for effective distillation. Considering the local minimization behavior of DMD, we respace the molecule generation timesteps, providing the generator with much better initialization and enables effective distillation. Additionally, to mitigate the mode-seeking behavior of DMD and improve diversity, we further regularize it with a Jensen-Shannon divergence term, which incorporates the mean-seeking behavior of the forward KL divergence. Extensive experiments on QM9 and GEOM-DRUG datasets demonstrate that FlashMol matches and even surpasses the original 1000-step teacher, achieving up to 250$\times$ acceleration in sampling speed while maintaining high molecular quality.

cs.LG↗

arXiv2025

SDR-Empowered Environment Sensing Design and Experimental Validation Using OTFS-ISAC Signals

This paper investigates the system design and experimental validation of integrated sensing and communication (ISAC) for environmental sensing, which is expected to be a critical enabler for next-generation wireless networks. We advocate exploiting orthogonal time frequency space (OTFS) modulation for its inherent sparsity and stability in delay-Doppler (DD) domain channels, facilitating a low-overhead environment sensing design. Moreover, a comprehensive environmental sensing framework is developed, encompassing DD domain channel estimation, target localization, and experimental validation. In particular, we first explore the OTFS channel estimation in the presence of fractional delay and Doppler shifts. Given the estimated parameters, we propose a three-ellipse positioning algorithm to localize the target's position, followed by determining the mobile transmitter's velocity. Additionally, to evaluate the performance of our proposed design, we conduct extensive simulations and experiments using a software-defined radio (SDR)-based platform with universal software radio peripheral (USRP). The experimental validations demonstrate that our proposed approach outperforms the benchmarks in terms of localization accuracy and velocity estimation, confirming its effectiveness in practical environmental sensing applications.

eess.SP↗

arXiv2025

KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model

As retrieval-augmented generation prevails in large language models, embedding models are becoming increasingly crucial. Despite the growing number of general embedding models, prior work often overlooks the critical role of training data quality. In this work, we introduce KaLM-Embedding, a general multilingual embedding model that leverages a large quantity of cleaner, more diverse, and domain-specific training data. Our model has been trained with key techniques proven to enhance performance: (1) persona-based synthetic data to create diversified examples distilled from LLMs, (2) ranking consistency filtering to remove less informative samples, and (3) semi-homogeneous task batch sampling to improve training efficacy. Departing from traditional BERT-like architectures, we adopt Qwen2-0.5B as the pre-trained model, facilitating the adaptation of auto-regressive language models for general embedding tasks. Extensive evaluations of the MTEB benchmark across multiple languages show that our model outperforms others of comparable size, setting a new standard for multilingual embedding models with <1B parameters.

cs.CL↗

arXiv2023

Using Large Language Model to Solve and Explain Physics Word Problems Approaching Human Level

Our work demonstrates that large language model (LLM) pre-trained on texts can not only solve pure math word problems, but also physics word problems, whose solution requires calculation and inference based on prior physical knowledge. We collect and annotate the first physics word problem dataset-PhysQA, which contains over 1000 junior high school physics word problems (covering Kinematics, Mass&Density, Mechanics, Heat, Electricity). Then we use OpenAI' s GPT3.5 to generate the answer of these problems and found that GPT3.5 could automatically solve 49.3% of the problems through zero-shot learning and 73.2% through few-shot learning. This result demonstrates that by using similar problems and their answers as prompt, LLM could solve elementary physics word problems approaching human level performance. In addition to solving problems, GPT3.5 can also summarize the knowledge or topics covered by the problems, provide relevant explanations, and generate new physics word problems based on the input. Our work is the first research to focus on the automatic solving, explanation, and generation of physics word problems across various types and scenarios, and we achieve an acceptable and state-of-the-art accuracy. This underscores the potential of LLMs for further applications in secondary education.

cs.CL↗

arXiv2022

Z$_2$ nontrivial topology of rare-earth binary oxide superconductor

Recently, superconductivity has been discovered in rock-salt structured binary lanthanum monoxide LaO through state-of-the-art oxide thin-film epitaxy. In this work, we reveal that the normal state of superconducting LaO is a $Z_2$ nontrivial topological metal, where the Dirac point protected by the crystal symmetry is located around the Fermi energy. By analysing the orbital characteristics, we show that the nature of the topological band structure of LaO originates from the intra-atomic transition from the outer shell La 5$d$ to the inner shell 4$f$ orbitals driven by the strong octahedral crystal-field. Furthermore, the appearance of novel surface states unambiguously demonstrates the topological signature of LaO superconductor. Our theoretical findings not only shed new light into the understanding of the exotic quantum behaviors in LaO superconductor with intimate correlation between 4$f$ and 5$d$ orbitals in La, but also provide an exciting platform to explore the interplay between nontrivial topology and superconductivity.

cond-mat.supr-con↗

arXiv2020

Strong in-plane magnetic field induced reemergent superconductivity in the van der Waals heterointerface of NbSe2 and CrCl3

A magnetic field is generally considered to be incompatible with superconductivity as it tends to spin-polarize electrons and breaks apart the opposite-spin singlet superconducting Cooper pairs. Here, an experimental phenomenon is observed that an intriguing reemergent superconductivity evolves from a conventional superconductivity undergoing a hump-like intermediate phase with a finite electric resistance in the van der Waals heterointerface of layered NbSe2 and CrCl3 flakes. This phenomenon merely occurred when the applied magnetic field is parallel to the sample plane and perpendicular to the electric current direction as compared to the reference sample of a NbSe2 thin flake. The strong anisotropy of the reemergent superconducting phase is pointed to the nature of the Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) state driven by the strong interfacial spin-orbit coupling between NbSe2 and CrCl3 layers. The theoretical picture of FFLO state nodes induced by Josephson vortices collectively pinning is presented for well understanding the experimental observation of the reemergent superconductivity. This finding sheds light on an opportunity to search for the exotic FFLO state in the van der Waals heterostructures with strong interfacial spin-orbit coupling.

cond-mat.supr-con↗

arXiv2020

Evidence the ferromagnetic order on CoSb layer of LaCoSb$_2$

The emergence of unconventional superconductivity is generally considered to be related to spin fluctuations. Unveiling the intriguing behaviors of spin fluctuations in parent compounds with layered transition-metal ions may shed light on the search for exotic unconventional superconductors. Here, based on the framework of the first-principles calculations, we theoretically propose that LaCoSb$_2$ is a weak antiferromagnetic layered metal with an in-plane ferromagnetic moment of 0.88 $μ_B$ at the Co sites, as a candidate parent compound of the cobalt-based superconductors. Importantly, this theoretical finding is experimentally supported by our magnetization measurements on polycrystalline samples of LaCo$_{0.78}$Sb$_2$. Following the symmetry analysis, we suggest a possible $p$-wave superconductivity hosted in doped LaCoSb$_2$ emerging at the verge of ferromagnetic spin fluctuations, which implies potential applications in topological quantum computing in future.

cond-mat.supr-con↗

arXiv2019

Universal critical behavior in the ferromagnetic superconductor Eu(Fe$_{0.75}$Ru$_{0.25}$)$_{2}$As$_{2}$

The study of universal critical behavior is a crucial issue in a continuous phase transition, which groups various critical phenomena into universality classes for revealing microscopic electronic behaviors. The understanding of the nature of magnetism in Eu-based ferromagnetic superconductors is largely impeded by the infeasibility of performing inelastic neutron scattering measurements to deduce the microscopic magnetic behaviors and the effects on the superconductivity, due to the significant neutron absorption effect of natural $^{152}$Eu and unavailability of large single crystals. However, by systematically combining the neutron diffraction experiment, the first-principles calculations, and the quantum Monte Carlo simulations, we have obtained a perfectly consistent universal critical exponent value of $β=0.385(13)$ experimentally and theoretically for Eu(Fe$_{0.75}$Ru$_{0.25}$)$_{2}$As$_{2}$, from which the magnetism in the Eu-based ferromagnetic superconductors is identified as the universal class of a three-dimensional anisotropic quantum Heisenberg model with long-range magnetic exchange coupling. This study not only clarifies the nature of microscopic magnetic behaviors in the Eu-based ferromagnetic superconductors, but also opens a new avenue of systemic methodology for studying the universal critical behaviors associated with magnetic phase transitions in the area of magnetism and the spin fluctuations effects on the unconventional superconductivity.

cond-mat.str-el↗

arXiv2018

Inevitable high density of oxygen vacancies on the surface of LaAlO3/SrTiO3 heterostructures

Using density-functional-theory (DFT) calculations with the HSE06 hybrid functional, we accurately evaluate the critical thickness of LaAlO3 film for the intrinsic doping in LaAlO3/SrTiO3 (LAO/STO) heterstructures. The calculated critical thickness of 6 unit-cell (uc) layers suggests to rule out the intrinsic doping mechanism. We also calculate the density of oxygen vacancies on the LAO surface at varying LAO thicknesses, preparation oxygen pressures and temperatures by using the condition of chemical equilibrium and DFT calculations. We find that once LAO thickness >=3 uc high-density (~ 10^14 cm^-2 ) oxygen vacancies will inevitably exist on the LAO surface of the LAO/STO heterstructures even though the samples are grown under high oxygen pressure. The oxygen vacancies are stabilized by releasing the electrostatic energy in the LAO film.

cond-mat.mtrl-sci↗

arXiv2017

Chern insulators without band inversion in MoS2 monolayers with 3d adatoms

Electronic and topological properties of MoS2 monolayers endowed with 3d transition metal (TM) adatoms (V-Fe) are explored by using ab initio methods and k.p models. Without the consideration of the Hubbard U interaction, the V, Cr, and Fe adatoms tend to locate on the top of the Mo atoms, while the most stable site for the Mn atom is at the hollow position of the Mo-S hexagon. After the Hubbard U is applied, the most stable sites of all the systems become the top of the Mo atoms. Chern insulators without band inversion are achieved in these systems. The V and Fe adsorption systems are the best candidates to produce the topological states. The k.p model calculations indicate that these topological states are determined by the TM magnetism, the C3v crystal field from the MoS2 substrate, and the TM atomic spin-orbit coupling (SOC). The special two-meron pseudospin texture is found to contribute to the topology. The apparent difference between the Berry curvatures for the V and Fe adsorption systems are also explored. Our results widen the understanding to the Chern insulators and are helpful for the applications of the MoS2 monolayers in the future electronics and spintronics.

cond-mat.mtrl-sci↗