arXiv subjects

Kai Wang

Publications and source records attributed to Kai Wang.

At least 271 records · Page 15Linked to original sources

arXiv2025

Covariances for Free: Exploiting Mean Distributions for Training-free Federated Learning

Using pre-trained models has been found to reduce the effect of data heterogeneity and speed up federated learning algorithms. Recent works have explored training-free methods using first- and second-order statistics to aggregate local client data distributions at the server and achieve high performance without any training. In this work, we propose a training-free method based on an unbiased estimator of class covariance matrices which only uses first-order statistics in the form of class means communicated by clients to the server. We show how these estimated class covariances can be used to initialize the global classifier, thus exploiting the covariances without actually sharing them. We also show that using only within-class covariances results in a better classifier initialization. Our approach improves performance in the range of 4-26% with exactly the same communication cost when compared to methods sharing only class means and achieves performance competitive or superior to methods sharing second-order statistics with dramatically less communication overhead. The proposed method is much more communication-efficient than federated prompt-tuning methods and still outperforms them. Finally, using our method to initialize classifiers and then performing federated fine-tuning or linear probing again yields better performance. Code is available at https://github.com/dipamgoswami/FedCOF.

cs.LG↗

arXiv2025

Self-Improvement in Multimodal Large Language Models: A Survey

Recent advancements in self-improvement for Large Language Models (LLMs) have efficiently enhanced model capabilities without significantly increasing costs, particularly in terms of human effort. While this area is still relatively young, its extension to the multimodal domain holds immense potential for leveraging diverse data sources and developing more general self-improving models. This survey is the first to provide a comprehensive overview of self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview of the current literature and discuss methods from three perspectives: 1) data collection, 2) data organization, and 3) model optimization, to facilitate the further development of self-improvement in MLLMs. We also include commonly used evaluations and downstream applications. Finally, we conclude by outlining open challenges and future research directions.

cs.CL↗

arXiv2025

Joint Deblurring and 3D Reconstruction for Macrophotography

Macro lens has the advantages of high resolution and large magnification, and 3D modeling of small and detailed objects can provide richer information. However, defocus blur in macrophotography is a long-standing problem that heavily hinders the clear imaging of the captured objects and high-quality 3D reconstruction of them. Traditional image deblurring methods require a large number of images and annotations, and there is currently no multi-view 3D reconstruction method for macrophotography. In this work, we propose a joint deblurring and 3D reconstruction method for macrophotography. Starting from multi-view blurry images captured, we jointly optimize the clear 3D model of the object and the defocus blur kernel of each pixel. The entire framework adopts a differentiable rendering method to self-supervise the optimization of the 3D model and the defocus blur kernel. Extensive experiments show that from a small number of multi-view images, our proposed method can not only achieve high-quality image deblurring but also recover high-fidelity 3D appearance.

cs.CV↗

arXiv2025

On-the-Fly Data Augmentation via Gradient-Guided and Sample-Aware Influence Estimation

Data augmentation has been widely employed to improve the generalization of deep neural networks. Most existing methods apply fixed or random transformations. However, we find that sample difficulty evolves along with the model's generalization capabilities in dynamic training environments. As a result, applying uniform or stochastic augmentations, without accounting for such dynamics, can lead to a mismatch between augmented data and the model's evolving training needs, ultimately degrading training effectiveness. To address this, we introduce SADA, a Sample-Aware Dynamic Augmentation that performs on-the-fly adjustment of augmentation strengths based on each sample's evolving influence on model optimization. Specifically, we estimate each sample's influence by projecting its gradient onto the accumulated model update direction and computing the temporal variance within a local training window. Samples with low variance, indicating stable and consistent influence, are augmented more strongly to emphasize diversity, while unstable samples receive milder transformations to preserve semantic fidelity and stabilize learning. Our method is lightweight, which does not require auxiliary models or policy tuning. It can be seamlessly integrated into existing training pipelines as a plug-and-play module. Experiments across various benchmark datasets and model architectures show consistent improvements of SADA, including +7.3\% on fine-grained tasks and +4.3\% on long-tailed datasets, highlighting the method's effectiveness and practicality.

cs.LG↗

arXiv2025

IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance

Ensuring precise multimodal alignment between diffusion-generated images and input prompts has been a long-standing challenge. Earlier works finetune diffusion weight using high-quality preference data, which tends to be limited and difficult to scale up. Recent editing-based methods further refine local regions of generated images but may compromise overall image quality. In this work, we propose Implicit Multimodal Guidance (IMG), a novel re-generation-based multimodal alignment framework that requires no extra data or editing operations. Specifically, given a generated image and its prompt, IMG a) utilizes a multimodal large language model (MLLM) to identify misalignments; b) introduces an Implicit Aligner that manipulates diffusion conditioning features to reduce misalignments and enable re-generation; and c) formulates the re-alignment goal into a trainable objective, namely Iteratively Updated Preference Objective. Extensive qualitative and quantitative evaluations on SDXL, SDXL-DPO, and FLUX show that IMG outperforms existing alignment methods. Furthermore, IMG acts as a flexible plug-and-play adapter, seamlessly enhancing prior finetuning-based alignment methods. Our code will be available at https://github.com/SHI-Labs/IMG-Multimodal-Diffusion-Alignment.

cs.CV↗

arXiv2025

StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training

Training Transformer models on long sequences in a distributed setting poses significant challenges in terms of efficiency and scalability. Current methods are either constrained by the number of attention heads or excessive communication overheads. To address this problem, we propose StarTrail, a multi-dimensional concentric distributed training system for long sequences, fostering an efficient communication paradigm and providing additional tuning flexibility for communication arrangements. Specifically, StarTrail introduces an extra parallel dimension and divides the peer-to-peer communication into sub-rings to substantially reduce communication volume and avoid bandwidth bottlenecks. Through comprehensive experiments across diverse hardware environments and on both Natural Language Processing (NLP) and Computer Vision (CV) tasks, we demonstrate that our approach significantly surpasses state-of-the-art methods that support Long sequence lengths, achieving performance improvements of up to 77.12% on GPT-style models and up to 114.33% on DiT (Diffusion Transformer) models without affecting the computations results.

cs.DC↗

arXiv2025

A defect-chemistry-informed phase-field model of grain growth in oxide ceramics: application to Fe-doped SrTiO3

Dopants can significantly affect the properties of oxide ceramics through their impact on the property-determined microstructure characteristics such as grain boundary (GB) segregation, space charge layer formation in the GB vicinity, and the grain growth deviating from normal patterns. To support the rational design of oxide ceramics, we propose a defect-chemistry-informed phase-field grain growth model to simulate the microstructure evolution of oxide ceramics. It fully respects the defect-chemistry theory by accounting for the distinct segregation energies and available site densities of charged point defects (oxygen vacancies and acceptor dopants) in both the grain interior and boundaries, and it considers the competing kinetics of defect diffusion and GB movement. The proposed phase-field model is benchmarked against well-known bicrystal models, including the Mott-Schottky and Gouy-Chapman models. Various simulation results are presented to reveal the effect of different defect-chemistry parameters on the space charge layer formation and key microstructural aspects. In particular, simulation results confirm that the solute drag effect alone can lead to skewed grain size distribution that do not follow the log-normal distribution, without any contribution from grain misorientation and other anisotropy. Interestingly, simulations also demonstrate that grain boundary potentials can vary substantially: GBs of larger grains tend to have lower potentials than those of smaller grains. Such heterogeneous GB potential distribution may inspire a new material optimization strategy through microstructure design. This study provides a comprehensive framework for defect-chemistry-consistent investigations of microstructure evolution in polycrystalline oxide ceramics, offering fundamental insights into microscopic processes during critical manufacturing stages.

cond-mat.mtrl-sci↗

arXiv2025

RAPID^3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformer

Diffusion Transformers (DiTs) excel at visual generation yet remain hampered by slow sampling. Existing training-free accelerators - step reduction, feature caching, and sparse attention - enhance inference speed but typically rely on a uniform heuristic or a manually designed adaptive strategy for all images, leaving quality on the table. Alternatively, dynamic neural networks offer per-image adaptive acceleration, but their high fine-tuning costs limit broader applicability. To address these limitations, we introduce RAPID3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformers, a framework that delivers image-wise acceleration with zero updates to the base generator. Specifically, three lightweight policy heads - Step-Skip, Cache-Reuse, and Sparse-Attention - observe the current denoising state and independently decide their corresponding speed-up at each timestep. All policy parameters are trained online via Group Relative Policy Optimization (GRPO) while the generator remains frozen. Meanwhile, an adversarially learned discriminator augments the reward signal, discouraging reward hacking by boosting returns only when generated samples stay close to the original model's distribution. Across state-of-the-art DiT backbones, including Stable Diffusion 3 and FLUX, RAPID3 achieves nearly 3x faster sampling with competitive generation quality.

cs.CV↗

arXiv2025

Fuzzy Reasoning Chain (FRC): An Innovative Reasoning Framework from Fuzziness to Clarity

With the rapid advancement of large language models (LLMs), natural language processing (NLP) has achieved remarkable progress. Nonetheless, significant challenges remain in handling texts with ambiguity, polysemy, or uncertainty. We introduce the Fuzzy Reasoning Chain (FRC) framework, which integrates LLM semantic priors with continuous fuzzy membership degrees, creating an explicit interaction between probability-based reasoning and fuzzy membership reasoning. This transition allows ambiguous inputs to be gradually transformed into clear and interpretable decisions while capturing conflicting or uncertain signals that traditional probability-based methods cannot. We validate FRC on sentiment analysis tasks, where both theoretical analysis and empirical results show that it ensures stable reasoning and facilitates knowledge transfer across different model scales. These findings indicate that FRC provides a general mechanism for managing subtle and ambiguous expressions with improved interpretability and robustness.

cs.CL↗

arXiv2025

Manipulating terahertz phonon-polariton in the ultrastrong coupling regime with bound states in the continuum

The strong coupling between photons and phonons in polar materials gives rise to phonon-polaritons that encapsulate a wealth of physical information, offering crucial tools for the ultrafast terahertz sources and the topological engineering of terahertz light. However, it is still quite challenging to form and manipulate the terahertz phonon-polaritons under the ultrastrong coupling regime till now. In this work, we demonstrate the ultrastrong coupling between the phonon (at 0.95 THz) in a MaPbI 3 film and the metallic bound states in the continuum (BICs) in Au metasurfaces. The Rabi splitting can be continuously tuned from 28% to 48.4% of the phonon frequency by adjusting the parameters (size, shape and period) of Au metasurfaces, reaching the ultrastrong coupling regime. By introducing wavelet transform, the mode evolution information of the terahertz phonon-polariton is successfully extracted. It indicates that the phonon radiation intensity of the MaPbI 3 film is enhanced as the coupling strength is increased. This work not only establishes a new platform for terahertz devices but also opens new avenues for exploring the intricate dynamics of terahertz phonon-polaritons.

physics.optics↗

arXiv2025

DD-Ranking: Rethinking the Evaluation of Dataset Distillation

In recent years, dataset distillation has provided a reliable solution for data compression, where models trained on the resulting smaller synthetic datasets achieve performance comparable to those trained on the original datasets. To further improve the performance of synthetic datasets, various training pipelines and optimization objectives have been proposed, greatly advancing the field of dataset distillation. Recent decoupled dataset distillation methods introduce soft labels and stronger data augmentation during the post-evaluation phase and scale dataset distillation up to larger datasets (e.g., ImageNet-1K). However, this raises a question: Is accuracy still a reliable metric to fairly evaluate dataset distillation methods? Our empirical findings suggest that the performance improvements of these methods often stem from additional techniques rather than the inherent quality of the images themselves, with even randomly sampled images achieving superior results. Such misaligned evaluation settings severely hinder the development of DD. Therefore, we propose DD-Ranking, a unified evaluation framework, along with new general evaluation metrics to uncover the true performance improvements achieved by different methods. By refocusing on the actual information enhancement of distilled datasets, DD-Ranking provides a more comprehensive and fair evaluation standard for future research advancements.

cs.CV↗

arXiv2025

Neutron star phase transition as the origin for the fast radio bursts and soft gamma-ray repeaters of SGR J1935+2154

Magnetars are believed as neutron stars (NSs) with strong magnetic fields. X-ray flares and fast radio bursts (FRBs) have been observed from the magnetar (soft gamma-ray repeater, SGR J1935+2154). We propose that the phase transition of the NS can power the FRBs and SGRs.Based on the equation of state provided by the MIT bag model and the mean field approximation, we solve the Tolman-Oppenheimer-Volkoff equations to get the NS structure. With spin-down of the NS, the hadronic shell gradually transfers to the quark shell.The gravitational potential energy released by one time of the phase transition can be achieved. The released energy, time interval between two successive phase transitions, and glitch are all consistent with the observations of the FRBs and the X-ray flares from SGR J1935+2154. We conclude that the phase transition of an NS is a plausible mechanism to power the SGRs as well as the repeating FRBs.

astro-ph.HE↗

arXiv2025

TFLAG:Towards Practical APT Detection via Deviation-Aware Learning on Temporal Provenance Graph

Advanced Persistent Threat (APT) have grown increasingly complex and concealed, posing formidable challenges to existing Intrusion Detection Systems in identifying and mitigating these attacks. Recent studies have incorporated graph learning techniques to extract detailed information from provenance graphs, enabling the detection of attacks with greater granularity. Nevertheless, existing studies have largely overlooked the continuous yet subtle temporal variations in the structure of provenance graphs, which may correspond to surreptitious perturbation anomalies in ongoing APT attacks. Therefore, we introduce TFLAG, an advanced anomaly detection framework that for the first time integrates the structural dynamic extraction capabilities of temporal graph model with the anomaly delineation abilities of deviation networks to pinpoint covert attack activities in provenance graphs. This self-supervised integration framework leverages the graph model to extract neighbor interaction data under continuous temporal changes from historical benign behaviors within provenance graphs, while simultaneously utilizing deviation networks to accurately distinguish authentic attack activities from false positive deviations due to unexpected subtle perturbations. The experimental results indicate that, through a comprehensive design that utilizes both attribute and temporal information, it can accurately identify the time windows associated with APT attack behaviors without prior knowledge (e.g., labeled data samples), demonstrating superior accuracy compared to current state-of-the-art methods in differentiating between attack events and system false positive events.

cs.CR↗

arXiv2025

InterFormer: Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction

Click-through rate (CTR) prediction, which predicts the probability of a user clicking an ad, is a fundamental task in recommender systems. The emergence of heterogeneous information, such as user profile and behavior sequences, depicts user interests from different aspects. A mutually beneficial integration of heterogeneous information is the cornerstone towards the success of CTR prediction. However, most of the existing methods suffer from two fundamental limitations, including (1) insufficient inter-mode interaction due to the unidirectional information flow between modes, and (2) aggressive information aggregation caused by early summarization, resulting in excessive information loss. To address the above limitations, we propose a novel module named InterFormer to learn heterogeneous information interaction in an interleaving style. To achieve better interaction learning, InterFormer enables bidirectional information flow for mutually beneficial learning across different modes. To avoid aggressive information aggregation, we retain complete information in each data mode and use a separate bridging arch for effective information selection and summarization. Our proposed InterFormer achieves state-of-the-art performance on three public datasets and a large-scale industrial dataset.

cs.IR↗

arXiv2025

MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillance

General-domain large multimodal models (LMMs) have achieved significant advances in various image-text tasks. However, their performance in the Intelligent Traffic Surveillance (ITS) domain remains limited due to the absence of dedicated multimodal datasets. To address this gap, we introduce MITS (Multimodal Intelligent Traffic Surveillance), the first large-scale multimodal benchmark dataset specifically designed for ITS. MITS includes 170,400 independently collected real-world ITS images sourced from traffic surveillance cameras, annotated with eight main categories and 24 subcategories of ITS-specific objects and events under diverse environmental conditions. Additionally, through a systematic data generation pipeline, we generate high-quality image captions and 5 million instruction-following visual question-answer pairs, addressing five critical ITS tasks: object and event recognition, object counting, object localization, background analysis, and event reasoning. To demonstrate MITS's effectiveness, we fine-tune mainstream LMMs on this dataset, enabling the development of ITS-specific applications. Experimental results show that MITS significantly improves LMM performance in ITS applications, increasing LLaVA-1.5's performance from 0.494 to 0.905 (+83.2%), LLaVA-1.6's from 0.678 to 0.921 (+35.8%), Qwen2-VL's from 0.584 to 0.926 (+58.6%), and Qwen2.5-VL's from 0.732 to 0.930 (+27.0%). We release the dataset, code, and models as open-source, providing high-value resources to advance both ITS and LMM research.

cs.CV↗

arXiv2025

Finding a Multiple Follower Stackelberg Equilibrium: A Fully First-Order Method

In this work, we propose the first fully first-order method to compute an epsilon stationary Stackelberg equilibrium with convergence guarantees. To achieve this, we first reframe the leader follower interaction as single level constrained optimization. Second, we define the Lagrangian and show that it can approximate the leaders gradient in response to the equilibrium reached by followers with only first-order gradient evaluations. These findings suggest a fully first order algorithm that alternates between (i) approximating followers best responses through gradient descent and (ii) updating the leaders strategy via approximating the gradient using Lagrangian.

math.OC↗

arXiv2025

Quantum Purification for Amplitude Damping Noise

Noise poses a fundamental challenge to quantum information processing, with amplitude-damping (AD) noise being particularly detrimental. Preserving high-fidelity quantum systems therefore relies critically on effective error correction and purification methods. In this work, we introduce a novel approach for mitigating AD noise that can be applied to both state and channel purification. Our method achieves a substantial enhancement in the fidelity of affected states or channels while maintaining a low resource overhead, requiring only one or two ancilla qubits in combination with two Clifford gates, and exhibits a relatively high success probability. This approach provides a practical and scalable framework for addressing AD noise in realistic quantum systems.

quant-ph↗

arXiv2025

Revealing Potential Biases in LLM-Based Recommender Systems in the Cold Start Setting

Large Language Models (LLMs) are increasingly used for recommendation tasks due to their general-purpose capabilities. While LLMs perform well in rich-context settings, their behavior in cold-start scenarios, where only limited signals such as age, gender, or language are available, raises fairness concerns because they may rely on societal biases encoded during pretraining. We introduce a benchmark specifically designed to evaluate fairness in zero-context recommendation. Our modular pipeline supports configurable recommendation domains and sensitive attributes, enabling systematic and flexible audits of any open-source LLM. Through evaluations of state-of-the-art models (Gemma 3 and Llama 3.2), we uncover consistent biases across recommendation domains (music, movies, and colleges) including gendered and cultural stereotypes. We also reveal a non-linear relationship between model size and fairness, highlighting the need for nuanced analysis.

cs.IR↗