arXiv subjects

Jiayi Liu

Publications and source records attributed to Jiayi Liu.

At least 55 records · Page 3Linked to original sources

arXiv2025

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning

Fine-tuning pre-trained large language models (LLMs) presents a dual challenge of balancing parameter efficiency and model capacity. Existing methods like low-rank adaptations (LoRA) are efficient but lack flexibility, while Mixture-of-Experts (MoE) enhance model capacity at the cost of more & under-utilized parameters. To address these limitations, we propose Structural Mixture of Residual Experts (S'MoRE), a novel framework that seamlessly integrates the efficiency of LoRA with the flexibility of MoE. Conceptually, S'MoRE employs hierarchical low-rank decomposition of expert weights, yielding residuals of varying orders interconnected in a multi-layer structure. By routing input tokens through sub-trees of residuals, S'MoRE emulates the capacity of numerous experts by instantiating and assembling just a few low-rank matrices. We craft the inter-layer propagation of S'MoRE's residuals as a special type of Graph Neural Network (GNN), and prove that under similar parameter budget, S'MoRE improves structural flexibility of traditional MoE (or Mixture-of-LoRA) by exponential order. Comprehensive theoretical analysis and empirical results demonstrate that S'MoRE achieves superior fine-tuning performance, offering a transformative approach for efficient LLM adaptation. Our implementation is available at: https://github.com/ZimpleX/SMoRE-LLM.

cs.CL↗

arXiv2025

Emerging axion detection in artificial magnetoelectric materials

Axions are considered a key component of dark matter, characterized by very weak couplings to fermions and Chern-Simons couplings to gauge fields. We propose a novel detection mechanism based on symmetry-breaking magnetoelectric materials with a linear axionic coupling between magnetization and ferroelectric polarization. The focus is on a strain gradient Sr2IrO4 film, where the breaking of space-inversion symmetry results in an emergent polar phase and an out-of-plane magnetic moment, exhibiting a flexomagnetoelectric effect. In this material, the linear P||M enables a direct coupling between the external axion field and the intrinsic axion-like field, which amplifies the weak electromagnetic signals induced by axions, paving the way for pioneering axion detection. In contrast to conventional detection techniques, this mechanism is expected to enhance the sensitivity of the axion-electron and axion-photon coupling, providing a novel platform for axion detection and advancing the study of dark matter through the magnetoelectric effect.

hep-ph↗

arXiv2025

Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding

Understanding and reasoning over tables is a critical capability for many real-world applications. Large language models (LLMs) have shown promise on this task, but current approaches remain limited. Fine-tuning based methods strengthen language reasoning; yet they are prone to arithmetic errors and hallucination. In contrast, tool-based methods enable precise table manipulation but rely on rigid schemas and lack semantic understanding. These complementary drawbacks highlight the need for approaches that integrate robust reasoning with reliable table processing. In this work, we propose Mixture-of-Minds, a multi-agent framework that decomposes table reasoning into three specialized roles: planning, coding, and answering. This design enables each agent to focus on a specific aspect of the task while leveraging code execution for precise table manipulation. Building on this workflow, we introduce a self-improvement training framework that employs Monte Carlo Tree Search (MCTS) rollouts to generate pseudo-gold trajectories and optimize agents with reinforcement learning (RL). Extensive experiments show that Mixture-of-Minds delivers substantial gains, reaching 62.13% on TableBench and surpassing OpenAI-o4-mini-high. These results demonstrate the promise of combining structured multi-agent workflows with RL to advance table understanding.

cs.CL↗

arXiv2025

From Watch to Imagine: Steering Long-horizon Manipulation via Human Demonstration and Future Envisionment

Generalizing to long-horizon manipulation tasks in a zero-shot setting remains a central challenge in robotics. Current multimodal foundation based approaches, despite their capabilities, typically fail to decompose high-level commands into executable action sequences from static visual input alone. To address this challenge, we introduce Super-Mimic, a hierarchical framework that enables zero-shot robotic imitation by directly inferring procedural intent from unscripted human demonstration videos. Our framework is composed of two sequential modules. First, a Human Intent Translator (HIT) parses the input video using multimodal reasoning to produce a sequence of language-grounded subtasks. These subtasks then condition a Future Dynamics Predictor (FDP), which employs a generative model that synthesizes a physically plausible video rollout for each step. The resulting visual trajectories are dynamics-aware, explicitly modeling crucial object interactions and contact points to guide the low-level controller. We validate this approach through extensive experiments on a suite of long-horizon manipulation tasks, where Super-Mimic significantly outperforms state-of-the-art zero-shot methods by over 20%. These results establish that coupling video-driven intent parsing with prospective dynamics modeling is a highly effective strategy for developing general-purpose robotic systems.

cs.RO↗

arXiv2025

Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalization

The generalization capabilities of vision-language-action (VLA) models to unseen tasks are crucial to achieving general-purpose robotic manipulation in open-world settings. However, the cross-task generalization capabilities of existing VLA models remain significantly underexplored. To address this gap, we introduce AGNOSTOS, a novel simulation benchmark designed to rigorously evaluate cross-task zero-shot generalization in manipulation. AGNOSTOS comprises 23 unseen manipulation tasks for testing, distinct from common training task distributions, and incorporates two levels of generalization difficulty to assess robustness. Our systematic evaluation reveals that current VLA models, despite being trained on diverse datasets, struggle to generalize effectively to these unseen tasks. To overcome this limitation, we propose Cross-Task In-Context Manipulation (X-ICM), a method that conditions large language models (LLMs) on in-context demonstrations from seen tasks to predict action sequences for unseen tasks. Additionally, we introduce a dynamics-guided sample selection strategy that identifies relevant demonstrations by capturing cross-task dynamics. On AGNOSTOS, X-ICM significantly improves cross-task zero-shot generalization performance over leading VLAs. We believe AGNOSTOS and X-ICM will serve as valuable tools for advancing general-purpose robotic manipulation.

cs.RO↗

arXiv2025

Spectropolarimetric Inversion in Four Dimensions with Deep Learning (SPIn4D): II. A Physics-Informed Machine Learning Method for 3D Solar Photosphere Reconstruction

Inferring the three-dimensional (3D) solar atmospheric structures from observations is a critical task for advancing our understanding of the magnetic fields and electric currents that drive solar activity. In this work, we introduce a novel, Physics-Informed Machine Learning method to reconstruct the 3D structure of the lower solar atmosphere based on the output of optical depth sampled spectropolarimetric inversions, wherein both the fully disambiguated vector magnetic fields and the geometric height associated with each optical depth are returned simultaneously. Traditional techniques typically resolve the 180-degree azimuthal ambiguity assuming a single layer, either ignoring the intrinsic non-planar physical geometry of constant optical-depth surfaces (e.g., the Wilson depression in sunspots), or correcting the effect as a post-processing step. In contrast, our approach simultaneously maps the optical depths to physical heights, and enforces the divergence-free condition for magnetic fields fully in 3D. Tests on magnetohydrodynamic simulations of quiet Sun, plage, and a sunspot demonstrate that our method reliably recovers the horizontal magnetic field orientation in locations with appreciable magnetic field strength. By coupling the resolutions of the azimuthal ambiguity and the geometric heights problems, we achieve a self-consistent reconstruction of the 3D vector magnetic fields and, by extension, the electric current density and Lorentz force. This physics-constrained, label-free training paradigm is a generalizable, physics-anchored framework that extends across solar magnetic environments while improving the understanding of various solar puzzles.

astro-ph.SR↗

arXiv2025

Spectropolarimetric Inversion in Four Dimensions with Deep Learning (SPIn4D): I. Overview, Magnetohydrodynamic Modeling, and Stokes Profile Synthesis

The National Science Foundation's Daniel K. Inouye Solar Telescope (DKIST) will provide high-resolution, multi-line spectropolarimetric observations that are poised to revolutionize our understanding of the Sun. Given the massive data volume, novel inference techniques are required to unlock its full potential. Here, we provide an overview of our "SPIn4D" project, which aims to develop deep convolutional neural networks (CNNs) for estimating the physical properties of the solar photosphere from DKIST spectropolarimetric observations. We describe the magnetohydrodynamic (MHD) modeling and the Stokes profile synthesis pipeline that produce the simulated output and input data, respectively. These data will be used to train a set of CNNs that can rapidly infer the four-dimensional MHD state vectors by exploiting the spatiotemporally coherent patterns in the Stokes profile time series. Specifically, our radiative MHD model simulates the small-scale dynamo actions that are prevalent in quiet-Sun and plage regions. Six cases with different mean magnetic fields have been conducted; each case covers six solar-hours, totaling 109 TB in data volume. The simulation domain covers at least $25\times25\times8$ Mm with $16\times16\times12$ km spatial resolution, extending from the upper convection zone up to the temperature minimum region. The outputs are stored at a 40 s cadence. We forward model the Stokes profile of two sets of Fe I lines at 630 and 1565 nm, which will be simultaneously observed by DKIST and can better constrain the parameter variations along the line of sight. The MHD model output and the synthetic Stokes profiles are publicly available, with 13.7 TB in the initial release.

astro-ph.SR↗

arXiv2025

Graphical Perception of Icon Arrays versus Bar Charts for Value Comparisons in Health Risk Communication

Visualizations support critical decision making in domains like health risk communication. This is particularly important for those at higher health risks and their care providers, allowing for better risk interpretation which may lead to more informed decisions. However, the kinds of visualizations used to represent data may impart biases that influence data interpretation and decision making. Both continuous representations using bar charts and discrete representations using icon arrays are pervasive in health risk communication, but express the same quantities using fundamentally different visual paradigms. We conducted a series of studies to investigate how bar charts, icon arrays, and their layout (juxtaposed, explicit encoding, explicit encoding plus juxtaposition) affect the perception of value comparison and subsequent decision-making in health risk communication. Our results suggest that icon arrays and explicit encoding combined with juxtaposition can optimize for both accurate difference estimation and perceptual biases in decision making. We also found misalignment between estimation accuracy and decision making, as well as between low and high literacy groups, emphasizing the importance of tailoring visualization approaches to specific audiences and evaluating visualizations beyond perceptual accuracy alone. This research contributes empirically-grounded design recommendations to improve comparison in health risk communication and support more informed decision-making across domains.

cs.HC↗

arXiv2025

Chemotaxis of branched cells in complex environments

Cell migration in vivo is often guided by chemical signals. Such chemotaxis, such as performed by immune cells migrating to a wound site, is complicated by the complex geometry inside living tissues. In this study, we extend our theoretical model of branched-cell migration on a network by introducing chemokine sources to explore the cellular response. The model predicts a speed-accuracy tradeoff, whereby slow cells are significantly more accurate and able to follow efficiently a weak chemoattractant signal. We then compare the model's predictions with experimental observations of neutrophils migrating to the site of laser-inflicted wound in a zebrafish larva fin, and migrating in-vitro inside a regular lattice of pillars. We find that the model captures the details of the sub-cellular response to the chemokine gradient, as well as the large-scale migration response. This comparison suggests that the neutrophils behave as fast cells, compromising their chemotaxis accuracy, which explains the functionality of these immune cells.

physics.bio-ph↗

arXiv2025

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment

Recent advances in large language models (LLMs) have demonstrated significant progress in performing complex tasks. While Reinforcement Learning from Human Feedback (RLHF) has been effective in aligning LLMs with human preferences, it is susceptible to spurious correlations in reward modeling. Consequently, it often introduces biases-such as length bias, sycophancy, conceptual bias, and discrimination-that hinder the model's ability to capture true causal relationships. To address this, we propose a novel causal reward modeling approach that integrates causality to mitigate these spurious correlations. Our method enforces counterfactual invariance, ensuring reward predictions remain consistent when irrelevant variables are altered. Through experiments on both synthetic and real-world datasets, we show that our approach mitigates various types of spurious correlations effectively, resulting in more reliable and fair alignment of LLMs with human preferences. As a drop-in enhancement to the existing RLHF workflow, our causal reward modeling provides a practical way to improve the trustworthiness and fairness of LLM finetuning.

cs.LG↗

arXiv2025

DTBIA: An Immersive Visual Analytics System for Brain-Inspired Research

The Digital Twin Brain (DTB) is an advanced artificial intelligence framework that integrates spiking neurons to simulate complex cognitive functions and collaborative behaviors. For domain experts, visualizing the DTB's simulation outcomes is essential to understanding complex cognitive activities. However, this task poses significant challenges due to DTB data's inherent characteristics, including its high-dimensionality, temporal dynamics, and spatial complexity. To address these challenges, we developed DTBIA, an Immersive Visual Analytics System for Brain-Inspired Research. In collaboration with domain experts, we identified key requirements for effectively visualizing spatiotemporal and topological patterns at multiple levels of detail. DTBIA incorporates a hierarchical workflow - ranging from brain regions to voxels and slice sections - along with immersive navigation and a 3D edge bundling algorithm to enhance clarity and provide deeper insights into both functional (BOLD) and structural (DTI) brain data. The utility and effectiveness of DTBIA are validated through two case studies involving with brain research experts. The results underscore the system's role in enhancing the comprehension of complex neural behaviors and interactions.

cs.HC↗

arXiv2025

LLM-Enhanced Feature Engineering for Multi-Factor Electricity Price Predictions

Accurately forecasting electricity price volatility is crucial for effective risk management and decision-making. Traditional forecasting models often fall short in capturing the complex, non-linear dynamics of electricity markets, particularly when external factors like weather conditions and market volatility are involved. These limitations hinder their ability to provide reliable predictions in markets with high volatility, such as the New South Wales (NSW) electricity market. To address these challenges, we introduce FAEP, a Feature-Augmented Electricity Price Prediction framework. FAEP leverages Large Language Models (LLMs) combined with advanced feature engineering to enhance prediction accuracy. By incorporating external features such as weather data and price volatility jumps, and utilizing Retrieval-Augmented Generation (RAG) for effective feature extraction, FAEP overcomes the shortcomings of traditional approaches. A hybrid XGBoost-LSTM model in FAEP further refines these augmented features, resulting in a more robust prediction framework. Experimental results demonstrate that FAEP achieves state-of-art (SOTA) performance compared to other electricity price prediction models in the Australian New South Wale electricity market, showcasing the efficiency of LLM-enhanced feature engineering and hybrid machine learning architectures.

cs.CE↗

arXiv2025

AdapCsiNet: Environment-Adaptive CSI Feedback via Scene Graph-Aided Deep Learning

Accurate channel state information (CSI) is critical for realizing the full potential of multiple-antenna wireless communication systems. While deep learning (DL)-based CSI feedback methods have shown promise in reducing feedback overhead, their generalization capability across varying propagation environments remains limited due to their data-driven nature. Existing solutions based on online training improve adaptability but impose significant overhead in terms of data collection and computational resources. In this work, we propose AdapCsiNet, an environment-adaptive DL-based CSI feedback framework that eliminates the need for online training. By integrating environmental information -- represented as a scene graph -- into a hypernetwork-guided CSI reconstruction process, AdapCsiNet dynamically adapts to diverse channel conditions. A two-step training strategy is introduced to ensure baseline reconstruction performance and effective environment-aware adaptation. Simulation results demonstrate that AdapCsiNet achieves up to 46.4% improvement in CSI reconstruction accuracy and matches the performance of online learning methods without incurring additional runtime overhead.

cs.IT↗

arXiv2025

CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuning

The rapid advancement of large vision-language models (LVLMs) has driven significant progress in multimodal tasks, enabling models to interpret, reason, and generate outputs across both visual and textual domains. While excelling in generative tasks, existing LVLMs often face limitations in tasks requiring high-fidelity representation learning, such as generating image or text embeddings for retrieval. Recent work has proposed finetuning LVLMs for representational learning, but the fine-tuned model often loses its generative capabilities due to the representational learning training paradigm. To address this trade-off, we introduce CAFe, a contrastive-autoregressive fine-tuning framework that enhances LVLMs for both representation and generative tasks. By integrating a contrastive objective with autoregressive language modeling, our approach unifies these traditionally separate tasks, achieving state-of-the-art results in both multimodal retrieval and multimodal generative benchmarks, including object hallucination (OH) mitigation. CAFe establishes a novel framework that synergizes embedding and generative functionalities in a single model, setting a foundation for future multimodal models that excel in both retrieval precision and coherent output generation.

cs.CV↗

arXiv2025

Survey on Modeling of Human-made Articulated Objects

3D modeling of articulated objects is a research problem within computer vision, graphics, and robotics. Its objective is to understand the shape and motion of the articulated components, represent the geometry and mobility of object parts, and create realistic models that reflect articulated objects in the real world. This survey provides a comprehensive overview of the current state-of-the-art in 3D modeling of articulated objects, with a specific focus on the task of articulated part perception and articulated object creation (reconstruction and generation). We systematically review and discuss the relevant literature from two perspectives: geometry modeling (i.e., structure and shape of articulated parts) and articulation modeling (i.e., dynamics and motion of parts). Through this survey, we highlight the substantial progress made in these areas, outline the ongoing challenges, and identify gaps for future research. Our survey aims to serve as a foundational reference for researchers and practitioners in computer vision and graphics, offering insights into the complexities of articulated object modeling.

cs.CV↗

arXiv2025

SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects

We address the challenge of creating 3D assets for household articulated objects from a single image. Prior work on articulated object creation either requires multi-view multi-state input, or only allows coarse control over the generation process. These limitations hinder the scalability and practicality for articulated object modeling. In this work, we propose a method to generate articulated objects from a single image. Observing the object in resting state from an arbitrary view, our method generates an articulated object that is visually consistent with the input image. To capture the ambiguity in part shape and motion posed by a single view of the object, we design a diffusion model that learns the plausible variations of objects in terms of geometry and kinematics. To tackle the complexity of generating structured data with attributes in multiple domains, we design a pipeline that produces articulated objects from high-level structure to geometric details in a coarse-to-fine manner, where we use a part connectivity graph and part abstraction as proxies. Our experiments show that our method outperforms the state-of-the-art in articulated object creation by a large margin in terms of the generated object realism, resemblance to the input image, and reconstruction quality.

cs.CV↗

arXiv2025

Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillation

Hydra-MDP++ introduces a novel teacher-student knowledge distillation framework with a multi-head decoder that learns from human demonstrations and rule-based experts. Using a lightweight ResNet-34 network without complex components, the framework incorporates expanded evaluation metrics, including traffic light compliance (TL), lane-keeping ability (LK), and extended comfort (EC) to address unsafe behaviors not captured by traditional NAVSIM-derived teachers. Like other end-to-end autonomous driving approaches, \hydra processes raw images directly without relying on privileged perception signals. Hydra-MDP++ achieves state-of-the-art performance by integrating these components with a 91.0% drive score on NAVSIM through scaling to a V2-99 image encoder, demonstrating its effectiveness in handling diverse driving scenarios while maintaining computational efficiency.

cs.CV↗

arXiv2025

The Advancement of Personalized Learning Potentially Accelerated by Generative AI

The rapid development of Generative AI (GAI) has sparked revolutionary changes across various aspects of education. Personalized learning, a focal point and challenge in educational research, has also been influenced by the development of GAI. To explore GAI's extensive impact on personalized learning, this study investigates its potential to enhance various facets of personalized learning through a thorough analysis of existing research. The research comprehensively examines GAI's influence on personalized learning by analyzing its application across different methodologies and contexts, including learning strategies, paths, materials, environments, and specific analyses within the teaching and learning processes. Through this in-depth investigation, we find that GAI demonstrates exceptional capabilities in providing adaptive learning experiences tailored to individual preferences and needs. Utilizing different forms of GAI across various subjects yields superior learning outcomes. The article concludes by summarizing scenarios where GAI is applicable in educational processes and discussing strategies for leveraging GAI to enhance personalized learning, aiming to guide educators and learners in effectively utilizing GAI to achieve superior learning objectives.

cs.AI↗