arXiv subjects

Jian Chen

Publications and source records attributed to Jian Chen.

At least 127 records · Page 7Linked to original sources

arXiv2025

MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) have achieved remarkable visual reasoning abilities in natural images, text-rich documents, and graphic designs. However, their ability to interpret music sheets remains underexplored. To bridge this gap, we introduce MusiXQA, the first comprehensive dataset for evaluating and advancing MLLMs in music sheet understanding. MusiXQA features high-quality synthetic music sheets generated via MusiXTeX, with structured annotations covering note pitch and duration, chords, clefs, key/time signatures, and text, enabling diverse visual QA tasks. Through extensive evaluations, we reveal significant limitations of current state-of-the-art MLLMs in this domain. Beyond benchmarking, we developed Phi-3-MusiX, an MLLM fine-tuned on our dataset, achieving significant performance gains over GPT-based methods. The proposed dataset and model establish a foundation for future advances in MLLMs for music sheet understanding. Code, data, and model will be released upon acceptance.

cs.CV↗

arXiv2025

Multimodal LLMs as Customized Reward Models for Text-to-Image Generation

We introduce LLaVA-Reward, an efficient reward model designed to automatically evaluate text-to-image (T2I) generations across multiple perspectives, leveraging pretrained multimodal large language models (MLLMs). Existing MLLM-based approaches require instruction-following data for supervised fine-tuning and evaluate generation quality on analyzing text response, which is time-consuming and difficult to train. To address this problem, we propose LLaVA-Reward, which directly utilizes the hidden states of MLLMs given text-image pairs. To enhance the bidirectional interaction between visual and textual representations in decoder-only MLLMs, we further propose adding a Skip-connection Cross Attention (SkipCA) module. This design enhances text-image correlation reasoning by connecting early-layer visual features with later-layer hidden representations. In addition, LLaVA-Reward supports different types of preference data for efficient fine-tuning, including paired preference data and unpaired data. We train LLaVA-Reward on four evaluation perspectives: text-image alignment, fidelity/artifact, safety, and overall ranking. Empirical results demonstrate that LLaVA-Reward outperforms conventional and MLLM-based methods in generating human-aligned scores for automatic evaluations and inference-time scaling in text-to-image generations.

cs.CV↗

arXiv2025

Orthorhombic nitride perovskite CeTaN3-δ with switchable and robust ferroelectric polarization

Perovskite-type ternary nitrides with predicted exciting ferroelectricity and many other outstanding properties hold great promise to be an emerging class of advanced ferroelectrics for manufacturing diverse technologically important devices. However, such nitride ferroelectrics have not yet been experimentally identified, mainly due to the challenging sample synthesis by traditional methods at ambient pressure. Here we report the successful high-pressure synthesis of a high-quality ferroelectric nitride perovskite of CeTaN3-δ with nitrogen deficiency, adopting an orthorhombic Pmn21 polar structure. This material is electrically insulating and exhibits switchable and robust electric polarization for producing ferroelectricity. Furthermore, a number of other extraordinary properties are also revealed in this nitride such as excellent mechanical properties and chemical inertness, which would make it practically useful for many device-relevant applications and fundamentally important for the study of condensed-matter physics.

cond-mat.mtrl-sci↗

arXiv2025

ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios

The emergence of Large Language Models (LLMs) presents transformative opportunities for education, generating numerous novel application scenarios. However, significant challenges remain: evaluation metrics vary substantially across different educational scenarios, while many emerging scenarios lack appropriate assessment metrics. Current benchmarks predominantly measure general intelligence rather than pedagogical capabilities. To address this gap, we introduce ELMES, an open-source automated evaluation framework specifically designed for assessing LLMs in educational settings. ELMES features a modular architecture that enables researchers to create dynamic, multi-agent dialogues through simple configuration files, facilitating flexible scenario design without requiring extensive programming expertise. The framework incorporates a hybrid evaluation engine that objectively quantifies traditionally subjective pedagogical metrics using an LLM-as-a-Judge methodology. We conduct systematic benchmarking of state-of-the-art LLMs across four critical educational scenarios: Knowledge Point Explanation, Guided Problem-Solving Teaching, Interdisciplinary Lesson Plan Generation, and Contextualized Question Generation, employing fine-grained metrics developed in collaboration with education specialists. Our results demonstrate distinct capability distributions among models, revealing context-specific strengths and limitations. ELMES provides educators and researchers with an accessible evaluation framework that significantly reduces adaptation barriers for diverse educational applications while advancing the practical implementation of LLMs in pedagogy. The framework is publicly available at \emph{https://github.com/sii-research/elmes.git}.

cs.CY↗

arXiv2025

MGHFT: Multi-Granularity Hierarchical Fusion Transformer for Cross-Modal Sticker Emotion Recognition

Although pre-trained visual models with text have demonstrated strong capabilities in visual feature extraction, sticker emotion understanding remains challenging due to its reliance on multi-view information, such as background knowledge and stylistic cues. To address this, we propose a novel multi-granularity hierarchical fusion transformer (MGHFT), with a multi-view sticker interpreter based on Multimodal Large Language Models. Specifically, inspired by the human ability to interpret sticker emotions from multiple views, we first use Multimodal Large Language Models to interpret stickers by providing rich textual context via multi-view descriptions. Then, we design a hierarchical fusion strategy to fuse the textual context into visual understanding, which builds upon a pyramid visual transformer to extract both global and local sticker features at multiple stages. Through contrastive learning and attention mechanisms, textual features are injected at different stages of the visual backbone, enhancing the fusion of global- and local-granularity visual semantics with textual guidance. Finally, we introduce a text-guided fusion attention mechanism to effectively integrate the overall multimodal features, enhancing semantic understanding. Extensive experiments on 2 public sticker emotion datasets demonstrate that MGHFT significantly outperforms existing sticker emotion recognition approaches, achieving higher accuracy and more fine-grained emotion recognition. Compared to the best pre-trained visual models, our MGHFT also obtains an obvious improvement, 5.4% on F1 and 4.0% on accuracy. The code is released at https://github.com/cccccj-03/MGHFT_ACMMM2025.

cs.CV↗

arXiv2025

A High Magnifications Histopathology Image Dataset for Oral Squamous Cell Carcinoma Diagnosis and Prognosis

Oral Squamous Cell Carcinoma (OSCC) is a prevalent and aggressive malignancy where deep learning-based computer-aided diagnosis and prognosis can enhance clinical assessments.However, existing publicly available OSCC datasets often suffer from limited patient cohorts and a restricted focus on either diagnostic or prognostic tasks, limiting the development of comprehensive and generalizable models. To bridge this gap, we introduce Multi-OSCC, a new histopathology image dataset comprising 1,325 OSCC patients, integrating both diagnostic and prognostic information to expand existing public resources. Each patient is represented by six high resolution histopathology images captured at x200, x400, and x1000 magnifications-two per magnification-covering both the core and edge tumor regions.The Multi-OSCC dataset is richly annotated for six critical clinical tasks: recurrence prediction (REC), lymph node metastasis (LNM), tumor differentiation (TD), tumor invasion (TI), cancer embolus (CE), and perineural invasion (PI). To benchmark this dataset, we systematically evaluate the impact of different visual encoders, multi-image fusion techniques, stain normalization, and multi-task learning frameworks. Our analysis yields several key insights: (1) The top-performing models achieve excellent results, with an Area Under the Curve (AUC) of 94.72% for REC and 81.23% for TD, while all tasks surpass 70% AUC; (2) Stain normalization benefits diagnostic tasks but negatively affects recurrence prediction; (3) Multi-task learning incurs a 3.34% average AUC degradation compared to single-task models in our multi-task benchmark, underscoring the challenge of balancing multiple tasks in our dataset. To accelerate future research, we publicly release the Multi-OSCC dataset and baseline models at https://github.com/guanjinquan/OSCC-PathologyImageDataset.

eess.IV↗

arXiv2025

Nonlinear skin effect regime when a radio frequency electromagnetic field penetrates into a background plasma

Two-dimensional, electromagnetic particle-in-cell simulations are employed to study particle kinetics and power deposition in the skin layer when a Radio Frequency (RF) electromagnetic field penetrates into a background plasma. We identify a new regime at low frequency ($\sim\mathrm{MHz}$) and low pressure, where the motion of electrons can be highly nonlinear in the skin region. Through most of the RF cycle, the electrons are trapped in the effective potential formed by the vector and electrostatic potentials, with energy deposition being small and magnetic moment $μ$ no longer being an adiabatic invariant. However, for a brief period around the null of the oscillating magnetic field, the electrons get detrapped, causing a jet-like current penetrating into the bulk plasma. During these brief periods, the power deposition becomes high, exhibiting a periodic burst nature. Based on kinetic theory, we provide analytical expressions for the plasma current and energy deposition in the new regime. A criterion for transition between the newly identified low-frequency, periodic-burst regime and the usual anomalous non-local skin effect regime is proposed and verified.

physics.plasm-ph↗

arXiv2025

Three-dimensional Helical-rotating Plasma Structures in Beam-generated Partially Magnetized Plasmas

Azimuthal structures emerging in beam-generated partially magnetized plasmas are investigated using three-dimensional particle-in-cell/Monte Carlo collision simulations. Two distinct instability regimes are identified at low pressures. When the gas pressure is sufficiently high, quasi-neutrality is attained and 2D spiral-arm structures form as a result of the development of a lower-hybrid instability, resulting in enhanced cross-field transport. At lower pressures, quasi-neutrality is not achieved and a 3D helical-rotating plasma structure forms due to development of the diocotron instability. Analytical formulas are proposed for the critical threshold pressure between these regimes and for the rotation frequency of the helical structures. Preliminary experimental verification is provided.

physics.plasm-ph↗

arXiv2025

Implementing van der Waals forces for polytope particles in DEM simulations of clay

Clay minerals are non-spherical nano-scale particles that usually form flocculated, house-of-card like structures under the influence of inter-molecular forces. Numerical modeling of clays is still in its infancy as the required inter-particle forces are available only for spherical particles. A polytope approach would allow shape-accurate forces and torques while simultaneously being more performant. The Anandarajah solution provides an analytical formulation for van der Waals forces for cuboid particles but in its original form is not suitable for implementation in DEM simulations. In this work, we discuss the necessary changes for a functional implementation of the Anandarajah solution in a DEM simulation of rectangular particles and their extension to cuboid particles.

cond-mat.soft↗

arXiv2025

Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation

Test-time adaptation (TTA) aims to boost the generalization capability of a trained model by conducting self-/unsupervised learning during the testing phase. While most existing TTA methods for video primarily utilize visual supervisory signals, they often overlook the potential contribution of inherent audio data. To address this gap, we propose a novel approach that incorporates audio information into video TTA. Our method capitalizes on the rich semantic content of audio to generate audio-assisted pseudo-labels, a new concept in the context of video TTA. Specifically, we propose an audio-to-video label mapping method by first employing pre-trained audio models to classify audio signals extracted from videos and then mapping the audio-based predictions to video label spaces through large language models, thereby establishing a connection between the audio categories and video labels. To effectively leverage the generated pseudo-labels, we present a flexible adaptation cycle that determines the optimal number of adaptation iterations for each sample, based on changes in loss and consistency across different views. This enables a customized adaptation process for each sample. Experimental results on two widely used datasets (UCF101-C and Kinetics-Sounds-C), as well as on two newly constructed audio-video TTA datasets (AVE-C and AVMIT-C) with various corruption types, demonstrate the superiority of our approach. Our method consistently improves adaptation performance across different video classification models and represents a significant step forward in integrating audio information into video TTA. Code: https://github.com/keikeiqi/Audio-Assisted-TTA.

cs.CV↗

arXiv2025

Rectified Sparse Attention

Efficient long-sequence generation is a critical challenge for Large Language Models. While recent sparse decoding methods improve efficiency, they suffer from KV cache misalignment, where approximation errors accumulate and degrade generation quality. In this work, we propose Rectified Sparse Attention (ReSA), a simple yet effective method that combines block-sparse attention with periodic dense rectification. By refreshing the KV cache at fixed intervals using a dense forward pass, ReSA bounds error accumulation and preserves alignment with the pretraining distribution. Experiments across math reasoning, language modeling, and retrieval tasks demonstrate that ReSA achieves near-lossless generation quality with significantly improved efficiency. Notably, ReSA delivers up to 2.42$\times$ end-to-end speedup under decoding at 256K sequence length, making it a practical solution for scalable long-context inference. Code is available at https://aka.ms/ReSA-LM.

cs.CL↗

arXiv2025

Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoning

Artificial intelligence (AI)-based chest X-ray (CXR) interpretation assistants have demonstrated significant progress and are increasingly being applied in clinical settings. However, contemporary medical AI models often adhere to a simplistic input-to-output paradigm, directly processing an image and an instruction to generate a result, where the instructions may be integral to the model's architecture. This approach overlooks the modeling of the inherent diagnostic reasoning in chest X-ray interpretation. Such reasoning is typically sequential, where each interpretive stage considers the images, the current task, and the contextual information from previous stages. This oversight leads to several shortcomings, including misalignment with clinical scenarios, contextless reasoning, and untraceable errors. To fill this gap, we construct CXRTrek, a new multi-stage visual question answering (VQA) dataset for CXR interpretation. The dataset is designed to explicitly simulate the diagnostic reasoning process employed by radiologists in real-world clinical settings for the first time. CXRTrek covers 8 sequential diagnostic stages, comprising 428,966 samples and over 11 million question-answer (Q&A) pairs, with an average of 26.29 Q&A pairs per sample. Building on the CXRTrek dataset, we propose a new vision-language large model (VLLM), CXRTrekNet, specifically designed to incorporate the clinical reasoning flow into the VLLM framework. CXRTrekNet effectively models the dependencies between diagnostic stages and captures reasoning patterns within the radiological context. Trained on our dataset, the model consistently outperforms existing medical VLLMs on the CXRTrek benchmarks and demonstrates superior generalization across multiple tasks on five diverse external datasets. The dataset and model can be found in our repository (https://github.com/guanjinquan/CXRTrek).

cs.CV↗

arXiv2025

Learning with Selectively Labeled Data from Multiple Decision-makers

We study the problem of classification with selectively labeled data, whose distribution may differ from the full population due to historical decision-making. We exploit the fact that in many applications historical decisions were made by multiple decision-makers, each with different decision rules. We analyze this setup under a principled instrumental variable (IV) framework and rigorously study the identification of classification risk. We establish conditions for the exact identification of classification risk and derive tight partial identification bounds when exact identification fails. We further propose a unified cost-sensitive learning (UCL) approach to learn classifiers robust to selection bias in both identification settings. Finally, we theoretically and numerically validate the efficacy of our proposed method.

stat.ML↗

arXiv2025

Photoswitchable exceptional points derived from bound states in the continuum

Bound states in the continuum (BICs) and exceptional points (EPs), as two distinct physical singularities represented by complex frequencies in non-Hermitian systems, have garnered significant attention and clear definitions in their respective fields in recent years. They share overlapping applications in areas such as high-sensitivity sensing and laser emission. However, the transition between the two, inspired by these intersections, remains largely unexplored. In this work, we reveal the transition process in a non-Hermitian two-mode system, evolving from one bound singularity to a two-dimensional exceptional ring, where the EP is the coalescent state of the quasi-Friedrich-Wintgen (FW)-BIC. This phenomenon is experimentally validated through pored dielectric metasurfaces in terahertz band. Furthermore, external pumping induced photocarriers as the dissipative perturbation, facilitates the breaking of degeneracy in the complex eigenfrequency and enables dynamic EP switching. Finally, we experimentally demonstrate a switchable terahertz beam deflection driven by the phase singularities of the EP. These findings are instrumental in advancing the development of compact devices for sensing and wavefront control within non-Hermitian systems.

physics.optics↗

arXiv2025

Integrated Sensing and Communications for Pinching-Antenna Systems (PASS)

An integrated sensing and communication (ISAC) design for pinching antenna systems (PASS) is proposed, where the pinching antennas are deployed to establish reliable line-of-sight communication and sensing links. More particularly, a separated ISAC design is proposed for the two-waveguide PASS, where one waveguide is used to emit the information-bearing signals for ISAC transmission while the other waveguide is used to receive the reflected echo signals. Based on this framework, a penalty-based alternating optimization algorithm is proposed to maximize the illumination power as well as ensure the communication quality-of-service requirement. Numerical results demonstrate that the proposed PASS-ISAC scheme outperforms the conventional antenna scheme.

cs.IT↗

arXiv2025

TopicVD: A Topic-Based Dataset of Video-Guided Multimodal Machine Translation for Documentaries

Most existing multimodal machine translation (MMT) datasets are predominantly composed of static images or short video clips, lacking extensive video data across diverse domains and topics. As a result, they fail to meet the demands of real-world MMT tasks, such as documentary translation. In this study, we developed TopicVD, a topic-based dataset for video-supported multimodal machine translation of documentaries, aiming to advance research in this field. We collected video-subtitle pairs from documentaries and categorized them into eight topics, such as economy and nature, to facilitate research on domain adaptation in video-guided MMT. Additionally, we preserved their contextual information to support research on leveraging the global context of documentaries in video-guided MMT. To better capture the shared semantics between text and video, we propose an MMT model based on a cross-modal bidirectional attention module. Extensive experiments on the TopicVD dataset demonstrate that visual information consistently improves the performance of the NMT model in documentary translation. However, the MMT model's performance significantly declines in out-of-domain scenarios, highlighting the need for effective domain adaptation methods. Additionally, experiments demonstrate that global context can effectively improve translation performance. % Dataset and our implementations are available at https://github.com/JinzeLv/TopicVD

cs.CL↗

arXiv2025

Generative Semantic Communication via Textual Prompts: Latency Performance Tradeoffs

This paper develops an edge-device collaborative Generative Semantic Communications (Gen SemCom) framework leveraging pre-trained Multi-modal/Vision Language Models (M/VLMs) for ultra-low-rate semantic communication via textual prompts. The proposed framework optimizes the use of M/VLMs on the wireless edge/device to generate high-fidelity textual prompts through visual captioning/question answering, which are then transmitted over a wireless channel for SemCom. Specifically, we develop a multi-user Gen SemCom framework using pre-trained M/VLMs, and formulate a joint optimization problem of prompt generation offloading, communication and computation resource allocation to minimize the latency and maximize the resulting semantic quality. Due to the nonconvex nature of the problem with highly coupled discrete and continuous variables, we decompose it as a two-level problem and propose a low-complexity swap/leaving/joining (SLJ)-based matching algorithm. Simulation results demonstrate significant performance improvements over the conventional semanticunaware/non-collaborative offloading benchmarks.

cs.IT↗

arXiv2025

Simulation of the Background from $^{13}$C$(α, n)^{16}$O Reaction in the JUNO Scintillator

Large-scale organic liquid scintillator detectors are highly efficient in the detection of MeV-scale electron antineutrinos. These signal events can be detected through inverse beta decay on protons, which produce a positron accompanied by a neutron. A noteworthy background for antineutrinos coming from nuclear power reactors and from the depths of the Earth (geoneutrinos) is generated by ($α, n$) reactions. In organic liquid scintillator detectors, $α$ particles emitted from intrinsic contaminants such as $^{238}$U, $^{232}$Th, and $^{210}$Pb/$^{210}$Po, can be captured on $^{13}$C nuclei, followed by the emission of a MeV-scale neutron. Three distinct interaction mechanisms can produce prompt energy depositions preceding the delayed neutron capture, leading to a pair of events correlated in space and time within the detector. Thus, ($α, n$) reactions represent an indistinguishable background in liquid scintillator-based antineutrino detectors, where their expected rate and energy spectrum are typically evaluated via Monte Carlo simulations. This work presents results from the open-source SaG4n software, used to calculate the expected energy depositions from the neutron and any associated de-excitation products. Also simulated is a detailed detector response to these interactions, using a dedicated Geant4-based simulation software from the JUNO experiment. An expected measurable $^{13}$C$(α, n)^{16}$O event rate and reconstructed prompt energy spectrum with associated uncertainties, are presented in the context of JUNO, however, the methods and results are applicable and relevant to other organic liquid scintillator neutrino detectors.

physics.ins-det↗