arXiv subjects

Xinyi Wang

Publications and source records attributed to Xinyi Wang.

At least 127 records · Page 7Linked to original sources

arXiv2025

Logits DeConfusion with CLIP for Few-Shot Learning

With its powerful visual-language alignment capability, CLIP performs well in zero-shot and few-shot learning tasks. However, we found in experiments that CLIP's logits suffer from serious inter-class confusion problems in downstream tasks, and the ambiguity between categories seriously affects the accuracy. To address this challenge, we propose a novel method called Logits DeConfusion, which effectively learns and eliminates inter-class confusion in logits by combining our Multi-level Adapter Fusion (MAF) module with our Inter-Class Deconfusion (ICD) module. Our MAF extracts features from different levels and fuses them uniformly to enhance feature representation. Our ICD learnably eliminates inter-class confusion in logits with a residual structure. Experimental results show that our method can significantly improve the classification performance and alleviate the inter-class confusion problem. The code is available at https://github.com/LiShuo1001/LDC.

cs.CV↗

arXiv2025

NLS: Natural-Level Synthesis for Hardware Implementation Through GenAI

This paper introduces Natural-Level Synthesis, an innovative approach for generating hardware using generative artificial intelligence on both the system level and component-level. NLS bridges a gap in current hardware development processes, where algorithm and application engineers' involvement typically ends at the requirements stage. With NLS, engineers can participate more deeply in the development, synthesis, and test stages by using Gen-AI models to convert natural language descriptions directly into Hardware Description Language code. This approach not only streamlines hardware development but also improves accessibility, fostering a collaborative workflow between hardware and algorithm engineers. We developed the NLS tool to facilitate natural language-driven HDL synthesis, enabling rapid generation of system-level HDL designs while significantly reducing development complexity. Evaluated through case studies and benchmarks using Performance, Power, and Area metrics, NLS shows its potential to enhance resource efficiency in hardware development. This work provides a extensible, efficient solution for hardware synthesis and establishes a Visual Studio Code Extension to assess Gen-AI-driven HDL generation and system integration, laying a foundation for future AI-enhanced and AI-in-the-loop Electronic Design Automation tools.

cs.AR↗

arXiv2025

ReLaX-VQA: Residual Fragment and Layer Stack Extraction for Enhancing Video Quality Assessment

With the rapid growth of User-Generated Content (UGC) exchanged between users and sharing platforms, the need for video quality assessment in the wild is increasingly evident. UGC is typically acquired using consumer devices and undergoes multiple rounds of compression (transcoding) before reaching the end user. Therefore, traditional quality metrics that employ the original content as a reference are not suitable. In this paper, we propose ReLaX-VQA, a novel No-Reference Video Quality Assessment (NR-VQA) model that aims to address the challenges of evaluating the quality of diverse video content without reference to the original uncompressed videos. ReLaX-VQA uses frame differences to select spatio-temporal fragments intelligently together with different expressions of spatial features associated with the sampled frames. These are then used to better capture spatial and temporal variabilities in the quality of neighbouring frames. Furthermore, the model enhances abstraction by employing layer-stacking techniques in deep neural network features from Residual Networks and Vision Transformers. Extensive testing across four UGC datasets demonstrates that ReLaX-VQA consistently outperforms existing NR-VQA methods, achieving an average SRCC of 0.8658 and PLCC of 0.8873. Open-source code and trained models that will facilitate further research and applications of NR-VQA can be found at https://github.com/xinyiW915/ReLaX-VQA.

eess.IV↗

arXiv2025

A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data

Automatic radiology report generation can alleviate the workload for physicians and minimize regional disparities in medical resources, therefore becoming an important topic in the medical image analysis field. It is a challenging task, as the computational model needs to mimic physicians to obtain information from multi-modal input data (i.e., medical images, clinical information, medical knowledge, etc.), and produce comprehensive and accurate reports. Recently, numerous works have emerged to address this issue using deep-learning-based methods, such as transformers, contrastive learning, and knowledge-base construction. This survey summarizes the key techniques developed in the most recent works and proposes a general workflow for deep-learning-based report generation with five main components, including multi-modality data acquisition, data preparation, feature learning, feature fusion and interaction, and report generation. The state-of-the-art methods for each of these components are highlighted. Additionally, we summarize the latest developments in large model-based methods and model explainability, along with public datasets, evaluation methods, current challenges, and future directions in this field. We have also conducted a quantitative comparison between different methods in the same experimental setting. This is the most up-to-date survey that focuses on multi-modality inputs and data fusion for radiology report generation. The aim is to provide comprehensive and rich information for researchers interested in automatic clinical report generation and medical image analysis, especially when using multimodal inputs, and to assist them in developing new algorithms to advance the field.

cs.CV↗

arXiv2025

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data

The impressive capabilities of large language models (LLMs) have sparked debate over whether these models genuinely generalize to unseen tasks or predominantly rely on memorizing vast amounts of pretraining data. To explore this issue, we introduce an extended concept of memorization, distributional memorization, which measures the correlation between the LLM output probabilities and the pretraining data frequency. To effectively capture task-specific pretraining data frequency, we propose a novel task-gram language model, which is built by counting the co-occurrence of semantically related $n$-gram pairs from task inputs and outputs in the pretraining corpus. Using the Pythia models trained on the Pile dataset, we evaluate four distinct tasks: machine translation, factual question answering, world knowledge understanding, and math reasoning. Our findings reveal varying levels of memorization, with the strongest effect observed in factual question answering. Furthermore, while model performance improves across all tasks as LLM size increases, only factual question answering shows an increase in memorization, whereas machine translation and reasoning tasks exhibit greater generalization, producing more novel outputs. This study demonstrates that memorization plays a larger role in simpler, knowledge-intensive tasks, while generalization is the key for harder, reasoning-based tasks, providing a scalable method for analyzing large pretraining corpora in greater depth.

cs.CL↗

arXiv2025

The effect of remote work on urban transportation emissions: evidence from 141 cities

The overall impact of working from home (WFH) on transportation emissions remains a complex issue, with significant implications for policymaking. This study matches socioeconomic information from American Community Survey (ACS) to the global carbon emissions dataset for selected Metropolitan Statistical Areas (MSAs) in the US. We analyze the impact of WFH on transportation emissions before and during the COVID-19 pandemic. Employing cross-sectional multiple regression models and Blinder-Oaxaca decomposition, we examine how WFH, commuting mode, and car ownership influence transportation emissions across 141 MSAs in the United States. We find that the prevalence of WFH in 2021 is associated with lower transportation emissions, whereas WFH in 2019 did not significantly impact transportation emissions. After controlling for public transportation usage and car ownership, we find that a 1% increase in WFH corresponds to a 0.17 kilogram or 1.8% reduction of daily average transportation emissions per capita. The Blinder-Oaxaca decomposition shows that WFH is the main driver in reducing transportation emissions per capita during the pandemic. Our results show that the reductive influence of public transportation on transportation emissions has declined, while the impact of car ownership on increasing transportation emissions has risen. Collectively, these results indicate a multifaceted impact of WFH on transportation emissions. This study underscores the need for a nuanced, data-driven approach in crafting WFH policies to mitigate transportation emissions effectively.

econ.GN↗

arXiv2025

Estimating Magnitude Completeness in Earthquake Catalogs: A Comparative Study of Catalog-based Methods

Without rigorous attention to the completeness of earthquake catalogs, claims of new discoveries or forecasting skills cannot be deemed credible. Therefore, estimating the completeness magnitude (Mc) is a critical step. Among various approaches, catalog-based methods are the simplest, most straightforward, and most commonly used. However, current evaluation frameworks for these methods lack a unified simulation strategy for generating catalogs that are independent of specific Mc estimation methods. An effective strategy should also be capable of simulating datasets with non-uniform Mc distributions across both spatial and temporal dimensions. In this study, we assess nine catalog-based methods under a robust evaluation framework specifically tailored for this purpose. These methods are tested on datasets with homogeneous and heterogeneous Mc distributions, as well as on real-world earthquake catalogs. The method of b-value stability by Woessner and Wiemer (2005), referred to as MBS-WW in this study, demonstrates the best overall performance. The prior model generated by MBS-WW is used as the foundation for generating an updated Mc map for China with the Bayesian Magnitude of Completeness (BMC) method. We also introduce, BSReLU, an augmented Gutenberg-Richter model with a novel probabilistic framework for modeling. The BSReLU model replaces deterministic estimates of Mc with a probabilistic framework that models the smooth transition in detection likelihood from zero to one as earthquake magnitudes increase. By evaluating the limitations of these foundational catalog-based methods, this study seeks to refine our understanding of their appropriate applications, offering a clearer, unbiased perspective on seismicity through improved observational data quality.

physics.geo-ph↗

arXiv2025

Optical Propulsion and Levitation of Metajets

The quintessential hallmark distinguishing metasurfaces from traditional optical components is the engineering of subwavelength meta-atoms to manipulate light at will. Enabling this freedom, in a reverse manner, to control objects constituted by metasurfaces could expand our capability of optical manipulation to go beyond the predominant microscopic and sub-microscopic scales. Here, we introduce a driving metaphotonic force fully controllable by meta-atoms to manipulate structured objects named metajets. Upon Newton's law of motion that can apply to classical and relativistic mechanics, we develop a first-principles theory to analyze optical forces generated by refraction and reflection at an interface. We find that three-dimensional motions of metajets would be possible if one could introduce an extra wavevector component. We achieve that by creating a spatially distributed phase gradient with deliberately arranged silicon nanopillars. Our experiments and simulations reveal an in-plane propulsion and, very importantly, out-of-plane levitation of the metajets, aligning well with the theory. We also find that the metaphotonic force augments with increased light power but is not limited by the size of metajets, which could unleash new opportunities for metaphotonic control in large settings, such as interstellar light sails.

physics.optics↗

arXiv2025

Grid Monitoring with Synchro-Waveform and AI Foundation Model Technologies

Purpose:This article advocates for the development of a next-generation grid monitoring and control system designed for future grids dominated by inverter-based resources. Leveraging recent progress in generative artificial intelligence (AI), machine learning, and networking technology, we develop a physics-based AI foundation model with high-resolution synchro-waveform measurement technology to enhance grid resilience and reduce economic losses from outages. Methods and Results:The proposed framework adopts the AI Foundation Model paradigm, where a generative and pre-trained (GPT) foundation model extracts physical features from power system measurements, enabling adaptation to a wide range of grid operation tasks. Replacing the large language models used in popular AI foundation models, this approach is based on the Wiener-Kallianpur-Rosenblatt innovation model for power system time series, trained to capture the physical laws of power flows and sinusoidal characteristics of grid measurements. The pre-trained foundation model causally extracts sufficient statistics from grid measurement time series for various downstream applications, including anomaly detection, over-current protection, probabilistic forecasting, and data compression for streaming synchro-waveform data. Numerical simulations using field-collected data demonstrate significantly improved fault detection accuracy and detection speed. Conclusion:The future grid will be rich in inverter-based resources, making it highly dynamic, stochastic, and low inertia. This work underscores the limitations of existing Supervisory-Control-and-Data-Acquisition and Phasor-Measurement-Unit monitoring systems and advocates for AI-enabled monitoring and control with high-resolution synchro-waveform technology to provide accurate situational awareness, rapid response to faults, and robust network protection.

eess.SY↗

arXiv2025

AugRefer: Advancing 3D Visual Grounding via Cross-Modal Augmentation and Spatial Relation-based Referring

3D visual grounding (3DVG), which aims to correlate a natural language description with the target object within a 3D scene, is a significant yet challenging task. Despite recent advancements in this domain, existing approaches commonly encounter a shortage: a limited amount and diversity of text3D pairs available for training. Moreover, they fall short in effectively leveraging different contextual clues (e.g., rich spatial relations within the 3D visual space) for grounding. To address these limitations, we propose AugRefer, a novel approach for advancing 3D visual grounding. AugRefer introduces cross-modal augmentation designed to extensively generate diverse text-3D pairs by placing objects into 3D scenes and creating accurate and semantically rich descriptions using foundation models. Notably, the resulting pairs can be utilized by any existing 3DVG methods for enriching their training data. Additionally, AugRefer presents a language-spatial adaptive decoder that effectively adapts the potential referring objects based on the language description and various 3D spatial relations. Extensive experiments on three benchmark datasets clearly validate the effectiveness of AugRefer.

cs.CV↗

arXiv2024

IMNet: Interference-Aware Channel Knowledge Map Construction and Localization

This paper presents a novel two-stage method for constructing channel knowledge maps (CKMs) specifically for A2G (Aerial-to-Ground) channels in the presence of non-cooperative interfering nodes (INs). We first estimate the interfering signal strength (ISS) at sampling locations based on total received signal strength measurements and the desired communication signal strength (DSS) map constructed with environmental topology. Next, an ISS map construction network (IMNet) is proposed, where a negative value correction module is included to enable precise reconstruction. Subsequently, we further execute signal-to-interference-plus-noise ratio map construction and IN localization. Simulation results demonstrate lower construction error of the proposed IMNet compared to baselines in the presence of interference.

eess.SP↗

arXiv2024

Real-time Hosting Capacity Assessment for Electric Vehicles: A Sequential Forecast-then-Optimize Method

Hosting capacity (HC) assessment for electric vehicles (EVs) is crucial for EV secure integration and reliable power system operation. Existing methods primarily focus on a long-term perspective (e.g., system planning), and consider the EV charging demands as scalar values, which introduces inaccuracies in real-time operations due to the inherently stochastic nature of EVs. In this regard, this paper proposes a real-time HC assessment method for EVs through a three-step process, involving real-time probabilistic forecasting, risk analysis and probabilistic optimization. Specifically, we conduct real-time probabilistic forecasting to capture the stochastic nature of EV charging demands across multiple charging stations by performing deterministic forecasting and fitting the distribution of forecasting errors. The deterministic forecasting is conducted using an adaptive spatio-temporal graph convolutional network (ASTGCN). ASTGCN leverages adaptive spatial feature extraction, attention-based temporal feature extraction, and second-order graph representation to improve the forecasting performance. Subsequently, based on the probabilistic forecasting of EV charging demands, we conduct real-time risk analysis and operational boundary identification by utilizing probabilistic power flow calculations to assess potential violations of secure operation constraints. Furthermore, we present the formulation of real-time HC of EVs considering expected satisfaction of stochastic EV charging demands, and propose an optimization model for real-time HC assessment of EVs. Numerical experiments on a real-world dataset demonstrate that the proposed ASTGCN model outperforms state-of-the-art forecasting models by achieving the lowest root mean square error of 0.0442, and the real-time HC is improved by 64% compared to long-term HC assessment.

math.OC↗

arXiv2024

Underwater Image Enhancement with Cascaded Contrastive Learning

Underwater image enhancement (UIE) is a highly challenging task due to the complexity of underwater environment and the diversity of underwater image degradation. Due to the application of deep learning, current UIE methods have made significant progress. Most of the existing deep learning-based UIE methods follow a single-stage network which cannot effectively address the diverse degradations simultaneously. In this paper, we propose to address this issue by designing a two-stage deep learning framework and taking advantage of cascaded contrastive learning to guide the network training of each stage. The proposed method is called CCL-Net in short. Specifically, the proposed CCL-Net involves two cascaded stages, i.e., a color correction stage tailored to the color deviation issue and a haze removal stage tailored to improve the visibility and contrast of underwater images. To guarantee the underwater image can be progressively enhanced, we also apply contrastive loss as an additional constraint to guide the training of each stage. In the first stage, the raw underwater images are used as negative samples for building the first contrastive loss, ensuring the enhanced results of the first color correction stage are better than the original inputs. While in the second stage, the enhanced results rather than the raw underwater images of the first color correction stage are used as the negative samples for building the second contrastive loss, thus ensuring the final enhanced results of the second haze removal stage are better than the intermediate color corrected results. Extensive experiments on multiple benchmark datasets demonstrate that our CCL-Net can achieve superior performance compared to many state-of-the-art methods. The source code of CCL-Net will be released at https://github.com/lewis081/CCL-Net.

cs.CV↗

arXiv2024

Quantum Neural Network Classifier for Cancer Registry System Testing: A Feasibility Study

The Cancer Registry of Norway (CRN) is a part of the Norwegian Institute of Public Health (NIPH) and is tasked with producing statistics on cancer among the Norwegian population. For this task, CRN develops, tests, and evolves a software system called Cancer Registration Support System (CaReSS). It is a complex socio-technical software system that interacts with many entities (e.g., hospitals, medical laboratories, and other patient registries) to achieve its task. For cost-effective testing of CaReSS, CRN has employed EvoMaster, an AI-based REST API testing tool combined with an integrated classical machine learning model. Within this context, we propose Qlinical to investigate the feasibility of using, inside EvoMaster, a Quantum Neural Network (QNN) classifier, i.e., a quantum machine learning model, instead of the existing classical machine learning model. Results indicate that Qlinical can achieve performance comparable to that of EvoClass. We further explore the effects of various QNN configurations on performance and offer recommendations for optimal QNN settings for future QNN developers.

cs.SE↗

arXiv2024

T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback

Diffusion-based text-to-video (T2V) models have achieved significant success but continue to be hampered by the slow sampling speed of their iterative sampling processes. To address the challenge, consistency models have been proposed to facilitate fast inference, albeit at the cost of sample quality. In this work, we aim to break the quality bottleneck of a video consistency model (VCM) to achieve $\textbf{both fast and high-quality video generation}$. We introduce T2V-Turbo, which integrates feedback from a mixture of differentiable reward models into the consistency distillation (CD) process of a pre-trained T2V model. Notably, we directly optimize rewards associated with single-step generations that arise naturally from computing the CD loss, effectively bypassing the memory constraints imposed by backpropagating gradients through an iterative sampling process. Remarkably, the 4-step generations from our T2V-Turbo achieve the highest total score on VBench, even surpassing Gen-2 and Pika. We further conduct human evaluations to corroborate the results, validating that the 4-step generations from our T2V-Turbo are preferred over the 50-step DDIM samples from their teacher models, representing more than a tenfold acceleration while improving video generation quality.

cs.CV↗

arXiv2024

Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization

Parameter-efficient fine-tuning (PEFT) using labeled task data can significantly improve the performance of large language models (LLMs) on the downstream task. However, there are 7000 languages in the world and many of these languages lack labeled data for real-world language generation tasks. In this paper, we propose to improve zero-shot cross-lingual transfer by composing language or task specialized parameters. Our method composes language and task PEFT modules via element-wise arithmetic operations to leverage unlabeled data and English labeled data. We extend our approach to cases where labeled data from more languages is available and propose to arithmetically compose PEFT modules trained on languages related to the target. Empirical results on summarization demonstrate that our method is an effective strategy that obtains consistent gains using minimal training of PEFT modules.

cs.CL↗

arXiv2024

Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

Large language models (LLMs) encode vast amounts of knowledge during pre-training (parametric knowledge, or PK) and can further be enhanced by incorporating contextual knowledge (CK). Can LLMs effectively integrate their internal PK with external CK to solve complex problems? In this paper, we investigate the dynamic interaction between PK and CK, categorizing their relationships into four types: Supportive, Complementary, Conflicting, and Irrelevant. To support this investigation, we introduce ECHOQA, a benchmark spanning scientific, factual, and commonsense knowledge. Our results show that LLMs tend to suppress their PK when contextual information is available, even when it is complementary or irrelevant. While tailored instructions can encourage LLMs to rely more on their PK, they still struggle to fully leverage it. These findings reveal a key vulnerability in LLMs, raising concerns about their reliability in knowledge-intensive tasks. Resources are available at https://github.com/sitaocheng/Knowledge_Interplay

cs.CL↗

arXiv2024

Probabilistic Forecasting of Real-Time Electricity Market Signals via Interpretable Generative AI

This paper introduces a generative AI approach to probabilistic forecasting of real-time electricity market signals, including locational marginal prices, interregional price spreads, and demand-supply imbalances. We present WIAE-GPF, a Weak Innovation AutoEncoder-based Generative Probabilistic Forecasting architecture that generates future samples of multivariate time series. Unlike traditional black-box models, WIAE-GPF offers interpretability through the Wiener-Kallianpur innovation representation for nonparametric time series, making it a nonparametric generalization of the Wiener/Kalman filter-based forecasting. A novel learning algorithm with structural convergence guarantees is proposed, ensuring that, under ideal training conditions, the generated forecast samples match the ground truth conditional probability distribution. Extensive tests using publicly available data from U.S. independent system operators under various point and probabilistic forecasting metrics demonstrate that WIAE-GPF consistently outperforms classical methods and cutting-edge machine learning techniques.

eess.SP↗