arXiv subjects

Dingyi Lu

Publications and source records attributed to Dingyi Lu.

4 recordsLinked to original sources

arXiv2026

InfraPatch: Cross-Task Targeted Grayscale Patch Attacks on Infrared-Adapted Vision-Language Models

Infrared vision-language models (IR-VLMs) have emerged as a promising paradigm for multimodal perception under low-visibility conditions, yet their robustness to targeted adversarial attacks remains poorly understood. Existing adversarial patch methods mainly study RGB-based models or a single downstream task and do not characterize whether localized perturbations can induce an intended semantic target in IR-VLMs. We propose InfraPatch, a white-box, per-instance framework for targeted digital grayscale patch attacks against IR-VLMs. InfraPatch optimizes a compact single-channel patch within an approximately 5% local-area budget, combines proxy-guided placement with task-adaptive semantic objectives, and induces target behaviors in image classification, image captioning, and binary visual question answering. We evaluate ten infrared-adapted model variants on 300 synthetic infrared-style images generated by applying DiffV2IR to a fixed 30-category COCO subset, using clean-conditioned targeted success criteria. InfraPatch achieves targeted attack success rates from 86.00% to 100% across the ten variants. On CLIP and BLIP-2, proxy location search improves success by 6.67 and 10.33 percentage points over optimized random placement, respectively; LLaVA-1.5 remains saturated near 100% under both settings. Patch-area and objective ablations further expose substantial differences in vulnerability across architectures and task formats. These results show that small grayscale patches can inject chosen target semantics across IR-VLM families under a controlled digital threat model, motivating stronger robustness evaluation for infrared multimodal systems.

cs.CV↗

arXiv2026

FusionRS: A Large-Scale RGB-Infrared-Style Remote Sensing Dataset for Cross-Modal Vision-Language Learning

Remote sensing vision-language models have advanced Earth observation, but available large-scale vision-language resources remain RGB-centered, leaving complementary infrared information underexplored. Infrared observations provide distinctive intensity structures, object boundaries, and illumination-invariant cues that complement conventional RGB imagery, yet large-scale RGB-infrared-text resources remain scarce. We introduce FusionRS, the first large-scale RGB-infrared-style-text dataset for controlled dual-modal remote sensing vision-language learning. It contains 600,000 spatially aligned pairs created by translating diverse public RGB remote sensing images into infrared-style counterparts. Each pair retains a conventional scene caption, and a curated subset adds 45,913 IR-aware captions describing observable intensity, contrast, texture, and structure while preserving scene semantics. We train CLIP-style models for RGB-infrared-style-text alignment and adapt a generative vision-language model with mixed task-conditioned caption supervision. Evaluation covers cross-modal retrieval, scaling and supervision ablations, sensor-captured transfer, and strictly held-out captioning and VQA. FusionRS substantially improves RGB-infrared-style alignment and infrared-to-text retrieval over RGB-only and non-IR-aware settings. Ablations show that IR-aware captions improve task-conditioned infrared description, demonstrating the value of modality-specific supervision. FusionRS provides a scalable foundation for controlled RGB-infrared remote sensing vision-language learning.

cs.CV↗

arXiv2026

Multimodal Large Language Model Enabled Robust Beamforming for HAP Downlink Communications

Small changes in high altitude platform (HAP) attitude can cause significant deviations in HAP downlink beam directions, thereby severely degrading HAP downlink communication performance. In this paper, we develop a multimodal large language model (LLM) enabled beamforming framework to achieve robust HAP downlink communications.Specifically, we design a vision-language LLM (VL-LLM) that learns from multivariate flight telemetry to forecast short-term HAP attitudes under platform shaking and support delay-aware proactive beam steering.We design an offline forecast-error calibration procedure to obtain upper bounds on forecast errors and improve the reliability of proactive analog beam steering.Based on the attitude forecasts, we proactively update the analog beamformer and propose a QoS-driven beamforming and admission method with a lightweight feasibility-enforcement step to satisfy instantaneous transmit-power and QoS requirements.Simulation results indicate that the designed VL-LLM can accurately capture changes in the HAP attitude and the proposed beamforming method achieves a 22.1% higher user service ratio and a 12.5% higher sum-rate than representative baselines.The measured mean and p99 total latencies are 36.24 ms and 40.13 ms, respectively, supporting practical delay-aware deployment.

cs.NI↗

arXiv2026

Generative AI Agent Empowered Power Allocation for HAP Propulsion and Communication Systems

High altitude platforms (HAPs) are emerging as a key enabler for 6G coverage, yet limited energy must be split between propulsion and communications. Most prior HAP studies ignore propulsion power or rely on surrogates that miss hull-propeller interference, leading to misestimated communication power budgets and degraded beamforming. More importantly, HAP power allocation is intrinsically a multi-system, multidisciplinary problem in which aerodynamics, propulsion-system efficiency, and communication-system performance (quality of service (QoS) and energy efficiency (EE)) are tightly coupled.To address these challenges, this paper designs an interactive generative artificial intelligence (AI)-empowered HAP power allocation agent.By interacting with the AI agent, we develop an accurate propulsion power consumption model that takes into account the aerodynamic interference between the HAP's hull and the propeller. Assisted by the AI agent, we further formulate a HAP beamforming problem to improve user QoS and enhance the EE of the HAP communication system.This paper also proposes a QoS-enhanced energy-efficient (Q3E) beamforming algorithm to solve the formulated problem.Simulation results demonstrate the accuracy of the propulsion-power model and the effectiveness of the Q3E algorithm.

cs.NI↗