Search arXiv⌕ Search

arXiv · 2611.10257

VeV: Video-Enhanced Vibration Data Augmentation For Structural Difference

Abstract

Surface-mounted vibration sensing provides non-invasive structural monitoring with high temporal fidelity across diverse ubiquitous computing scenarios such as smart farms, yet structure-borne wave models remain fragile to physical relocation and structural shifts. However, addressing this spatial data discontinuity remains limited by costly manual recalibration, insufficient physical grounding in data-driven synthesis, and substantial sim-to-real gaps in numerical digital twins. To resolve this spatial bottleneck without labor-intensive data recollection, we propose VeV, a physics-consistent spatial augmentation framework that synthesizes vibration responses at unmeasured target coordinates from sparse reference observations by coupling a monocular Vision-Derived Propagation Possibility Map for structural wave-guiding paths with a Visual Feature Map for transient source kinetics and implicit material dynamics. Evaluated on a 1,237 sensor-hour sow pig and piglet farm across disparate structures, VeV reduces waveform reconstruction MSE by over 57.5% compared to data-driven baselines and achieves up to 81.69% zero-shot classification accuracy, while reaching peak performance within just 30-50 calibration chunks to effectively eliminate cold-start retraining overhead and establish a scalable foundation extending to human-centric floor vibration sensing and ambient intelligence.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Joonhee Lee, Jiale Zhang, Sungmin Lee, Ruiqing Wang, Jeffrey Shulkin, Yiwen Dong, Pei Zhang, JeongGil Ko. 2026-11-06. VeV: Video-Enhanced Vibration Data Augmentation For Structural Difference. https://arxiv.org/abs/2611.10257

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

HexTiles and Semantic Icons for MAUP-Aware Multivariate Geospatial Visualizations

We introduce HexTiles, a hexagonal-tiling based visual encoding design for multivariate geospatial data. Visualizing several geospatial variables at once is difficult because map position is already reserved for geography, and because variables are often reported on different, irregular spatial units whose aggregation is subject to the Modifiable Areal Unit Problem (MAUP). HexTiles bins data of differing spatial discretizations into hierarchical hexagons and encodes up to three variables per tile in a base color, an inner ring, and semantic icons whose count conveys value, letting the visualization designer guide attention to specific variables and their interactions. To make the effects of MAUP visible, we compute a spatially-weighted variance for each variable within each tile and encode it as a confidence value in every channel, so that the variability hidden by aggregation is not lost. Because interactive maps are increasingly viewed in tilted, 3D perspective, HexTiles encodes quantitative values only in channels that perspective does not distort, namely color and icon count. In a user study comparing HexTiles to Square-glyphs, participants rated HexTiles as significantly less demanding and frustrating, and performed more consistently with it. Domain experts in ecology and hydrology further noted its value for communicating regional patterns, and that the confidence encoding signals where aggregated values should be interpreted with care. Our results suggest that, although 3D is commonly discouraged, it can work for multivariate geospatial visualization when restricted to perspective-robust channels.

cs.HC↗

AURA: A Reinforcement Learning Framework for AI-Driven Adaptive Conversational Surveys

Conventional online surveys provide limited personalization, often resulting in low engagement and superficial responses. Although AI survey chatbots improve convenience, most are still reactive: they rely on fixed dialogue trees or static prompt templates and therefore cannot adapt within a session to fit individual users, which leads to generic follow-ups and weak response quality. We address these limitations with AURA (Adaptive Understanding through Reinforcement Learning for Assessment), a reinforcement learning framework for AI-driven adaptive conversational surveys. AURA quantifies response quality using a four-dimensional LSDE metric (Length, Self-disclosure, Emotion, and Specificity) and selects follow-up question types via an epsilon-greedy policy that updates the expected quality gain within each session. Initialized with priors extracted from 96 prior campus-climate conversations (467 total chatbot-user exchanges), the system balances exploration and exploitation across 10-15 dialogue exchanges, dynamically adapting to individual participants in real time. In controlled evaluations, AURA achieved a +0.076 mean gain in response quality and a statistically significant improvement over non-adaptive baselines (p=0.044, d=0.66), driven by a 63% reduction in specification prompts and a 10x increase in validation behavior. These results demonstrate that reinforcement learning can give survey chatbots improved adaptivity, transforming static questionnaires into interactive, self-improving assessment systems.

cs.HC↗

Human-AI Collaboration in Large Language Model-Integrated Building Energy Management Systems: The Role of User Domain Knowledge and AI Literacy

This study aimed to comprehend how user domain knowledge and artificial intelligence (AI) literacy impact the effective use of human-AI interactive building energy management system (BEMS). While prior studies have investigated the potential of integrating large language models (LLMs) into BEMS or building energy modeling, very few studies have examined how user interact with such systems. We conducted a systematic role-playing experiment, where 85 human subjects interacted with an advanced generative pre-trained transformer (OpenAI GPT-4o). Participants were tasked with identifying the top five behavioral changes that could reduce home energy use with the GPT model that functioned as an LLM-integrated BEMS. Then, the collected prompt-response data and participant conclusions were analyzed using an analytical framework that hierarchically assessed and scored human-AI interactions and their home energy analysis approaches. Also, participants were classified into four groups based on their self-evaluated domain knowledge of building energy use and AI literacy, and Kruskal-Wallis H tests with post-hoc pairwise comparisons were conducted across 20 quantifiable metrics. Key takeaways include: most participants employed concise prompts (median: 16.2 words) and relied heavily on GPT's analytical capabilities; and notably, only 1 of 20 metrics, appliance identification rate, showed statistically significant group differences (p=0.037), driven by AI literacy rather than domain knowledge, suggesting an equalizing effect of LLMs across expertise levels. This study provides foundational insights into human-AI collaboration dynamics and promising development directions in the context of LLM-integrated BEMS and contributes to realizing human-centric LLM-integrated energy systems.

cs.HC↗