Search arXiv⌕ Search

arXiv subjects

Shipeng Liu

Publications and source records attributed to Shipeng Liu.

At least 19 recordsLinked to original sources
arXiv2026

Rapid-Deployment Crack Measurement Based on SAM3 Semantic-Edge Response Decoding

Reliable crack measurement is essential for infrastructure condition assessment, yet existing image-based approaches typically depend on pixel-wise annotations, task-specific segmentation training, and mask-based geometric measurement, making cross-scene deployment costly and sensitive to segmentation errors. We identify an output-interface mismatch in SAM3: its prompt-conditioned semantic response preserves crack evidence that is often suppressed or spatially distorted in the final candidate masks. Across six public crack datasets, the internal response achieves 82.66% average crack-pixel recall, compared with 74.66% for the retained SAM3 proposals, with an average mismatch ratio of 8.73%. Based on this observation, we propose Semantic-Edge Response Decoding (SERD) to calibrate the semantic response using a fixed Sobel structural field, and further develop SERD-DQ, a training-free framework that directly estimates crack centerline and transverse geometry from the continuous decoded response without generating an intermediate predicted mask. Experiments verify both segmentation fidelity and direct geometric measurement against manually established pixel-level references. Compared with native SAM3 mask-based quantification, SERD-DQ reduces width MAE from 6.072 to 5.547 pixels, length relative error from 22.245% to 17.355%, and area relative error from 33.921% to 26.228%, while achieving a latent geometry recovery rate of 0.394. The results indicate that continuous semantic-edge responses provide a more reliable interface for training-free crack quantification than conventional mask-mediated measurement.

cs.CV↗
arXiv2026

CoRe-SAM3: Conditional Semantic--Visual Reconciliation for SAM3 Crack Segmentation

Crack segmentation requires a model to recognize target semantics while accurately recovering thin, low-contrast, and topologically continuous local structures. Although SAM3 provides strong open-concept segmentation, its direct application to the crack domain still misses weak cracks, activates crack-like background regions, and produces local boundary errors. We first diagnose the functional differences between the internal prompt-conditioned semantic representation and native visual representation of SAM3 on five crack datasets. The results show that the semantic representation already carries most task information for crack prediction, whereas the utility of the visual representation depends on the current semantic state. Directly combining the two representations does not yield consistent gains. Based on this finding, we propose Conditional Semantic--Visual Reconciliation, termed CoRe. CoRe retains semantic prediction as the primary decision path, applies lightweight semantic calibration to adjust the target-domain decision mapping, and uses spatially aligned native visual evidence to generate a zero-initialized, bounded, and regularized conditional residual that selectively corrects existing predictions. Across five domains, CoRe-SAM3 improves the average Crack IoU from 62.34% to 70.47% and clDice from 81.98% to 89.24%, while introducing only 18.914 K trainable parameters. Prediction-transition analysis further shows that CoRe corrects an average of 34.38% of native errors, with a damage rate of only 0.23% on pixels correctly classified by native SAM3. These results demonstrate that constrained prediction correction based on the functional differences between internal representations provides an effective and parameter-efficient target-domain adaptation strategy for vision foundation models with strong task-specific semantic priors.

cs.CV↗
arXiv2026

Describe-to-Score: A text-guided framework for image complexity assessment

Accurately assessing image complexity (IC) is essential for many vision tasks, yet existing approaches rely almost exclusively on visual features and therefore fail to capture the high-level semantics that humans often use when judging complexity. We introduce a multimodal perspective for IC modeling by integrating visual representations with caption-derived textual semantics. This integration enriches the representational space and provides complementary structural cues that are difficult to infer from vision alone. From an information theoretic and representation viewpoint, we offer an idealized analysis suggesting how semantic guidance can regularize the hypothesis space and support more stable generalization. We propose D2S (Describe-to-Score), a text-guided framework that uses caption-derived semantics only during training to regularize visual complexity modeling, while preserving a vision-only inference pipeline with no additional multimodal overhead at inference. Concretely, D2S transfers semantic structure into the visual branch through feature alignment and entropy distribution alignment, encouraging the visual encoder to internalize complexity-relevant semantic regularities. Experiments show that D2S achieves state-of-the-art performance on the IC9600 benchmark and remains competitive on no-reference image quality assessment (NR-IQA) tasks. Additional analyses further clarify the sample-imbalance issue in the small samples training setting and the distribution-shift limitations observed in cross-dataset transfer. Code is available at: https://github.com/xauat-liushipeng/D2S.

cs.CV↗
arXiv2026

Contrastive Learning for Image Complexity Representation

Quantifying and evaluating image complexity can be instrumental in enhancing the performance of various computer vision tasks. Supervised learning can effectively learn image complexity features from well-annotated datasets. However, creating such datasets requires expensive manual annotation costs. The models may learn human subjective biases from it. In this work, we introduce the MoCo v2 framework. We utilize contrastive learning to represent image complexity, named CLIC (Contrastive Learning for Image Complexity). We find that there are complexity differences between different local regions of an image, and propose Random Crop and Mix (RCM), which can produce positive samples consisting of multi-scale local crops. RCM can also expand the train set and increase data diversity without introducing additional data. We conduct extensive experiments with CLIC, comparing it with both unsupervised and supervised methods. The results demonstrate that the performance of CLIC is comparable to that of state-of-the-art supervised methods. In addition, we establish the pipelines that can apply CLIC to computer vision tasks to effectively improve their performance.

cs.CV↗
arXiv2026

Rethinking Efficient Crack Segmentation with Task-Aligned Structural-Directional Modeling

Recent crack segmentation methods often follow generic semantic segmentation designs, using stronger backbones, hybrid CNN-Transformer-Mamba encoders, and auxiliary enhancement branches. Although effective, this raises whether stronger generic feature mixing is the most suitable direction for crack segmentation. We instead formulate crack segmentation as sparse structural recovery. Cracks have limited category-level semantics but strong morphological regularities, being thin, sparse, anisotropic, locally fragmented, and easily confused with textures or shadows. Thus, the key bottleneck lies in preserving weak structural evidence, recovering directional continuity, and suppressing background coupling. We propose RIFT, a compact family of morphology-aligned crack segmentation models. Rather than compressing a complex generic architecture, RIFT is simple by design, preserving local evidence, aggregating cooperative directional continuity, and restoring crack structures through lightweight multi-scale fusion. Experiments on four public benchmarks show that RIFT achieves the best or tied-best results across the 16 main metrics against reproduced representative baselines. RIFT-B gives the strongest overall accuracy, while RIFT-T provides the best deployment efficiency with only 0.47M parameters and high inference speed. Topology-aware evaluation, ablations, transfer experiments, and visualizations further verify that task-aligned simplicity can match or surpass complex hybrid architectures when its inductive bias fits crack morphology. Code: https://github.com/xauat-liushipeng/RIFT

cs.CV↗
arXiv2026

Training-Free Tunnel Defect Inspection and Engineering Interpretation via Visual Recalibration and Entity Reconstruction

Tunnel inspection requires outputs that can support defect localization, measurement, severity grading, and engineering documentation. Existing training-free foundation-model pipelines usually stop at coarse open-vocabulary proposals, which are difficult to use directly in interference-heavy tunnel scenes. We propose a training-free framework TunnelMIND. Specifically, language-guided defect proposals are not treated as final outputs; instead, their spatial support is recalibrated at inference time through dense visual consistency, so that coarse semantic anchors can be transformed into more reliable prompts under tunnel-specific hard negatives. The resulting masks are further reconstructed into structured defect entities with category, location, geometry, severity, and context attributes, which are then mapped to retrieval-grounded explanation and engineering-readable report generation under expert knowledge constraints. On visible, GPR, and road defect tasks, TunnelMIND achieves F1 scores of 0.68, 0.78, and 0.72, respectively. Overall, TunnelMIND shows that training-free tunnel inspection can move beyond coarse localization toward structured defect evidence for engineering assessment.

cs.CV↗
arXiv2026

Legged Autonomous Surface Science In Analogue Environments (LASSIE): Making Every Robotic Step Count in Planetary Exploration

The ability to efficiently and effectively explore planetary surfaces is currently limited by the capability of wheeled rovers to traverse challenging terrains, and by pre-programmed data acquisition plans with limited in-situ flexibility. In this paper, we present two novel approaches to address these limitations: (i) high-mobility legged robots that use direct surface interactions to collect rich information about the terrain's mechanics to guide exploration; (ii) human-inspired data acquisition algorithms that enable robots to reason about scientific hypotheses and adapt exploration priorities based on incoming ground-sensing measurements. We successfully verify our approach through lab work and field deployments in two planetary analog environments. The new capability for legged robots to measure soil mechanical properties is shown to enable effective traversal of challenging terrains. When coupled with other geologic properties (e.g., composition, thermal properties, and grain size data etc), soil mechanical measurements reveal key factors governing the formation and development of geologic environments. We then demonstrate how human-inspired algorithms turn terrain-sensing robots into teammates, by supporting more flexible and adaptive data collection decisions with human scientists. Our approach therefore enables exploration of a wider range of planetary environments and new substrate investigation opportunities through integrated human-robot systems that support maximum scientific return.

cs.RO↗
arXiv2026

Proprioceptive Safe Active Navigation and Exploration for Planetary Environments

Deformable granular terrains introduce significant locomotion and immobilization risks in planetary exploration and are difficult to detect via remote sensing (e.g., vision). Legged robots can sense terrain properties through leg-terrain interactions during locomotion, offering a direct means to assess traversability in deformable environments. How to systematically exploit this interaction-derived information for navigation planning, however, remains underexplored. We address this gap by presenting PSANE, a Proprioceptive Safe Active Navigation and Exploration framework that leverages leg-terrain interaction measurements for safe navigation and exploration in unknown deformable environments. PSANE learns a traversability model via Gaussian Process regression to estimate and certify safe regions and identify exploration frontiers online, and integrates these estimates with a reactive controller for real-time navigation. Frontier selection is formulated as a multi-objective optimization that balances safe-set expansion probability and goal-directed cost, with subgoals selected via scalarization over the Pareto-optimal frontier set. PSANE safely explores unknown granular terrain and reaches specified goals using only proprioceptively estimated traversability, while achieving performance improvements over baseline methods.

cs.RO↗
arXiv2026

Inverse Resistive Force Theory (I-RFT): Learning granular properties through robot-terrain physical interactions

For robots to navigate safely and efficiently on soft, granular terrains, it is crucial to gather information about the terrain's mechanical properties, which directly affect locomotion performance. Recent research has developed robotic legs that can accurately sense ground reaction forces during locomotion. However, existing tests of granular property estimation often rely on specific foot trajectories, such as vertical penetration or horizontal shear, limiting their applicability during natural locomotion. To address this limitation, we introduce a physics-informed machine learning framework, Inverse Resistive Force Theory (I-RFT), which integrates the Granular Resistive Force Theory model with Gaussian Processes to infer terrain properties from proprioceptively measured contact forces under arbitrary gait trajectories. By embedding the granular force model within the learning process, I-RFT preserves physical consistency while enabling generalization across diverse motion primitives. Experimental results demonstrate that I-RFT accurately estimates terrain properties across multiple gait trajectories and toe shapes. Moreover, we show that the quantified uncertainty over the terrain resistance stress map could enable robots to optimize foot design and gait trajectories for efficient information gathering. This approach establishes a new foundation for data-efficient characterization of complex granular environments and opens new avenues for locomotion strategies that actively adapt gait for autonomous terrain exploration.

cs.RO↗
arXiv2026

Bio-inspired tail oscillation enables robot fast crawling on deformable granular terrains

Deformable substrates such as sand and mud present significant challenges for terrestrial robots due to complex robot-terrain interactions. Inspired by mudskippers, amphibious animals that naturally adjust their tail morphology and movement jointly to navigate such environments, we investigate how tail design and control can jointly enhance flipper-driven locomotion on granular media. Using a bio-inspired robot modeled after the mudskipper, we experimentally compared locomotion performance between idle and actively oscillating tail configurations. Tail oscillation increased robot speed by 67% and reduced body drag by 46%. Shear force measurements revealed that this improvement was enabled by tail oscillation fluidizing the substrate, thereby reducing resistance. Additionally, tail morphology strongly influenced the oscillation strategy: designs with larger horizontal surface areas leveraged the oscillation-reduced shear resistance more effectively by limiting insertion depth. Based on these findings, we present a design principle to inform tail action selection based on substrate strength and tail morphology. Our results offer new insights into tail design and control for improving robot locomotion on deformable substrates, with implications for agricultural robotics, search and rescue, and environmental exploration.

cs.RO↗
arXiv2026

Scout-Rover cooperation: online terrain strength mapping and traversal risk estimation for planetary-analog explorations

Robot-aided exploration of planetary surfaces is essential for understanding geologic processes, yet many scientifically valuable regions, such as Martian dunes and lunar craters, remain hazardous due to loose, deformable regolith. We present a scout-rover cooperation framework that expands safe access to such terrain using a hybrid team of legged and wheeled robots. In our approach, a high-mobility legged robot serves as a mobile scout, using proprioceptive leg-terrain interactions to estimate regolith strength during locomotion and construct spatially resolved terrain maps. These maps are integrated with rover locomotion models to estimate traversal risk and inform path planning. We validate the framework through analogue missions at the NASA Ames Lunar Simulant Testbed and the White Sands Dune Field. Experiments demonstrate (1) online terrain strength mapping from legged locomotion and (2) rover-specific traversal-risk estimation enabling safe navigation to scientific targets. Results show that scout-generated terrain maps reliably capture spatial variability and predict mobility failure modes, allowing risk-aware path planning that avoids hazardous regions. By combining embodied terrain sensing with heterogeneous rover cooperation, this framework enhances operational robustness and expands the reachable science workspace in deformable planetary environments.

cs.RO↗
arXiv2026

Human-in-the-Loop Multi-Robot Information Gathering with Inverse Submodular Maximization

We consider a new type of inverse combinatorial optimization, Inverse Submodular Maximization (ISM), for its application in human-in-the-loop multi-robot information gathering. Forward combinatorial optimization - solving a combinatorial problem given the reward (cost)-related parameters - is widely used in multi-robot coordination. In the standard pipeline, domain experts design the reward (cost)-related parameters offline. These parameters are utilized for coordinating robots online. What if non-expert human supervisors desire to change these parameters during task execution to adapt to some new requirements? We are interested in the case where human supervisors can suggest what path primitives to take, and the robots need to change the internal decision-making parameters accordingly. We study such problems from the perspective of inverse combinatorial optimization, i.e., the process of finding parameters that give certain solutions to the problem. Specifically, we propose a new formulation for ISM for a family of multi-robot information gathering scenarios, in which we aim to find a new set of parameters that minimally deviates from the current parameters while causing a greedy algorithm to output path primitives that are the same as those desired by the human supervisors. We show that for the case with a single suggestion, such problems can be formulated as a Mixed Integer Quadratic Program (MIQP), which is intractable for existing solvers when the problem size is large. We propose a new Branch $\&$ Bound algorithm to solve such problems. For the case with multiple suggestions from several human supervisors, the problem can be cast as a multi-objective optimization and can be solved using Pareto Monte Carlo Tree Search. In numerical simulations, we demonstrate how to use ISM in multi-robot scientific data collection and event detection-driven coverage control.

cs.RO↗
arXiv2026

Collision Risk Estimation via Loss Prediction in End-to-End Autonomous Driving

Collision risk estimation and avoidance play central roles in the safety of autonomous driving (AD) systems. Recently emerged end-to-end AD systems gain collision avoidance ability by minimizing losses to penalize planning trajectories that are too close to other objects. Despite a significant collision rate during testing, most end-to-end planners do not explicitly quantify the collision risk in their outputs. To address this, we introduce RiskMonitor, an efficient plug-and-play module that interprets planning and motion tokens from state-of-the-art end-to-end planners to estimate collision risk. Inspired by loss prediction based uncertainty quantification, RiskMonitor predicts whether the collision loss -- commonly adopted to train end-to-end planners -- is positive along planned waypoints, framing collision risk estimation as a binary classification task. We evaluate RiskMonitor on the real-world nuScenes dataset (open-loop) and the neural-rendering based simulator, NeuroNCAP (closed-loop). Our token-driven method outperforms prediction-driven approaches, including deterministic rules, Gaussian mixture models, and Monte Carlo Dropout. When integrated with a simple braking policy, RiskMonitor improves collision avoidance ability by $66.5\%$ in a closed-loop test on safety-critical scenarios. These results demonstrate that monitoring collision risk using plan and motion tokens enhances the safety of end-to-end AD without retraining it.

cs.RO↗
arXiv2026

MATTER: Multiscale Attention for Registration Error Regression

Point cloud registration (PCR) is crucial for many downstream tasks, such as simultaneous localization and mapping (SLAM) and object tracking. This makes detecting and quantifying registration misalignment, i.e., PCR quality validation, an important task. All existing methods treat validation as a classification task, aiming to assign the PCR quality to a few classes. In this work, we instead use regression for PCR validation, allowing for a more fine-grained quantification of the registration quality. We also extend previously used misalignment-related features by using multiscale extraction and attention-based aggregation. This leads to accurate and robust registration error estimation on diverse datasets, especially for point clouds with heterogeneous spatial densities. Furthermore, when used to guide a mapping downstream task, our method significantly improves the mapping quality for a given amount of re-registered frames, compared to the state-of-the-art classification-based method.

cs.CV↗
arXiv2026

Towards Real-time Adaptation of Embodied Agent in Human-Robot Collaboration

Large Language Models (LLMs) have opened transformative possibilities for human-robot collaboration. However, enabling real-time collaboration requires both low latency and robust reasoning, and most LLMs suffer from high latency. To address this gap, we first propose a fine-grained benchmark that explicitly assesses agents' proactive adaptability and temporal responsiveness in the Overcooked-AI environment. Based on evaluation results, we propose MonTA (Monitor-then-Adapt), a hierarchical framework inspired by cognitive science research. MonTA contains three key modules: a lightweight Monitor that operates at high frequency (7 Hz) to detect adaptation needs, and two proficient Adapters for subtask and path adaptation reasoning that provide instructions to humans at a lower frequency. Our results demonstrate that MonTA significantly outperforms baseline agents on our proposed benchmark, achieving superior performance across layouts with varying teaming fluency. User studies confirm the high reasonableness of adaptation plans and consistent language instructions provided by our framework to humans.

cs.AI↗
arXiv2025

Safe Active Navigation and Exploration for Planetary Environments Using Proprioceptive Measurements

Legged robots can sense terrain through force interactions during locomotion, offering more reliable traversability estimates than remote sensing and serving as scouts for guiding wheeled rovers in challenging environments. However, even legged scouts face challenges when traversing highly deformable or unstable terrain. We present Safe Active Exploration for Granular Terrain (SAEGT), a navigation framework that enables legged robots to safely explore unknown granular environments using proprioceptive sensing, particularly where visual input fails to capture terrain deformability. SAEGT estimates the safe region and frontier region online from leg-terrain interactions using Gaussian Process regression for traversability assessment, with a reactive controller for real-time safe exploration and navigation. SAEGT demonstrated its ability to safely explore and navigate toward a specified goal using only proprioceptively estimated traversability in simulation.

cs.RO↗
arXiv2025

Adaptive Locomotion on Mud through Proprioceptive Sensing of Substrate Properties

Muddy terrains present significant challenges for terrestrial robots, as subtle changes in composition and water content can lead to large variations in substrate strength and force responses, causing the robot to slip or get stuck. This paper presents a method to estimate mud properties using proprioceptive sensing, enabling a flipper-driven robot to adapt its locomotion through muddy substrates of varying strength. First, we characterize mud reaction forces through actuator current and position signals from a statically mounted robotic flipper. We use the measured force to determine key coefficients that characterize intrinsic mud properties. The proprioceptively estimated coefficients match closely with measurements from a lab-grade load cell, validating the effectiveness of the proposed method. Next, we extend the method to a locomoting robot to estimate mud properties online as it crawls across different mud mixtures. Experimental data reveal that mud reaction forces depend sensitively on robot motion, requiring joint analysis of robot movement with proprioceptive force to determine mud properties correctly. Lastly, we deploy this method in a flipper-driven robot moving across muddy substrates of varying strengths, and demonstrate that the proposed method allows the robot to use the estimated mud properties to adapt its locomotion strategy, and successfully avoid locomotion failures. Our findings highlight the potential of proprioception-based terrain sensing to enhance robot mobility in complex, deformable natural environments, paving the way for more robust field exploration capabilities.

cs.RO↗
arXiv2025

Continuous Normalizing Flows for Uncertainty-Aware Human Pose Estimation

Human Pose Estimation (HPE) is increasingly important for applications like virtual reality and motion analysis, yet current methods struggle with balancing accuracy, computational efficiency, and reliable uncertainty quantification (UQ). Traditional regression-based methods assume fixed distributions, which might lead to poor UQ. Heatmap-based methods effectively model the output distribution using likelihood heatmaps, however, they demand significant resources. To address this, we propose Continuous Flow Residual Estimation (CFRE), an integration of Continuous Normalizing Flows (CNFs) into regression-based models, which allows for dynamic distribution adaptation. Through extensive experiments, we show that CFRE leads to better accuracy and uncertainty quantification with retained computational efficiency on both 2D and 3D human pose estimation tasks.

cs.CV↗