Comparative Evaluation of an XR Pen-based Control Interface for Semi-Autonomous Mobile Robot Navigation in Service Environments

Alicia Torc; Carl Tornberg; Eric Piette; Renaud Ronsse; Benoit Macq; Gustavo Alfonso Garcia Ricardez; Lotfi El Hafi; Tadahiro Taniguchi

arXiv · 2609.31117

Comparative Evaluation of an XR Pen-based Control Interface for Semi-Autonomous Mobile Robot Navigation in Service Environments

Alicia Torc·Carl Tornberg·Eric Piette·Renaud Ronsse·Benoit Macq·Gustavo Alfonso Garcia Ricardez·Lotfi El Hafi·Tadahiro Taniguchi

View original source ↗Read source document ↗Export metadata ↓

Abstract

Service robots remain difficult to deploy in domestic environments, partly because fully autonomous operation is not yet reliable in unpredictable surroundings, and partly because conventional control methods remain inaccessible to novice users. Extended Reality (XR) enables operators to visualize robot information overlaid onto the real world and to interact with augmented elements. Yet, common XR control methods, such as motion controllers and hand gestures, are still perceived as unintuitive. This paper presents a control interface that uses a commercial XR pen to command a semi-autonomous mobile robot in Augmented Reality (AR): the operator points at a position in the room, selects it, and drags an augmented arrow to set the desired orientation of the robot at this destination. Two additional interfaces, based on the XR motion controllers and hand gestures, were developed within the same framework. To assess the performance and users' perception of these interfaces, and of the XR pen in particular, a study with 10 participants compared four control methods, i.e., the XR pen, the XR motion controllers, hand gestures, and a computer-based baseline RViz, in navigation tasks performed in a home-like environment. Results show that the XR pen significantly outperforms the other methods in task selection time with the most consistent selections, and that the XR motion controllers obtain the best perceived workload and usability scores, ahead of the computer-based baseline, supporting XR-based control as an intuitive alternative for novice users. However, technical limitations in the integration of the recently released XR pen currently hold back its user experience.

Explore related subjects

cs.RO cs.HC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Alicia Torc, Carl Tornberg, Eric Piette, Renaud Ronsse, Benoit Macq, Gustavo Alfonso Garcia Ricardez, Lotfi El Hafi, Tadahiro Taniguchi. 2026-09-25. Comparative Evaluation of an XR Pen-based Control Interface for Semi-Autonomous Mobile Robot Navigation in Service Environments. https://arxiv.org/abs/2609.31117

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

OEL-AD: An Online Evolutive Learning Framework for Cross-Region Adaptive Autonomous Driving

Autonomous driving models have achieved great progress in recent years, yet most existing approaches remain static, following a train-then-deploy paradigm that lacks the ability to adjust internal parameters when encountering new environments. This limitation is particularly critical in cross-region scenarios, which are common in real-world deployments in autonomous driving where differences in road layouts, traffic patterns, and environmental conditions introduce distribution shifts. In this paper, we argue for a paradigm shift towards Online Evolutive Learning (OEL), a learning paradigm designed to enable adaptive, online model optimization through the interaction and coordination of intelligent components in dynamic environments. We introduce OEL-AD, a novel framework that instantiates this paradigm for autonomous driving. Specifically, OEL-AD leverages trajectory uncertainty from the planning module to determine whether adaptive updates are necessary. When triggered, the framework uses discrepancies between predicted agent behaviors and subsequent observations as a self-supervised learning signal to refine perception and prediction modules, thereby indirectly improving planning accuracy and safety. Experiments on the nuScenes dataset with a cross-region split between Singapore and Boston demonstrate that OEL-AD consistently improves perception and prediction performance. This enhancement in perception and prediction consequently results in improved planning capabilities, thereby confirming the promise of online evolutive learning as a fundamental approach for building adaptive and robust autonomous driving systems.

cs.RO↗

arXiv2026

CENIC: Convex Error-controlled Numerical Integration for Contact

State-of-the-art robotics simulators operate in discrete time. This requires users to choose a time step, which is both critical and challenging: large steps can produce non-physical artifacts, while small steps force the simulation to run slowly. Continuous-time error-controlled integration avoids such issues by automatically adjusting the time step to achieve a desired accuracy. But existing error-controlled integrators struggle with the stiff dynamics of contact, and cannot meet the speed and scalability requirements of modern robotics workflows. We introduce CENIC, a new variable-step integrator that brings together recent advances in convex time-stepping and error-controlled integration, inheriting benefits from both continuous integration and discrete time-stepping. CENIC runs at fast real-time rates comparable to discrete-time robotics simulators like MuJoCo, Drake and Isaac Sim, while also providing guarantees on accuracy and convergence.

cs.RO↗

arXiv2026

Reflection-Based Relative Localization for Cooperative UAV Teams Using Active Markers

Reflections of active markers in the environment are a common source of ambiguity in onboard visual relative localization. This work presents a novel approach that exploits these typically unwanted reflections for onboard relative localization in heterogeneous multi-UAV teams operating over specular, near-planar surfaces such as calm water and flat indoor floors. Using only a camera, active markers, and a single-beam rangefinder, the method requires no prior knowledge of robot size, marker configuration, or surface reflectance properties, and propagates the uncertainty introduced by surface irregularities, including those of dynamic water surfaces. We further characterize, analytically and experimentally, how surface tilt biases the estimate, and identify the slope range over which the level-surface assumption holds. Indoor multi-UAV and outdoor experiments validate the localization accuracy, demonstrating reliable operation across varying lighting conditions and greater effective range than state-of-the-art vision-based active-marker methods, with higher accuracy beyond 30 m. The project website with the video is available under the following link: https://timlakemann.github.io/reflection-based-relative-localization/.

cs.RO↗