Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2302.04708

A Perception-Aware NMPC for Vision-Based Target Tracking and Collision Avoidance with a Multi-Rotor UAV

Andriy Dmytruk·Giuseppe Silano·Davide Bicego·Daniel Bonilla Licea·Martin Saska
View original source ↗Read source document ↗Export metadata ↓

Abstract

A perception-aware Nonlinear Model Predictive Control (NMPC) strategy aimed at performing vision-based target tracking and collision avoidance with a multi-rotor aerial vehicle is presented in this paper. The proposed control strategy considers both realistic actuation limits at the torque level and visual perception constraints to enforce the visibility coverage of a target while complying with the mission objectives. Furthermore, the approach allows to safely navigate in a workspace area populated by dynamic obstacles with a ballistic motion. The formulation is meant to be generic and set upon a large class of multi-rotor vehicles that covers both coplanar designs like quadrotors as well as fully-actuated platforms with tilted propellers. The feasibility and effectiveness of the control strategy are demonstrated via closed-loop simulations achieved in MATLAB.

Explore related subjects

cs.RO

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Andriy Dmytruk, Giuseppe Silano, Davide Bicego, Daniel Bonilla Licea, Martin Saska. 2023-02-09. A Perception-Aware NMPC for Vision-Based Target Tracking and Collision Avoidance with a Multi-Rotor UAV. https://doi.org/10.1109/icuas54217.2022.9836071

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Control

Humans intuitively move to sound, but current humanoid robots lack expressive improvisational capabilities, confined to predefined motions or sparse commands. Generating motion from audio and then retargeting it to robots relies on explicit motion reconstruction, leading to cascaded errors, high latency, and disjointed acoustic-actuation mapping. We propose RoboPerform, the first unified audio-to-locomotion framework that can directly generate music-driven dance and speech-driven co-speech gestures from audio. Guided by the core principle of "motion = content + style", the framework treats audio as implicit style signals and eliminates the need for explicit motion reconstruction. RoboPerform integrates a ResMoE teacher policy for adapting to diverse motion patterns and a diffusion-based student policy for audio style injection. This retargeting-free design ensures low latency and high fidelity. Experimental validation shows that RoboPerform achieves promising results in physical plausibility and audio alignment, successfully transforming robots into responsive performers capable of reacting to audio.

cs.RO↗
arXiv2026

Reversible Simplex Supervision with Post-Action Debt Accounting for Goal-Reaching RL

Deploying reinforcement learning (RL) on multi-tonne robots calls for supervisory mechanisms that address both operational safety and progress toward task completion. However, repeated switching need not preserve task progress when a learned action increases storage before recovery takes control. We introduce reversible Simplex supervision with post-action debt accounting for a frozen finite-state policy and robust-adaptive recovery. Under exact sampled-state information and stated model and certificate conditions, we prove that recovery repayment exceeding a uniform triggering-edge debt bound guarantees finite switching and finite-sample goal entry. We formulate reachability-based certificate constructions for establishing these sufficient conditions. In 20 matched simulations, goal-entry counts are 20 with debt gating and 18 without it; the two remaining runs terminate under the supervisor's admissibility stopping rule. On an experimental 6000 kg robot, 24 asphalt and soft-terrain trials evaluate 50 ms supervision above a 1 kHz actuator stack; all eight triggered recoveries complete debt-gated re-entry. The experiments demonstrate the supervisory mechanism in the tested trials.

cs.RO↗
arXiv2026

Foundation and Small Models Coordination for Visuomotor Policy Learning

Visuomotor policy learning enables robots to perform a wide range of tasks, but small policy models often remain sensitive to changes in object and background appearance. In this work, we investigate the coordination of pretrained vision foundation models with small policy models to improve appearance generalization. We propose a framework in which a small policy model operates on task-relevant visual observations constructed through semantic repainting. A segmentation foundation model identifies the robot and target object, which are rendered with fixed role colors on a constant background. An alternative representation replaces the target's role color with normalized monocular depth predicted by a depth foundation model, providing additional geometric cues. The perception models are adapted using in-distribution data where needed and held fixed during policy training. This design combines the perceptual capabilities of foundation models with a small policy model trained on the resulting observations for action prediction. Evaluations with flow matching policies on simulation benchmarks, together with experiments on two real-world robotic tasks, demonstrate substantial improvements in task success under the evaluated appearance shifts.

cs.RO↗