Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.39745

UniffordInteractor: Reasoning the Unified Affordance for Human and Multi-Object Interaction

Chen Su·Xingqun Qi·Xurui Hu·Bingkun Yang·Muyi Sun·Man Zhang·Zhenan Sun
View original source ↗Read source document ↗Export metadata ↓

Abstract

Deriving plausible and dynamic Human Object Interaction (HOI) has gained tremendous progress in Embodied AI and Immersive AR/VR. While existing HOI methods have effectively explored human interaction with individual or fixed objects, they still show limitations in complex multi-object interaction scenes. Meanwhile, previous approaches focus more on the contact information held by human-object relation, overlooking the significance of the contact situation of various entities (i.e., object-object). To address these limitations, we propose UniffordInteractor, a Reasoning-then-Interacting framework that leverages unified affordance for realistic and long-horizon human and multi-object interaction modeling. At the reasoning stage, we devise a Unified Affordance Reasoner (UAR) that employs a fine-tuned 3D Multimodal Large Language Model for unified object-centric affordance prediction. At the interacting stage, we propose Hierarchical Interaction Generator (HIG), which formulates the interaction generation as a hierarchical (e.g., root to body to hand to objects) autoregressive prediction, conditioned on the obtained unified affordance. Extensive experiments demonstrate that UniffordInteractor surpasses SOTA counterparts in human and multi-object interaction generation.

Explore related subjects

cs.GR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Chen Su, Xingqun Qi, Xurui Hu, Bingkun Yang, Muyi Sun, Man Zhang, Zhenan Sun. 2026-10-29. UniffordInteractor: Reasoning the Unified Affordance for Human and Multi-Object Interaction. https://doi.org/10.1145/3842530

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Trajeform: Embedding Crowd Data

Crowd simulation aims to generate agent behaviours that are rich, natural, and appropriate to their surrounding environment. Recent advances in Machine Learning have enabled such behavioural patterns to be learned directly from trajectory data, allowing simulation models to capture increasingly complex crowd dynamics. However, the capabilities of these systems remain strongly dependent on the type and diversity of the data used during training. When evaluating how well crowd simulators generalise to real-world datasets, most existing metrics focus primarily on trajectory-level differences rather than behaviour-level similarity. In this paper, we present Trajeform, a framework that trains a reconstruction VAE (Variational Autoencoder) on real trajectory data to learn a meaningful continuous embedding that preserves behavioural characteristics of real crowds. The trained encoder can then be applied to synthetic trajectories generated by simulators and used to compare the resulting embedding distributions. Additionally, the learned embedding can be used to analyse supplementary data, such as trajectories corresponding to a simulator's principal behaviours or captured data from controlled environments (e.g., cycling), providing insight into their ability to span the real behavioural embedding space. The framework compares data from three simulators (CCP, modCCP, and RVO) against ground-truth datasets and is evaluated quantitatively and qualitatively using latent-space analysis, distributional metrics, and trajectory generation through sampling from the aggregated posterior distribution. The results indicate that the learned latent space captures meaningful trajectory dependencies found in the real data, making the framework useful both as an additional measure of synthetic data believability and as a tool for informing the training of future trajectory models.

cs.GR↗
arXiv2026

Mastering Multi-State Dynamics: Continuum Damage and Viscoplastic MPM for RL-Based Dexterous Manipulation

Robotic manipulation of multi-state materials needs a simulator that handles large deformation, topology change, and two-way contact between tools and materials while staying fast enough to drive online reinforcement learning. We present MG-CD-MPM, a tool-centered localized extension of CD-MPM. The implicit Ginzburg-Landau phase-field damage system is solved on a local patch by preconditioned conjugate gradients using a geometric-multigrid V-cycle as the preconditioner. The background MPM discretization remains uniform: only the damage solve is localized. Two-way rigid-tool coupling converts MPM contact impulses into forces and torques on the tool, exposing material resistance directly to the policy. The resulting Gym-like simulator covers soft solids, brittle materials, granular media, and viscoplastic or semi-fluid media within a unified per-particle constitutive layer. We demonstrate cutting, scooping, and shoveling in six scenes, combining scripted tool motions with PPO policies trained on two cutting tasks.

cs.GR↗
arXiv2026

Greenscapes: Optimized Vegetation Configurations in Cities

Vegetation plays a central role in cities, serving functional roles such as balancing shade and sky exposure, shaping the microclimate, carbon sequestration, and providing acoustic screening. Furthermore, aesthetic choices in landscape styles (formal, modernist, picturesque) are important for citizens' well-being and should be considered in the design of urban landscapes. In this paper, we present a novel method for placing vegetation, ranging from shrubs to large trees, in 3D urban scenes. Our method uses individual lots as input and employs a conditioned Vision-Language Model (VLM) to generate a top-down rendering of vegetation, placing large trees, small trees, and shrubs on lawn areas. This rendering is then converted into a 3D scene by detecting individual plants and selecting appropriate 3D assets. To refine the proposed configurations, we define an optimization loop that keeps the VLM-generated positions fixed after a deterministic feasibility repair and uses CMA-ES to optimize per-instance scale transformations along with discrete vegetation asset substitutions. To define the objective function for the optimization, we evaluate the generated 3D scenes with efficient surrogate models of shade and path-based acoustic exposure. Furthermore, we render in-scene views at each optimization step to compute view-dependent perceptual descriptors, enabling the generation of both aesthetic and functional vegetation layouts for urban scenes.

cs.GR↗