Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update

Sitian Shen; Jiuming Liu; Mengmeng Liu; Yian Wang; Michael Ying Yang; Francesco Nex; Hao Cheng; Daniele De Martini; Ayush Tewari; Per Ola Kristensson

arXiv · 2610.07031

Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update

Sitian Shen·Jiuming Liu·Mengmeng Liu·Yian Wang·Michael Ying Yang·Francesco Nex·Hao Cheng·Daniele De Martini·Ayush Tewari·Per Ola Kristensson

View original source ↗Read source document ↗Export metadata ↓

Abstract

Recent video world models have witnessed the paradigm shift from single-agent to multi-agent involvements, which can reveal more complicated dynamics and cross-agent interaction in the real world. However, existing approaches commonly adopt implicit inter-agent communications via cross attention, which lack explicit geometry constraints and unified 3D state, thereby leading to poor multi-view consistency and struggling with recovering out-of-sight agents. In addition, most of them assume a static background, failing to represent uncontrolled background dynamics. To address these problems, we propose Artemis: a geometry-grounded multi-agent world model with explicit memory sharing. An explicit 3D world map is reconstructed from multi-agent observations to enforce a unified 3D state across agents, offering high cross-view consistency. Specifically, an action-guided geometric injection module is developed to simultaneously render decomposed foreground-background control maps, which are then injected into a diffusion transformer through a designed GeoAdapter block. Compared to previous methods assuming static-only background, our GeoAdapter can also distinguish uncontrolled non-agent dynamics, which are conditioned on their own multi-frame history positions to provide consistent motion cues. Keyframes selected from progressive video rollouts are used to progressively update the reconstructed 3D world maps. To effectively capture complex dynamic patterns, we curate a novel dataset sampled from the CARLA simulator called MA-CARLA. Extensive experiments demonstrate the superiority of our proposed method in terms of visual fidelity and cross-view consistency in the generated videos. In addition, our Artemis can support simultaneous multi-modal rollouts with both 2D video and 3D point map maintenance, scale to scenarios beyond two agents and multi-camera setting.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Sitian Shen, Jiuming Liu, Mengmeng Liu, Yian Wang, Michael Ying Yang, Francesco Nex, Hao Cheng, Daniele De Martini, Ayush Tewari, Per Ola Kristensson. 2026-10-04. Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update. https://arxiv.org/abs/2610.07031

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

DetoxAI: a Python Toolkit for Debiasing Deep Learning Models in Computer Vision

While machine learning fairness has made significant progress in recent years, most existing solutions focus on tabular data and are poorly suited for vision-based classification tasks, which rely heavily on deep learning. To bridge this gap, we introduce DetoxAI, an open-source Python library for improving fairness in deep learning vision classifiers through post-hoc debiasing. DetoxAI implements state-of-the-art debiasing algorithms, fairness metrics, and visualization tools. It supports debiasing via interventions in internal representations and includes attribution-based visualization tools and quantitative algorithmic fairness metrics to show how bias is mitigated. This paper presents the motivation, design, and use cases of DetoxAI, demonstrating its tangible value to engineers and researchers.

cs.CV↗

arXiv2026

AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines

The contemporary paradigm of scaling data annotation, crucial for developing machine learning solutions, is to hire ordinary human annotators and instruct them with expert-crafted guidelines to label data. This paradigm is laborious, tedious, and costly, motivating us to study an open problem, auto-annotation with expert-crafted guidelines (dubbed AutoExpert). We develop benchmarks by redesigning the evaluation protocol and re-annotating data with nuScenes and PandaSet, two 3D detection datasets for autonomous driving research that provide expert-crafted annotation guidelines. Their guidelines define 18 and 25 object classes, respectively, using nuanced language descriptions and a few visual examples. Following the guidelines that require using 3D cuboids to label LiDAR data, AutoExpert requires algorithms to learn on few-shot labeled images and texts to perform the task of 3D detection on LiDAR data. Apparently, the challenges of AutoExpert lie in the data-modality and task discrepancy. Nevertheless, public foundation models (FMs) serve as promising tools to tackle these challenges. To address AutoExpert, we adopt a conceptually simple pipeline consisting of three components: (1) 2D object detection and segmentation in RGB images, (2) lifting 2D detections into 3D using known sensor poses, and (3) 3D cuboids generation for the 2D detections. Within this pipeline, we enhance and evaluate a variety of methods such as open-vocabulary detectors, few-shot detectors, and self-supervised learned detectors. We also develop novel techniques, leading to refined components that boost 3D detection mAP from 12.1 to 25.4 on the AutoExpert-nuScenes benchmark.

cs.CV↗

arXiv2026

Few-shot Classification as Multi-instance Verification: Effective Backbone-agnostic Transfer across Domains

We investigate cross-domain few-shot learning under the constraint that fine-tuning of backbones (i.e., feature extractors) is impossible or infeasible---a scenario that is increasingly common in practical use cases---and show how representing few-shot classification as a series of multiple instance verification (MIV) tasks enables competitive classification performance using the static embeddings produced by frozen, "black-box" backbones. More specifically, based on this novel representation, we introduce a new approach to few-shot domain adaptation named the "MIV-head": a classification head that is backbone-agnostic and computationally efficient. We show that the core components of the MIV-head, when trained on few-shot data from a target domain, collectively yield strong performance on test data from that domain. Importantly, they do so without fine-tuning the backbone, requiring only the "meta-testing" phase. In experiments under various settings on an extension of the Meta-Dataset benchmark for cross-domain few-shot image classification, using representative off-the-shelf convolutional neural network and vision transformer backbones pretrained on ImageNet1K, we demonstrate that the MIV-head achieves highly competitive accuracy when compared to state-of-the-art "adapter" (or "partial fine-tuning") methods applied to the same backbones, while incurring substantially lower adaptation cost. Moreover, compared to our approach, well-known "classification head" approaches achieve a substantially lower level of accuracy. We share our code at: https://github.com/xxweka/MIV-head.

cs.CV↗