Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.29014

GenXR-Bench: Benchmarking Coding Agents for Interactive XR Development

William Huang·David Li·Xiang 'Anthony' Chen·Xun Qian·Nels Numan·Zhongyi Zhou·Hyunsung Cho·Salman Muin Kayser Chishti·Ding Xia·Xincheng Huang·Jiahao Ren·Faraz Faruqi·Tianyu Xu·Yang Zhang·David Kim·Ruofei Du
View original source ↗Read source document ↗Export metadata ↓

Abstract

Coding agents are rapidly advancing from single-file generation to long-horizon repository-scale engineering, yet the evaluation of their capabilities to author interactive extended reality (XR) applications remains an open challenge. Unlike traditional software, XR applications operate within dynamic 3D physical environments where runtime robustness depends jointly on continuous user motion, spatial input, and environmental sensing. We introduce GenXR-Bench, the first benchmark for the end-to-end evaluation of coding agents in XR software development. GenXR-Bench contains 112 expert-authored tasks spanning 24 feature implementations, 21 bug repairs, and 67 application generation tasks. Held-out tests combine scripted and adaptive interactions with programmatic and visual assertions to evaluate core functionality and runtime behavior. GenXR-Bench's testing suite achieves 96.5% accuracy against blinded expert judgments of 124 candidate applications executed on consumer XR hardware. Evaluating 10 models on GenXR-Bench, we find that the strongest configuration achieves only 50.9% overall Pass@1. Failure analysis identifies recurring misunderstandings in asynchronous sensor lifecycles, physical grounding, and embodied interaction semantics. GenXR-Bench establishes a reproducible testbed for evaluating coding agents in the interactive and physically situated conditions of spatial computing.

Explore related subjects

cs.AIcs.GRcs.HC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

William Huang, David Li, Xiang 'Anthony' Chen, Xun Qian, Nels Numan, Zhongyi Zhou, Hyunsung Cho, Salman Muin Kayser Chishti, Ding Xia, Xincheng Huang, Jiahao Ren, Faraz Faruqi, Tianyu Xu, Yang Zhang, David Kim, Ruofei Du. 2026-10-21. GenXR-Bench: Benchmarking Coding Agents for Interactive XR Development. https://arxiv.org/abs/2610.29014

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Generative Artificial Intelligence for Internet of Things Computing: A Systematic Survey

The integration of Generative Artificial Intelligence (GenAI) within the Internet of Things (IoT) is garnering considerable interest. This growing attention stems from the continuous evolution and widespread adoption they are both having individually, enough to spontaneously reshape numerous sectors, including Healthcare, Manufacturing, and Smart Cities. Hence, their increasing popularity has catalyzed further extensive research for understanding the potential of the duo GenAI-IoT, how they interplay, and to which extent their synergy can innovate the state-of-the-art in their individual scenarios. However, despite the increasing prominence of GenAI for IoT Computing, much of the existing research remains focused on specific, narrowly scoped applications. This fragmented approach highlights the need for a more comprehensive analysis of the potential, challenges, and implications of GenAI integration within the broader IoT ecosystem. This survey exactly aims to address this gap by providing a holistic overview of the opportunities, issues, and considerations arising from the convergence of these mainstream paradigms. Our contribution is realized through a systematic literature review following the PRISMA methodology. A comparison framework is presented, and well-defined research questions are outlined to comprehensively explore the past, present, and future directions of GenAI integration with IoT Computing, offering valuable insights for both experts and newcomers.

cs.AI↗
arXiv2026

PRISM: Prompt-Refined In-Context System Modelling for Financial Retrieval

Deploying LLMs for financial information retrieval demands accuracy, cost efficiency, and reproducibility, yet most approaches require expensive fine-tuning. PRISM is a training-free framework that composes three deployable modules (prompt engineering, in-context learning, and a candidate refinement module) with a boundary-condition study of multi-agent coordination for financial document retrieval. On FiQA-2018, stacked reranking and $k$-reciprocal encoding raise Recall@100 from 0.7783 to 0.8326. On FinAgentBench, a Cohere-refined top-50 pool yields a higher observed end-to-end NDCG@5, below this evaluation's minimum detectable effect, while cutting per-query cost by about 35%. Added complexity does not always pay off: $k$-reciprocal encoding stops helping once the upstream stage is already high-recall, and multi-agent coordination is not a default for fine-grained chunk ranking. Matched pairs that hold prompt, model, and exemplars fixed lose 0.0702 and 0.0681 combined NDCG@5 to their single-path controls, and the whole gap sits at the chunk stage. We release an expert-annotated FinAgentBench evaluation set, together with a label-robustness sweep over 16 alternative ground-truth constructions, restoring a reproducible evaluation path after the original evaluator went offline. Without fine-tuning, PRISM placed third on the official FinAgentBench private leaderboard (leaderboard basis, NDCG@5 0.71181), reaches NDCG@10 0.6197 on FiQA-2018, and 98% accuracy on FinanceBench. We report per-query latency, token, and dollar accounting across roughly 40 configurations, present the quality-cost-latency Pareto frontier, and introduce FACET, a transparent SLA-specific scalarization for choosing among non-dominated configurations. Code and dataset are released at https://bit.ly/prism-ailens.

cs.AI↗
arXiv2026

Synthesis and Verification of Register Automata over Data Sequences (Technical Report)

Data sequences, such as time series, traces, and sensor streams, are words over large or infinite data domains. Their properties often depend on comparisons between values at different positions, which finite automata cannot express directly. We study synthesis and verification for Deterministic Register Automata (DRAs) over ordered data sequences. DRAs combine finite control with registers for remembered data values, providing compact symbolic descriptions of quantitative sequence languages. Our technical core is a local-search synthesis procedure and an exact robustness checker for DRAs. We fix the alphabet theory to ordered rationals $(\Q;<,=)$ with rational guard constants and use non-negative affine accumulator costs. For a fixed number of DRA registers, the checker runs in polynomial time; its dependence on that number is exponential. It reduces robustness to coverability and then to a shortest-path problem. Since the search ranges over $\Q^+$, it produces variable-length witnesses and, for RAA-definable alignment costs such as DTW, explores a counterexample space with no a priori length bound. As an application, we use learned DRAs as interpretable surrogates for queried neural sequence classifiers. A probably approximately correct (PAC) validation loop refines rejected candidates, returns genuine non-robustness witnesses for the classifier when found, and accepts a surrogate only with statistical guarantees for fidelity and local stability. We implement the framework in $\textsc{RAExc}$, enabling the synthesis of interpretable automata and the verification of their robustness. We evaluate the approach on benchmark languages and sequentialized MNIST, demonstrating high accuracy while verifying the robustness of the learned automata.

cs.AI↗