Simulate to Generalize: Scaling Stateful Supervision for API-calling Agents using LLM World Models

Abstract

Training agents that generalize to unseen, stateful environments requires a massive dataset of state-changing trajectories covering a vast and diverse set of APIs. However, scaling this broad supervision is severely bottlenecked by the immense effort required to implement and populate fully-executable environments across a broad spectrum of domains. To bypass this barrier, we introduce a data generation pipeline that decouples data synthesis from environment construction by leveraging LLMs as digital world models. Starting from only a list of broad domain names, our automated pipeline synthesizes diverse APIs and tasks. To produce trajectories, a teacher agent iteratively solves these tasks while an LLM simulator dynamically tracks state and provides coherent API responses on-the-fly. Finally, an automated judge filters the trajectories for quality. Fine-tuning on our broad synthetic dataset yields significant performance gains on AppWorld and OfficeBench, two challenging stateful benchmarks featuring environments completely unseen during training. These results establish our LLM world model-based synthesis approach as a highly scalable path for training generalizable, stateful API-calling agents.