arXiv subjects

Beiwen Zhang

Publications and source records attributed to Beiwen Zhang.

5 recordsLinked to original sources

arXiv2026

CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making

How can generative offline multi-agent reinforcement learning achieve both fast joint trajectory generation and effective cooperation? Multi-agent diffusion models generate joint trajectories through iterative sampling but incur high inference latency. Distillation reduces sampling steps but may lose the teacher's cross-agent dependencies, reducing the success rate of cooperative tasks. A joint few-step generator can be trained directly to avoid losing these dependencies during distillation. However, insufficient modeling of cross-agent dependencies may still impair cooperation. In addition, its consistency training process introduces heavy computational and memory costs. Therefore, we propose Coordinated Few-Step Flow (CoFlow), which directly learns a joint averaged-velocity field for distillation-free one-step and few-step multi-agent trajectory generation. To support cooperation during generation, Coordinated Velocity Attention (CVA) incorporates teammate trajectory information into each agent's prediction. To reduce the consistency-training cost of this joint model, we approximate the correction term using finite differences. Across 48 configurations on MPE and SMAC, CoFlow supports centralized and decentralized execution. Under centralized execution, CoFlow outperforms our reproduced baseline by 11.2%, averaging relative gains equally across both benchmark suites, with a 12.93-fold model-sampling speedup. CVA improves normalized scores by 165.1% on average over the same model with cross-agent attention disabled. Compared with the exact-derivative implementation, finite-difference training achieves a 1.78-fold speedup and reduces peak GPU memory by 41.1%. Project page: https://github.com/Guowei-Zou/coflow.

cs.AI↗

arXiv2026

PACT: Phenotype-Aware Contrastive Team Representation for Multi-Phenotype Grouped Ad Hoc Teamwork

Learning to collaborate with various unfamiliar teammates poses a great challenge in the domain of multi-agent systems. Existing ad hoc teamwork methods typically drive controlled agents to collaborate with a group of teammates exhibiting a single coordination phenotype shaped by the same reward function. However, in real-world applications, controlled agents should collaborate with unfamiliar teammates of diverse coordination phenotypes among groups that have never worked together. We formalize this as the Multi-Phenotype Grouped Ad Hoc Teamwork (MPG-AHT) problem, and propose Phenotype-Aware Contrastive Team Representation (PACT) to solve this problem. PACT is empowered with phenotype-aware contrastive learning and relational reasoning to accurately distinguish coordination phenotypes and capture inter-agent interactions. Extensive experiments on multi-phenotype collaboration tasks show that PACT outperforms state-of-the-art baselines on average, achieving a mean 21.0% gain in out-of-distribution evaluation and a mean 36.5% gain in sample efficiency.

cs.MA↗

arXiv2026

Distilling LLM Reasoning into an Interpretable Policy Tree for Human-AI Collaboration

Constructing efficient and reliable policies to assist humans is indispensable for human-AI collaboration. Existing methods mainly follow two lines of work. Most prior work relies on multi-agent reinforcement learning (MARL) to learn black-box policies, which limits interpretability and raises safety concerns. Recent methods query large language models (LLMs) at each decision step, causing slow responses and high inference costs. We propose Collaboration Policy Tree (Co-pi-tree), a closed-loop method that learns an executable policy tree consisting of a partner-behavior prediction tree and an agent-action selection tree. Co-pi-tree constructs a policy by distilling LLM reasoning into policy tree code. It then evaluates the policy through partner interaction, obtains feedback, and uses natural language to summarize the interaction feedback to improve problematic branches. Experiments in Overcooked-AI show that Co-pi-tree improves average reward by 35.4% over the baseline average, while reducing the number of LLM queries by 77.7% and test-time latency by 97.1%. Project page: https://beiwenzhang.github.io/Co-pi-tree/

cs.AI↗

arXiv2025

ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention

Transformer models rely on self-attention to capture token dependencies but face challenges in effectively integrating positional information while allowing multi-head attention (MHA) flexibility. Prior methods often model semantic and positional differences disparately or apply uniform positional adjustments across heads, potentially limiting representational capacity. This paper introduces ComplexFormer, featuring Complex Multi-Head Attention-CMHA. CMHA empowers each head to independently model semantic and positional differences unified within the complex plane, representing interactions as rotations and scaling. ComplexFormer incorporates two key improvements: (1) a per-head Euler transformation, converting real-valued query/key projections into polar-form complex vectors for head-specific complex subspace operation; and (2) a per-head adaptive differential rotation mechanism, exp[i(Adapt(ASmn,i) + Delta(Pmn),i)], allowing each head to learn distinct strategies for integrating semantic angle differences (ASmn,i) with relative positional encodings (Delta(Pmn),i). Extensive experiments on language modeling, text generation, code generation, and mathematical reasoning show ComplexFormer achieves superior performance, significantly lower generation perplexity , and improved long-context coherence compared to strong baselines like RoPE-Transformers. ComplexFormer demonstrates strong parameter efficiency, offering a more expressive, adaptable attention mechanism.

cs.LG↗

arXiv2025

Towards Analyzing and Understanding the Limitations of VAPO: A Theoretical Perspective

The VAPO framework has demonstrated significant empirical success in enhancing the efficiency and reliability of reinforcement learning for long chain-of-thought (CoT) reasoning tasks with large language models (LLMs). By systematically addressing challenges such as value model bias, heterogeneous sequence lengths, and sparse reward signals, VAPO achieves state-of-the-art performance. While its practical benefits are evident, a deeper theoretical understanding of its underlying mechanisms and potential limitations is crucial for guiding future advancements. This paper aims to initiate such a discussion by exploring VAPO from a theoretical perspective, highlighting areas where its assumptions might be challenged and where further investigation could yield more robust and generalizable reasoning agents. We delve into the intricacies of value function approximation in complex reasoning spaces, the optimality of adaptive advantage estimation, the impact of token-level optimization, and the enduring challenges of exploration and generalization.

cs.LG↗