Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2107.09807

Improved Reinforcement Learning in Cooperative Multi-agent Environments Using Knowledge Transfer

Mahnoosh Mahdavimoghaddam·Amin Nikanjam·Monireh Abdoos
View original source ↗Read source document ↗Export metadata ↓

Abstract

Nowadays, cooperative multi-agent systems are used to learn how to achieve goals in large-scale dynamic environments. However, learning in these environments is challenging: from the effect of search space size on learning time to inefficient cooperation among agents. Moreover, reinforcement learning algorithms may suffer from a long time of convergence in such environments. In this paper, a communication framework is introduced. In the proposed communication framework, agents learn to cooperate effectively and also by introduction of a new state calculation method the size of state space will decline considerably. Furthermore, a knowledge-transferring algorithm is presented to share the gained experiences among the different agents, and develop an effective knowledge-fusing mechanism to fuse the knowledge learnt utilizing the agents' own experiences with the knowledge received from other team members. Finally, the simulation results are provided to indicate the efficacy of the proposed method in the complex learning task. We have evaluated our approach on the shepherding problem and the results show that the learning process accelerates by making use of the knowledge transferring mechanism and the size of state space has declined by generating similar states based on state abstraction concept.

Explore related subjects

cs.MAcs.AIcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Mahnoosh Mahdavimoghaddam, Amin Nikanjam, Monireh Abdoos. 2022-01-17. Improved Reinforcement Learning in Cooperative Multi-agent Environments Using Knowledge Transfer. https://arxiv.org/abs/2107.09807

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The Mechanics of a Swarm: A Reproducible External Reconstruction of an Unintended Agent-Coordination Episode on a Third-Party Wiki

Between 24 May and 2 July 2026, autonomous language-model agents running inside a timed research-question evaluation wrote to a third party's public, world-writable wiki. OpenAI acknowledged the incident; independent researchers reconstructed it and published the wiki's archived revision history. We analyse that history (14,591 revisions, 3,103 names, 4,579 pages) as a behavioural record, attributing text to the revision that added it. Under an explicit identity model we reconstruct 907 cohorts and estimate about 876 episodes (95% interval 784-1008). Coordination formats converged within a day, and heterogeneous schedules over one question chain created large opportunities for information asymmetry: the first report of an item preceded a later cohort's own arrival by a median of 3.4 h. Across the 510 cohorts with an observable progress trace we find no robust positive association between measured coordination and documented progress. This version adds a source the export lacks: the wiki operator's own request log, 5,157,202 records over four months. It holds roughly 2.66M content requests and 1.58M searches, and 7,254 acting names against the export's 3,103; 2,578 names neither save nor open an edit form. Content requests before writing are observed for 1,034 of 1,140 coordinating names, and the first coordination page is requested 17 s after its creation. These records establish requests, not delivery or causal use. Among newcomers without a marker on their first written page, prior requests to other marker-bearing pages occur for 40.2% of marker adopters and 31.7% of non-adopters. The association remains, but our first-pass reading of it as transmission is withdrawn: page choice, shared behaviour and action-dependent nameability prevent causal identification. We list the claims from our earlier analyses that re-examination overturned, including one from this version's own first pass

cs.MA↗
arXiv2026

CityLearn v3: A Configurable Simulation and Evaluation Framework for Realistic Control Studies of Renewable Energy Communities

Renewable energy communities (RECs) coordinate buildings, photovoltaic generation, batteries, electric vehicles and flexible loads. Controller studies often simplify changing participation, equipment availability, service deadlines and data quality, so lower cost or peak demand can conceal missed services or infeasible power requests. This paper presents CityLearn v3, a configurable simulation and evaluation framework for REC control studies under these conditions. It represents changing members and assets, flexible-load deadlines, demand-response requests, local energy sharing, and data or equipment failures within one simulation environment. Building and phase power limits constrain controllable requests, while a declared timestep preserves consistent power-to-energy accounting. The framework records controller inputs and distinguishes requested actions from those applied to the simulated equipment. Reference controllers, service- and constraint-aware performance indicators, and trajectory exports support comparisons within and across communities. Software checks and application examples examine service delivery, electrical constraints, settlement and changing scenarios; a synthetic high-frequency trace replay illustrates how aggregation can conceal short peaks without changing annual energy. Together, these records allow aggregate performance to be interpreted alongside service failures, action reductions and participant-level outcomes.

cs.MA↗
arXiv2026

MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS

Large Language Model (LLM)-based Multi-Agent Systems (MAS) are susceptible to linguistic attacks that can trigger cascading failures across the network. Existing defenses face a fundamental dilemma: lightweight single-auditor methods are prone to single points of failure, while robust committee-based approaches incur prohibitive computational costs in multi-turn interactions. To address this challenge, we propose \textbf{MAS-Shield}, a secure and efficient defense framework designed with a coarse-to-fine filtering pipeline. Rather than applying uniform scrutiny, MAS-Shield dynamically allocates defense resources through a three-stage protocol: (1) \textbf{Critical Agent Selection } strategically targets high-influence nodes to narrow the defense surface; (2) \textbf{Light Auditing} employs lightweight sentry models to rapidly filter the majority of benign cases; and (3) \textbf{Global Consensus Auditing} escalates only suspicious or ambiguous signals to a heavyweight committee for definitive arbitration. This hierarchical design effectively optimizes the security-efficiency trade-off. Experiments demonstrate that MAS-Shield achieves a 92.5\% recovery rate against diverse adversarial scenarios and reduces defense latency by over 70\% compared to existing methods.

cs.MA↗