Search arXiv⌕ Search

arXiv · 2610.25064

One Skill Takes Over: Clashing Agent Skills and a One-File Fix

Abstract

Agent Skills are instruction files that tell a coding agent how to do one job. Users install skills from many authors, so two skills for the same job can be active at the same time. We ask what a model does when both skill files are in its context: does it combine their compatible rules, or follow one skill and drop the other? We build SkillClash, 16 pairs of same-job skills in which code checks every rule and a reference output shows the best result that can be reached. On seven models from five families, from 4B open models to Qwen3-235B and GPT-4.1, a skill's rules are followed 93 to 99% of the time when it is loaded alone and 61 to 62% when a same-job skill is loaded next to it. Most outputs show takeover: every rule that only one skill asks for is followed, and every rule that only the other skill asks for is dropped. The worked examples inside the skills drive this. When each skill file instead ends with one example that follows both skills, takeover falls to 1 to 15% on the six models we tested. Telling the model which skill has priority does not bring the dropped skill back, and telling it to apply both skills brings the dropped skill back only on the two strong models and leaves the interacting rules broken. One merged skill file written by the strongest model we used, Claude Opus 5.5, lets all seven models follow 96 to 99% of the achievable rules. Most of its gain over a plain list of both skills' rules comes from its one worked example, and such an example helps only when it is correct. Code and data: https://github.com/prazmara/skillclash

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Parsa Razmara, Seyedarmin Azizi, Ardalan Aryashad. 2026-10-18. One Skill Takes Over: Clashing Agent Skills and a One-File Fix. https://arxiv.org/abs/2610.25064

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Agents-as-Peers: Bias-Controlled LLM Evaluation as Multi-Agent Peer Assessment

Large language model (LLM) evaluation commonly relies on fixed human-authored benchmarks or centralized human/model judges. Fixed benchmarks can age and enter model training corpora, while centralized judges introduce their own scoring preferences. We cast LLM evaluation as a multi-agent peer-assessment problem and introduce Agents-as-Peers, a distributed evaluation protocol in which every agent generates tasks, answers the shared task set, and judges the answers of its peers, with no privileged evaluator. Self-ratings are excluded from the ranking, and judging is repeated under controlled identity and presentation regimes so that self bias, identity effects, presentation-position sensitivity, and judge severity are measured rather than left implicit. In an instantiation with 12 LLMs and 420 agent-generated questions, Agents-as-Peers aggregates 55,440 non-self peer judgments per regime. Most agents assign their own answers higher scores than those answers receive from peer judges; visible identity changes scores for some agents; the first fixed answer slot is associated with scores 0.39 points higher than the shuffled baseline; and judges differ markedly in severity. On labeled benchmarks, peer scores align strongly with accuracy on TruthfulQA (Pearson r=0.904) and GSM8K (r=0.873). On TruthfulQA, the peer-score correlation is substantially higher than the self-score correlation (r=0.538). A benchmark-overlap screen and item-level human audit further characterize the endogenous task set. These results support Agents-as-Peers as a cohort-relative measurement protocol whose ranking is reported together with the biases and rater effects that shape it. Code and dataset: https://github.com/caura-ai/caura-PeerRank

cs.AI↗

From Debate to Decision: Conformal Social Choice for Safe Multi-Agent Deliberation

Multi-agent debate improves LLM reasoning, yet agreement is not evidence of correctness. When agents converge on a wrong answer through social reinforcement, consensus stopping commits that error to automated action with no recourse. We introduce Conformal Social Choice, a post-hoc layer that converts debate outputs into calibrated act-versus-escalate decisions. Verbalized probability distributions from heterogeneous agents are aggregated via a linear opinion pool and calibrated with split conformal prediction, yielding fixed-round marginal coverage: at any pre-specified round, the correct answer is included with probability ${\geq}\,1{-}α$, without assumptions on individual model calibration. A hierarchical action policy maps singletons to autonomous action and larger sets to human escalation. Across eight MMLU-Pro domains and two additional benchmarks, fixed-round coverage stays close to target. The key finding is not that debate becomes more accurate, but that the conformal layer makes its failures actionable: at the final round, 81.9\% of wrong-consensus cases are flagged by non-singleton sets at $α{=}0.05$. Separately, among cases that reach a singleton before the final round, accuracy reaches 90.0--96.8\% on MMLU-Pro (up to 22.1pp above consensus stopping): a selection effect, not a reasoning improvement. Simple probability- and entropy-threshold rules attain comparable interception; conformal's distinct value is a finite-sample fixed-round coverage guarantee together with explicit candidate sets for escalation. Adaptive first-singleton stopping is evaluated empirically and not covered by this guarantee.

cs.AI↗

Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies

Reporting guidance for information-theoretic measures is rarely tested against ground truth. We test one guardrail in two multi-agent reinforcement learning games, a social dilemma and a coordination race, where directed influence between selected agent pairs is zero by construction, over 100 seeds. Omitting one precondition, exclusion of the non-stationary training transient, gives false-positive rates of 100.00% and 99.95%: agents annealing exploration independently, in runs that never met, are flagged as influencing one another. Excluding the transient reaches 3.05% in the social dilemma but 11.8% in the coordination game, so the non-stationarity must be treated and exclusion is neither the only way nor sufficient. What we recommend instead changes the null model. Series, statistic and estimand are left untouched: permuting the source within blocks of training time reaches 5.25% and 5.50%, the only one of the three at or below the size of the test in both games. The construction is not new, being a restricted permutation local to blocks of training time; the measurement is. A third null control, four learners in the social dilemma, defeats all three, and there conditioning costs three times what ours does, 15.20 points against 4.55. Titrating injected links separates it from conditioning on identical pairs, 82.5% against 58.0%, while the ablated test reports a positive with or without a link, so its apparent sensitivity is uninformative. Five block counts spanning 16 to 256 land in the nominal region. Code and data are released.

cs.AI↗