The Emergence of Strategic Reasoning of Large Language Models

Abstract

As large language models (LLMs) have demonstrated strong reasoning abilities in structured tasks (e.g., coding and mathematics), we explore whether these abilities extend to strategic multi-agent environments. We investigate strategic reasoning capabilities -- the process of choosing an optimal course of action by predicting and adapting to others' actions -- of LLMs by analyzing their performance in three classical games from behavioral economics. Using hierarchical models of bounded rationality, we evaluate three standard LLMs (ChatGPT-4, Claude-3.5-Sonnet, Gemini 1.5) and three reasoning LLMs (OpenAI-o1, Claude-4-Sonnet-Thinking, Gemini Flash Thinking 2.0). Our results show that reasoning LLMs exhibit superior strategic reasoning compared to standard LLMs (which do not demonstrate substantial capabilities) and often match or exceed human performance; this represents the first and thus most fundamental transition in strategic reasoning capabilities documented in LLMs. Since strategic reasoning is fundamental to future AI systems (including Agentic AI), our findings demonstrate the importance of dedicated reasoning capabilities in achieving effective strategic reasoning.