Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2606.05818

Benchmarks in Leipzig

Andrei Balakin·Miklós Bóna·Marie-Charlotte Brandenburg·Clara Briand·Veronica Calvo Cortes·Shelby Cox·Jesus A. De Loera·Danai Deligeorgaki·Hannah Friedman·Tim Gehrunger·Chiara Giardino·Stephen Griffeth·Baran Hashemi·Elena Hoster·Alexander Ivanov·Nupur Jain·Aryaman Jal·Leonie Kayser·Joris Koefler·Kevin Kühn·Mario Kummer·Felix Lotter·René Marczinzik·Victor S. Miller·Alejandro Morales·Greta Panova·Gianni Petrella·Nathan Pflueger·Lakshmi Ramesh·Nikolas Rieke·Carlos Rodriguez·Andrea Rosana·Flavio Salizzoni·Otto T. P. Schmidt·Sven Ulf Schmitz·Lina Maria Simbaqueba Marin·Luca Sodomaco·Christian Stump·Bernd Sturmfels·Alexander Taveira Blomenhofer·Simon Telen·Philipp Tuchel·Emil Verkama·Carl Felix Waller·Julian Weigert·Annette Werner·Nathan Williams·Claudius Zibrowius
View original source ↗Read source document ↗Export metadata ↓

Abstract

Between April 1 and May 15, 2026, a group of 49 mathematicians compiled a dataset of research-level mathematics questions with known answers. Most of the work was done during the 3-day workshop *Benchmarks in Leipzig* with 35 participants at the Max Planck Institute for Mathematics in the Sciences in Leipzig, Germany. We present the resulting collection of 100 questions. We evaluated these questions in three stages: a single attempt by five state-of-the-art LLMs, followed by a 20-runs-per-model evaluation with three of these models, and finally a 3-run attempt with two heavy-thinking models. After Stage 1, 41 questions remained completely unsolved; after Stage 2, this count dropped to 16; and we concluded Stage 3 with only 2 unsolved questions. This demonstrates that the mathematical reasoning capabilities of LLMs are becoming impressive.

Explore related subjects

math.HOcs.AImath.AGmath.COmath.RT

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Andrei Balakin, Miklós Bóna, Marie-Charlotte Brandenburg, Clara Briand, Veronica Calvo Cortes, Shelby Cox, Jesus A. De Loera, Danai Deligeorgaki, Hannah Friedman, Tim Gehrunger, Chiara Giardino, Stephen Griffeth, Baran Hashemi, Elena Hoster, Alexander Ivanov, Nupur Jain, Aryaman Jal, Leonie Kayser, Joris Koefler, Kevin Kühn, Mario Kummer, Felix Lotter, René Marczinzik, Victor S. Miller, Alejandro Morales, Greta Panova, Gianni Petrella, Nathan Pflueger, Lakshmi Ramesh, Nikolas Rieke, Carlos Rodriguez, Andrea Rosana, Flavio Salizzoni, Otto T. P. Schmidt, Sven Ulf Schmitz, Lina Maria Simbaqueba Marin, Luca Sodomaco, Christian Stump, Bernd Sturmfels, Alexander Taveira Blomenhofer, Simon Telen, Philipp Tuchel, Emil Verkama, Carl Felix Waller, Julian Weigert, Annette Werner, Nathan Williams, Claudius Zibrowius. 2026-06-04. Benchmarks in Leipzig. https://arxiv.org/abs/2606.05818

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The Stairs of Reconciliation: A Mathematical Tourist in Graz

Inside the Grazer Burg, two late-Gothic stone flights rise about distinct spindles, overlap, share several treads, and separate again. Their plan is governed not by a coaxial double helix but, to first approximation, by two intersecting circles. This elementary geometry yields a model of recurrent meeting and makes explicit the compatibility conditions that meeting requires. It also leads to a second object that geometers call a double spiral staircase - the helicoid - and to a useful distinction between resemblance and identity. The staircase becomes a meditation on how paths, models, and disciplines can meet without becoming the same.

math.HO↗
arXiv2026

On the Reconstruction of SAS from Other Triangle Congruence Criteria

Starting from a Hilbert plane and removing the Side-Angle-Side (SAS) congruence axiom, we investigate to what extent SAS can be recovered synthetically from the remaining classical triangle congruence criteria. We show that the Angle-Side-Angle criterion, together with a ray correspondence principle corresponding to Theorem 13 of Hilbert's \emph{Grundlagen der Geometrie}, suffices to reconstruct SAS. We further show that both the Side-Side-Side and the Side-Angle-Angle criteria also suffice, once combined with the ray correspondence principle and suitable auxiliary principles -- the existence of midpoints and a hypotenuse-angle criterion for right triangles in the first case, and the existence of angle bisectors, the congruence of supplements of congruent angles, and the Pons Asinorum in the second. Although the two routes rely on auxiliary principles of different character, we show that they converge on a single final argument once a common hypotenuse-angle criterion is established. A metamathematical analysis, based on an explicit model adapted from Hilbert's own independence construction, complements these reconstructions: it shows that the ray correspondence principle alone cannot reconstruct any of the classical criteria, and that the Pons Asinorum and the hypotenuse-angle criterion are each independent of the remaining auxiliary principles used in their respective reconstructions. The resulting picture is not a formal hierarchy of the congruence criteria, but it does show that the Angle-Side-Angle reconstruction rests on a provably more economical basis than those obtained from Side-Side-Side or Side-Angle-Angle.

math.HO↗
arXiv2026

Come for the vibe, stay for the math

This article describes our experiences in mathematical outreach over the past decade. We talk about specific activities, but also general principles that we've learned along the way.

math.HO↗