Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2605.24222

Analyzing the Effects of Two-Stage Peer Evaluation

Roy Fairstein·Harper Lyon·Oshri Damty·Omer Lev·Nicholas Mattei·Kobi Gal
View original source ↗Read source document ↗Export metadata ↓

Abstract

Peer-evaluation and selection systems are used when sets of agents evaluate each other in order to select the best $k$ among them. These are commonly used in real-world settings, including academic conferences where those reviewing papers are often the set of submitters. Conferences have attempted to better allocate their reviewing resources by moving to a two-stage mechanism, in which some papers are eliminated after a first stage of review and remaining papers receive additional reviewers. We investigate how two major strategyproof peer selection mechanisms, Partition and ExactDollarPartition, perform when adapted to a two-stage system, in order to try and understand the effect of the two-stage mechanism on which agents get selected. We also examine how the various parameters of the two-stage mechanism influence the outcome. We provide a theoretical basis by showing how a particular setting is influenced by the two stages. However, solving for the general case seems implausible at the moment, and we use extensive simulations of different scenarios and settings to observe which agents benefit and which are harmed by adopting two-stage mechanisms (and we vary this mechanisms parameters as well). We show that the two-stage mechanism's advantage depends the noisiness of reviewer beliefs. Borderline agents benefit most in a low noise environment, while high rank agents benefit more in noisy environments. We show that the effectiveness of these mechanisms is highly dependent on the number of chosen agents, the number of reviews requested from agents, and reviewers' correlation, indicating that organizers need to exercise caution when selecting these parameters for a reviewing process.

Explore related subjects

cs.GT

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Roy Fairstein, Harper Lyon, Oshri Damty, Omer Lev, Nicholas Mattei, Kobi Gal. 2026-05-22. Analyzing the Effects of Two-Stage Peer Evaluation. https://arxiv.org/abs/2605.24222

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Robust Information Design with Heterogeneous Beliefs in Bayesian Congestion Games

In many engineered systems, agents make decisions under incomplete information, creating opportunities for a planner to influence decentralized behavior through signaling. We study how such signaling can be designed in parallel-network, affine latency congestion games when users may not interpret recommendations using the same beliefs assumed by the planner. To do so, we consider Bayesian congestion games with private recommendations and formulate a robust information design problem in which obedience must hold uniformly over a neighborhood of a nominal prior. This addresses the previously uncharacterized issue of whether obedience itself remains reliable under belief heterogeneity, rather than only under the single prior used at the design stage. We characterize policy-level robustness radii, identify regimes in which the robust obedience region remains nonempty, and analyze the resulting robustness--performance tradeoff through a robust value function whose optimal cost is monotone in the robustness requirement and whose local sensitivity is governed by the active obedience constraints.

cs.GT↗
arXiv2026

Core stability recognition for minimum-cost spanning tree games: Parameterized perspective

Minimum-cost spanning tree game (MSTG) is a cooperative game played on an undirected edge-weighted graph $(G,w)$ representing the network, where each vertex corresponds to a player and each edge has an associated cost~$w$. A distinguished vertex $s \in V(G)$ represents the supply or source. For any coalition of players $S$, the characteristic cost function $c(S)$ is defined as the minimum cost of a spanning tree with respect to $w$, connecting exactly the vertices in $S \cup \{s\}$. In this paper we study the computational complexity of deciding core membership for MSTG. In general, deciding whether a given allocation is in the core is \textsf{coNP}-hard~(Faigle et al.,International Journal of Game Theory,1997). We study the core recognition problem under the name {\sc MSTG Core Non-Membership}. We extend the hardness to graphs which are very close to being planar. On the positive side, we present several algorithmic results within the framework of parameterized complexity. We show that {\sc MSTG Core Non-Membership} is fixed-parameter tractable when parameterized by the support size of the allocation. Turning into structural parameters of graphs, we show that the problem admits an FPT algorithm parameterized by treewidth and signed neighborhood diversity. Last but not least, we investigate kernelization. While in general graphs, under standard complexity-theoretical assumptions, {\sc MSTG Core Non-Membership} does not admit a polynomial kernel parameterized by the vertex cover number, we design a cubic kernel in planar graphs. Furthermore, in general graphs, we obtain quadratic kernel for signed neighborhood diversity and linear kernel for the parameter feedback edge number.

cs.GT↗
arXiv2026

Condorcet-type properties of the linear ordering problem with ties

The Kemeny rule aggregates multiple strict rankings into a single strict ranking that minimizes the sum of its distances from the input rankings. The resulting optimization problem, called the Kemeny problem (\texttt{KP}), is a special case of the linear ordering problem (\texttt{LOP}). The Kemeny rule satisfies several desirable properties in social choice theory, including the extended Condorcet criterion (\texttt{XCC}). Ando et al. strengthened this result by introducing the strong Condorcet criterion (\texttt{SCC}) and showing that it holds for every optimal solution to an arbitrary \texttt{LOP} instance. Yoo and Escobedo extended the Kemeny rule to rankings with ties and showed that the resulting rule satisfies the non-strict extended Condorcet criterion (\texttt{NXCC}). This criterion gives a condition under which one candidate must be ranked strictly above another in every optimal solution. In this paper, we introduce the non-strict strong Condorcet criterion (\texttt{NSCC}), a counterpart of the \texttt{SCC} for rankings with ties, and show that it holds for every optimal solution to an arbitrary instance of the linear ordering problem with ties (\texttt{LOPT}). We also establish a complementary structural property that gives conditions under which two candidates must be tied in every optimal solution to an arbitrary \texttt{LOPT} instance.

cs.GT↗