Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1508.05624

Quantifying the quality of peer reviewers through Zipf's law

Marcel Ausloos·Olgica Nedic·Agata Fronczak·Piotr Fronczak
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper introduces a statistical and other analysis of peer reviewers in order to approach their "quality" through some quantification measure, thereby leading to some quality metrics. Peer reviewer reports for the Journal of the Serbian Chemical Society are examined. The text of each report has first to be adapted to word counting software in order to avoid jargon inducing confusion when searching for the word frequency: e.g. C must be distinguished, depending if it means Carbon or Celsius, etc. Thus, every report has to be carefully "rewritten". Thereafter, the quantity, variety and distribution of words are examined in each report and compared to the whole set. Two separate months, according when reports came in, are distinguished to observe any possible hidden spurious effects. Coherence is found. An empirical distribution is searched for through a Zipf-Pareto rank-size law. It is observed that peer review reports are very far from usual texts in this respect. Deviations from the usual (first) Zipf's law are discussed. A theoretical suggestion for the "best (or worst) report" and by extension "good (or bad) reviewer", within this context, is provided from an entropy argument, through the concept of "distance to average" behavior. Another entropy-based measure also allows to measure the journal reviews (whence reviewers) for further comparison with other journals through their own reviewer reports.

Explore related subjects

physics.soc-phcs.DL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Marcel Ausloos, Olgica Nedic, Agata Fronczak, Piotr Fronczak. 2015-08-23. Quantifying the quality of peer reviewers through Zipf's law. https://doi.org/10.1007/s11192-015-1704-5

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Quantifying the Dynamics of Innovation Abandonment Across Scientific, Technological, Commercial, and Pharmacological Domains

Despite the vast literature on the diffusion of innovations that impacts a broad range of disciplines, our understanding of the abandonment of innovations remains limited yet is essential for a deeper understanding of the innovation lifecycle. Here, we analyze four large-scale datasets that capture the temporal and structural patterns of innovation abandonment across scientific, technological, commercial, and pharmacological domains. The paper makes three primary contributions. First, across these diverse domains, we uncover one simple pattern of preferential abandonment, whereby the probability for individuals or organizations to abandon an innovation increases with time and correlates with the number of network neighbors who have abandoned the innovation. Second, we find that the presence of preferential abandonment fundamentally alters the way in which the underlying ecosystem breaks down, inducing a novel structural collapse in networked systems commonly perceived as robust against abandonments. Third, we derive an analytical framework to systematically understand the impact of preferential abandonment on network dynamics, pinpointing specific conditions where it may accelerate, decelerate, or have an identical effect compared to random abandonment, depending on the network topology. Together, these results deepen our quantitative understanding of the abandonment of innovation within networked social systems, with implications for the robustness and functioning of innovation communities. Overall, they demonstrate that the dynamics of innovation abandonment follow simple yet reproducible patterns, suggesting that the uncovered preferential abandonment may be a generic property of the innovation lifecycle.

physics.soc-ph↗
arXiv2026

Emotions as intrinsic colored noise in biological systems

The idea that emotions in biological systems are analogous to intrinsic colored noise is advanced and justified. A model describing the dynamics of operation of biological networks under the influence of colored noise is suggested. The agents of a biological network can be represented either by biological species, such as humans and animals, or by neurons of the brain, or by the nodes of a neural network. Operational actions, or decisions, in a biological noisy network are based not only on the evaluation of utility of alternatives, but also on the agents emotions. The model is probabilistic, with the choice of alternatives characterized by the related probabilities. At the initial step, the agents make decisions individually and then start exchanging information with each other and imitating the actions of other agents, thus forming a biological network of interacting agents. Numerical simulations are accomplished for a heterogeneous society consisting of three groups of agents, one group possessing long-range memory, the other group, short-range memory, and the third group of super-rational agents acting strictly on the basis of utility, being deprived of emotions. Dynamics of opinions in different groups can be smooth, oscillatory, or chaotic. Altogether, eight types of operation are found, depending on the dynamics of group decisions. Under strong imitation effect, there appears chaotic motion in the evolution of decision choice. It is shown how the Ellsberg paradox can be resolved and how the exchange of information influences the dynamics of this paradox.

physics.soc-ph↗
arXiv2026

Beyond expressiveness in pairwise and higher-order models

The debate over pairwise and higher-order models is often cast as a contest of expressive power, but this framing is misleading. A graph with arbitrary multivariate node functions can emulate the node-level dynamics of many hypergraph models, yet this does not erase the grouping information encoded by the hypergraph: it may simply be shifted from structure to dynamics. We distinguish four notions that are often conflated: structural projection, functional representability, statistical identifiability, and mechanistic adequacy. We show that the interaction order of a finite-state map is an invariant of the map itself, so an exact change of representation cannot reduce the underlying order of dependence. We then recast the comparison in terms of description length. At the unrestricted algorithmic level, a fixed compiler can move information between structure and rule with only constant overhead, so expressiveness alone does not privilege graphs or hypergraphs. Preferences arise only relative to explicit model classes, coding schemes, regularity assumptions, and data. This motivates an operational minimum-description-length criterion combining structural cost, rule cost conditional on structure, and model fit. For $M$ disjoint groups of size $k$, we show that the clique projection requires an edge list asymptotically $k-1$ times longer than the corresponding hyperedge list, making projection a more expensive encoding of the same grouping. Examples spanning diffusion, Boolean dynamics, ecological interactions, and ambiguous projections illustrate graph-preferred, hypergraph-preferred, and unresolved cases. The resulting position is symmetric: higher-order structure should not be inferred from phenomenology alone, but neither should graph-based emulation be taken as evidence that a graph is the most parsimonious or scientifically adequate description.

physics.soc-ph↗