Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1401.5986

How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature

Lutz Bornmann
View original source ↗Read source document ↗Export metadata ↓

Abstract

As the subject of research excellence has received increasing attention (in science policy) over the last few decades, increasing numbers of bibliometric studies have been published dealing with excellent papers. However, many different methods have been used in these studies to identify excellent papers. The present quantitative analysis of the literature has been carried out in order to acquire an overview of these methods and an indication of an "average" or "most frequent" bibliometric practice. The search in the Web of Science yielded 321 papers dealing with "highly cited", "most cited", "top cited" and "most frequently cited". Of the 321 papers, 16 could not be used in this study. In around 80% of the papers analyzed in this study, a quantitative definition has been provided with which to identify excellent papers. With definitions which relate to an absolute number, either a certain number of top cited papers (58%) or papers with a minimum number of citations are selected (17%). Around 23% worked with percentile rank classes. Over these papers, there is an arithmetic average of the top 7.6% (arithmetic average) or of the top 3% (median). The top 1% is used most frequently in the papers, followed by the top 10%. With the thresholds presented in this study, in future, it will be possible to identify excellent papers based on an "average" or "most frequent" practice among bibliometricians.

Explore related subjects

cs.DL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Lutz Bornmann. 2014-01-22. How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature. https://arxiv.org/abs/1401.5986

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A framework for linking literature-based knowledge integration and infrastructure-supported knowledge integration: Opportunities and challenges from a case study

Integrating knowledge across disciplines is central to sustainability research, yet most evidence-synthesis methods rely on findings as reported in publications, limiting verification and reuse of underlying data and workflows. We develop a conceptual framework linking literature-based and infrastructure-supported knowledge integration, using a systematic review case study to examine when integration can extend beyond reported findings. We reviewed 37 studies on climate change, violent conflict, and household food security. Literature-based synthesis enabled integration across all included studies, whereas access to reusable outputs was limited: over half provided no data availability statement, 27% reported availability upon request, but reusable data and workflows were available for only 8%. To explore infrastructure-supported integration, we used the TIB Knowledge Loom to represent studies with accessible data and code as machine-readable outputs, and produced a knowledge gap map (KGM) from manually extracted and Loom-derived data, comparing manual and infrastructure-supported synthesis. Where outputs were reusable, synthesis could be produced directly from data and workflows rather than from publications. These findings show that literature-based synthesis can be complemented by infrastructure-supported integration where outputs are accessible and usable, and that advancing knowledge integration depends not only on infrastructures but on making data, code, and workflows accessible, executable, and reusable.

cs.DL↗
arXiv2026

TiAb Review Plugin: A Browser-Based Tool for AI-Assisted Study Selection in Systematic Reviews

Server-based screening tools impose subscription costs, while open-source alternatives require coding skills, and full-text screening has remained outside the scope of no-code open-source tools. We developed TiAb Review Plugin, an open-source Chrome browser extension that provides no-code, serverless artificial intelligence (AI)-assisted study selection covering both title and abstract (T&A) screening and full-text screening. It uses Google Sheets as a shared database and Google Drive as a PDF store, and users supply their own large language model (LLM) API key. For T&A screening, it offers manual review, LLM batch screening, and machine learning (ML) active learning. For full-text screening, it retrieves open-access PDFs from PubMed Central, Europe PMC, Unpaywall, OpenAlex, and publisher pages, supports blinded dual review with structured exclusion reasons and adjudication, optionally obtains an LLM judgment with page-anchored evidence, and computes PRISMA 2020 flow counts. We re-implemented the default ASReview algorithm (TF-IDF with Naive Bayes) in TypeScript and compared it with the Python original using 10-fold cross-validation on six datasets. For LLM T&A screening, we compared 16 parameter configurations on a benchmark dataset, validated the best (Gemini 3.0 Flash, low thinking budget, TopP 0.95) on five public datasets (1,038 to 5,628 records; 0.5% to 2.0% prevalence), and benchmarked nine further models from four developers. The TypeScript classifier produced top-100 rankings identical to ASReview on all six datasets. LLM T&A screening achieved recall of 94% to 100% with precision of 2% to 15%, and work saved over sampling at 95% recall (WSS@95) of 46.3% to 89.3%. No additional model exceeded the 96.1% recall of the reference configuration; the most recent models traded recall for precision. The classification accuracy of the full-text stage has not yet been evaluated.

cs.DL↗
arXiv2026

PaperAtlas: an automatically constructed atlas of computational methods and software from 6.4 million open-access articles

Computational methods and software are dispersed across a literature that is increasingly difficult to review manually, while curated registries capture only a subset of available resources. We present PaperAtlas, an automatically constructed atlas derived from the PubMed Central open-access corpus. Of 6,446,741 abstracts screened, 1,074,191 were classified as computational, with schema-valid records extracted for 1,074,140. Among these, 267,893 papers describing an algorithm, software package or web server formed 1,438 clusters, with 1,000 retained after restriction to parent-level biomedical categories. In 296 clusters linked to at least five bio.tools entries, the mean concentration of the most frequent EDAM topic was 61.7%, compared with 26.4% under permutation. Of 31,180 distinct software and web-server names, 61.0% lacked a strict match in bio.tools, PyPI, CRAN, Bioconductor or Bioconda. Among 12,207 bio.tools entries with defining papers in the corpus, 56.6% were recovered end to end, increasing to 83.6% among the 8,265 entries whose defining publication entered the atlas. PaperAtlas is fully open source and available as a web server at bhargavaresearch.org/paperatlas.

cs.DL↗