Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2601.01764

Evidence for studying interactions between science and policy: An exploration of scholarly and policy references in Overton-indexed policy documents

Biegzat Murat·Zhichao Fang·Ed Noyons·Rodrigo Costas
View original source ↗Read source document ↗Export metadata ↓

Abstract

Overton, a global policy index, provides new opportunities to study the interactions between science and policy. This study aims to characterize the presence of scholarly and policy references in Overton-indexed policy documents and examine their distribution across key bibliographic dimensions, thereby assessing Overton's potential as a data source for policy metrics. We analyze a dataset of approximately 17.5 million policy documents from Overton, incorporating metadata such as publication year, policy source, country, language, subject area, and policy topic. Descriptive statistics are employed to assess the presence and distribution of reference data across these dimensions. Overton indexes a substantial volume of policy documents and identifies considerable reference data within them: 7.7% of documents contain scholarly references and 10.6% contain policy references. However, the presence of references varies significantly across publications years, source types, countries, languages, subject areas, and policy topics, indicating coverage biases that may affect interpretations of policy impact. The analysis is based on the Overton database as of June 2025. As Overton is regularly updated, the distribution patterns of indexed documents and references may evolve over time. The findings offer insights into the opportunities and constraints of using Overton for investigating evidence-based policymaking and for assessing the policy uptake of research outputs in the context of research evaluation. This is the first large-scale study to systematically examine the distribution of reference data in Overton. It contributes a foundational understanding of this emerging source for policy metrics, highlighting both its potential applications and limitations, and underlining the importance of addressing current coverage imbalances.

Explore related subjects

cs.DL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Biegzat Murat, Zhichao Fang, Ed Noyons, Rodrigo Costas. 2026-01-05. Evidence for studying interactions between science and policy: An exploration of scholarly and policy references in Overton-indexed policy documents. https://doi.org/10.2478/jdis-2025-0054

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

From Code Archival to Knowledge Graph: Bridging Software Heritage, COAR Notify and Wikidata

Software is a first-class scientific object, yet validated links between source code and the scholarly record remain largely absent from the Linked Open Data (LOD) cloud, isolating archived artefacts from semantic discovery. This paper presents an end-to-end reconciliation pipeline that harvests, validates, and models publication-to-repository pairs from sources where the link between a paper and its source code is explicit and editorially verified: the software-centric journals JOSS, SoftwareX, and IPOL, together with the reproducibility reports of the SIGMOD Availability and Reproducibility Initiative (ARI). This yields a curated corpus of 4,397 $\langle$DOI, repository-URL$\rangle$ pairs. We design two distinct application profiles grounded in Wikidata classes (one for scholarly articles, one for software instances) aligned with the schema.org and CodeMeta vocabularies. This architectural separation enables rule-based reconciliation at two granularities: lightweight, inline publication references or standalone, first-class Wikidata software nodes equipped with SWHIDs, Software Heritage's content-addressed identifiers. A read-only lookup against Wikidata shows that only 82 of the harvested repositories were already modelled there; human-reviewed batches have since created 4{,}182 new software items cross-linked to their articles. We further show that payloads of the emerging COAR Notify protocol, an external effort we do not develop, map natively onto our input format, so the same backend could later serve a live enrichment stream. Our core contribution is a pair of application profiles that turn Wikidata into a connector between the scholarly record and archived source code; we openly release all code, application profiles, and harvested datasets.

cs.DL↗
arXiv2026

greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI

Conferences, journals, funders, schools, and universities are struggling with a surge of potentially AI-generated submissions from ostensibly human authors, who may not have exercised sufficient human oversight for their manuscripts. In turn, institutions evaluating submissions can no longer reliably credit expertise based solely on authors' names on submitted work. To address this problem, we propose greCAPTCHA, a proctored assessment approach that measures authors' understanding of research manuscripts via the construct of capacity to verify, which we define as the knowledge and reasoning required to critically assess the contents underlying one's contributions to a manuscript. greCAPTCHA generates questions assessing multiple levels of understanding and provides an evaluative report based on authors' responses. Using a prototype implementation, we conduct a user study and semi-structured interviews with $31$ researchers to evaluate greCAPTCHA. Its automated scores predict which papers were or were not authored by study participants with an AUC of $0.90$. Participants reported positive overall experiences with the system and remarked on the appropriate construct validity for author understanding, while also suggesting important changes to be made before deployment. Our results provide initial evidence that greCAPTCHA can assess manuscript-specific understanding under proctored conditions.

cs.DL↗
arXiv2026

Geospatial Metadata Improves Discoverability by Connecting Datasets Across Scientific Disciplines

Research data repositories are essential infrastructure for scientific inquiry and for ensuring that datasets follow FAIR (Findable, Accessible, Interoperable, and Reusable) principles. However, repository reuse depends on the quality and completeness of geospatial and thematic metadata, which researchers generally provide voluntarily. Given limited curation resources, it is unsurprising that even Harvard Dataverse, the world's largest general-purpose research repository, contains many incomplete metadata records. Missing fields represent lost information and reduce interoperability. We find that datasets with more missing metadata receive fewer downstream citations and have fewer resolvable connections to other datasets. The implications are particularly important for geospatial datasets: only 0.3% of research datasets include a bounding box, and most represent archival points rather than complete geographic shapes. Our analysis shows that geospatial metadata helps connect concepts across disciplines. After embedding Harvard Dataverse datasets in a metadata knowledge graph, we find that datasets are twice as likely to connect across scientific disciplines through shared geospatial metadata as through keywords. This suggests that geographic metadata is a more reliable basis for cross-disciplinary interoperability than keyword vocabularies, which often remain discipline-specific. We train and fine-tune a small language model using datasets from Harvard Dataverse. Through geospatial metadata enrichment, we increase the share of datasets from different disciplines connected through metadata elements from 58.5% to 63.2%.

cs.DL↗