How much does the map of science depend on the embedding model?

Alejandro Treny Ortega

Home/Archive/arXiv

arXiv · 2610.16639

How much does the map of science depend on the embedding model?

Alejandro Treny Ortega

View original source ↗Read source document ↗Export metadata ↓

Abstract

Science maps describe relationships between papers. Text embedding models build these relationships by turning a paper's text into a list of numbers that gives it a position. Would another model give the same picture? I compare ten models using titles and abstracts of 500,000 publications from the OpenAlex bibliographic catalogue, published in 2000-2024 across 26 disciplines, including Medicine and Physics. Representing each discipline by its papers' average position produces similar arrangements across models, including when other papers are selected. Yet within a discipline and period, two models share, on average, only eight of the 25 papers closest to a given paper. Supplying titles alone also changes much of that list, with effects that vary between models. Models can also disagree about which discipline's papers are more spread out. One measure of spread compares angles between lines drawn from a common point to each paper's position. Moving that point can reverse which discipline appears more spread out, even when selecting other papers leaves the answer unchanged. The broad arrangement can remain similar while answers about nearby papers and differences between disciplines depend on the model, the text it receives and how its output is processed.

Explore related subjects

cs.DL

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Alejandro Treny Ortega. 2026-10-12. How much does the map of science depend on the embedding model?. https://arxiv.org/abs/2610.16639

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Quantifying research uniqueness with Unbalanced Optimal Transport: Case studies of ten leading universities in neuroscience

Research evaluation has developed around quantity, citation impact and, more recently, the accumulation of research excellence, but these dimensions remain largely dependent on citations, which describe how research is received rather than what it contains. We propose uniqueness as a complementary dimension and measure it with the Research Uniqueness Index (RUI), which asks how far a university's research sits from what is typical in its field, using no citations but the content of the research itself. Each researcher is represented by an embedding of their abstracts, and an unbalanced version of Optimal Transport returns the share of that work which cannot be matched into a random sample of same-field researchers, the standard population; that unmatched share is the score. Because every researcher is scored, a university is a distribution rather than one number, and we treat that distribution as the indicator: it shows whether uniqueness is spread across an institution or concentrated in a few groups. Quantiles summarise it and, being quantiles, do not reward size. Since uniqueness is defined only relative to a field, we demonstrate the index within one, neuroscience (Scopus, 2021-2025; 9,861 reference researchers). All ten universities studied sit well above the standard (RUI_median 0.40-0.53 versus 0.27) and five specialised institutes higher still (0.57-0.73), the most unique work often confined to a narrow niche that the same scores name. RUI is uncorrelated with citation impact, and a controlled test confirms that it tracks displacement from the field rather than topical concentration; results are robust to an alternative embedding and replicate in computer science and economics. The index is not intended as a league table: the gap above the standard and the researcher-level niche structure are stable, whereas the order among adjacent universities is not.

cs.DL↗

arXiv2026

Interpreting Paper-to-Patent Citations in AI-Related PMC Articles

Paper-to-patent citations connect scientific and technological documents, but their interpretability depends on how patents are used in the citing text. This study examines citation contexts and paper-level outcome associations in AI-related articles in PubMed Central (PMC). Screening 189,073 articles identified 746 with explicitly tagged patent references (0.395%), including 713 with 1,544 identifiable in-text patent-citation instances. Three large language models classified these instances by represented content, motivation, textual position, and evaluative stance. Regression models examined associations among these dimensions, and propensity-score-matched regressions compared patent-citing papers with XML-screened controls. Background-oriented citations were predominantly neutral and introductory, whereas adoption-oriented citations were more closely associated with methodological positions and supportive evaluations. These patterns suggest heuristic configurations of contextualization, operationalization, and problematization, although classification overlap and exclusion of rare categories limit their interpretation. In matched samples, the patent-citing indicator was associated with lower scholarly-citation measures and more citations received from patents. These associations are observational, not causal. Together, these findings show that paper-to-patent citations encode heterogeneous forms of engagement with patented technology and should be interpreted as selective bibliographic signals rather than direct indicators of technology-to-science knowledge transfer or research value.

cs.DL↗

arXiv2026

Equitable Intelligence in Practice: A Fairness Audit of Open Large Language Models for Library Reference Services

As libraries explore large language models (LLMs) as a scalable layer for reference services, a core fairness question follows: can LLM-based services support all patrons fairly, regardless of demographic identity? LLMs could broaden access to information assistance, but they may also reproduce societal biases embedded in their training data and undermine libraries' commitments to impartial service. In this chapter, we apply an evaluation protocol that combines diagnostic classification, which detects systematic differences between responses, with linguistic analysis, which helps interpret them. Across three widely used open models (Llama-3.1 8B, Gemma-2 9B, and Ministral 8B), we find no compelling evidence of systematic differentiation by race/ethnicity, and only minor evidence of sex-linked differentiation in one model and one setting. We discuss implications for equitable AI adoption in libraries and the importance of ongoing monitoring in aligning LLM-based services with social justice goals.

cs.DL↗