Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Digital Libraries

cs.DL Watch Digital Libraries

Wednesday 30 September

Digital Libraries · Cross-listed from cs.CL

Drift Inspector: Exploring and Measuring Scientific Drift with Atomic Contribution Claims

Vsevolod Karimov, Stepan Ostarkov, Anastasia Poroshina et al. (+2 more)

Scientific abstracts mix contributions with background, motivation, and meta-language, so tools that read them as-is cannot separate what a field produces from what it discusses. We present Drift Inspector, an open-source system for measuring and exploring how a research field changes over time at the level of Atomic Contribution Claims (ACCs): decontextualized, contribution-bearing propositions an LLM extracts from each abstract before analysis. The system clusters these claims across years into an interactive map where every trend traces back to the claims and papers behind it. Applied to six years of EMNLP, it shows the field shifting away from classic NLP tasks toward LLM-era capabilities such as reasoning and multimodality – a movement that keyword or whole-abstract counts blur. The released data extend beyond EMNLP: the same pipeline has processed the full ACL Anthology (346k claims, 80k abstracts, 423 venues). Extraction is human-validated and clustering checked against an external manually constructed taxonomy.
Read the full abstract

All authors Vsevolod Karimov, Stepan Ostarkov, Anastasia Poroshina, Anatoly Frolov, Alexander Panchenko

Digital Libraries

Augmenting Italian Cultural Heritage with Virtual and Digital Technologies: the final outcomes of Project CHANGES' Spoke 4

Silvio Peroni, Gianluca Genovese, Roberto Balzani et al. (+107 more)

CHANGES (Cultural Heritage Innovation for Next-Gen Sustainable Society) was a project coordinated by the CHANGES Foundation, bringing together complementary disciplines and expertise across the entire cultural heritage lifecycle. This article focuses on the project's research area dedicated to applying virtual technologies to museums and art collections. This research enabled us to experiment with different types of museums and art collections across Italy, designing case studies and best practices that institutions and similar contexts can adapt and reuse. The meta-analysis of the case studies showed that museum types play a decisive role in shaping not only technological choices but also organisational strategies, training models, and pathways to sustainability. This work also allowed us to identify technological and organisational gaps that go beyond single cases and reflect shared challenges across museum contexts. Finally, we documented the success of the engagement, exploitation, and dissemination strategy proposed in the project.
Read the full abstract

All authors Silvio Peroni, Gianluca Genovese, Roberto Balzani, Silvano Montaldo, Sofia Pescarin, Cristina Caterina Amitrano, Luisa Ammirati, Riccardo Antonino, Giorgio Bacci, Davide Bagnaresi, Sebastian Barzaghi, Giuliana Benvenuti, Marco Bertini, Luca Bevilacqua, Marco Biffi, Elisa Bonacini, Federica Bonifazi, Alice Bordignon, Davide Borra, Andrea Bottino, Ivana Bruno, Daniele Caccavale, Marcello Calogero, Giuseppe Capotorto, Andrea Carpentieri, Veronica Casadei, Cristina Casero, Maurizio Cattani, Roberto Cavallo Perin, Vito Leonardo Chiechi, Fabio Ciotti, Luca Cipriani, Simona Colitti, Federica Collina, Michela Contessi, Lorenzo Copelli, Michele Corriero, Marianna Cuomo, Francesca D'Angelo, Rossana Damiano, Marilena Daquino, Stefano De Martino, Kevin De Vecchis, Davide Domenici, Nicole Dore, Francesca Fabbri, Bruno Fanini, Filippo Fantini, Fabrizio Federici, Daniele Ferdani, Enrico Ferraris, Giulia Fiorini, Anna Forte, Rocco Furferi, Riccardo Gagliarducci, Vincenzo Gattulli, Manuel Gentile, Federica Giacomini, Valentina Alena Girelli, Martina Grinello, Bianca Gualandi, Jasmine Habsy, Stephan Hassam, Ivan Heibi, Alessandro Iannucci, Alessandra Inglese, Marco Lamorte, Laura Travaglini, Federica Maietti, Rachele Manganelli Del Fà, Gianluigi Mangiapane, Nicola Mariniello, Giulia Marsili, Arcangelo Massari, Marcello Massidda, Michele Mellara, Enrico Mensa, Mattia Modugno, Roberto Montanari, Antonio Monteleone, Arianna Moretti, Elena Musiani, Francesca Nicolais, Sara Obbiso, Carmelo Occhipinti, Francesca Paruzzo, Cecilia Pennacini, Davide Perinciolo, Mario Petrella, Margherita Elena Pomero, Roberta Presta, Maria Felicia Rega, Giulia Renda, Diego Ronchi, Alessandro Rossi, Nicola Santopuoli, Chiara Scardozzi, Nicolò Serafino, Ilaria Serati, Ulderico Sicilia, Stefano Sorrentino, Daniele Spedicati, Cristina Stalteri, Riccardo Stracuzzi, Mattia Sullini, Ursula Thun Hohenstein, Maria Alessandra Tini, Federica Veratelli, Riccardo Zanzi, Guido Nicolas Zingari

Digital Libraries · Cross-listed from cs.CL

TRACE: Target-Aware Retrieval, Attributed Evidence, and Contract-Constrained Extraction for LitTraceQA

Sachin Gupta and Divya Godara

Finding a relevant paper is not the same as producing a verifiable answer from it. LitTraceQA requires canonical paper identifiers, exact evidence at the page or object level, and typed answers that match the evaluator. We call the separation between source access and scorer-visible correctness the grounding contract gap. TRACE - Target-Aware Retrieval, Attributed Evidence, and Contract-Constrained Extraction - addresses this gap with target-grouped retrieval, independent typed evidence localization, multimodal table extraction, schema-driven table construction, and fail-closed validation. It indexes 27,487 papers through passage, object, alias, citation, and dense representations while retaining the question target behind each signal. For tables, TRACE predicts the observation unit before extracting values and assembles rows with evaluator-compatible key normalization. Our audited selected clean-track artifact scores 0.760613 on the official 71-question test set, including 0.9728 paper F1, 0.6847 evidence F1, 0.9800 multiple-choice accuracy, 0.5423 table-row F1, and 0.3508 macro cell accuracy. On 11 public-development table records, a clean baseline and coordinate-aware visual fill obtain row F1 of 0.291 and 0.411, respectively; this diagnostic comparison includes fallback outputs and is not an official-test claim. Remaining errors chiefly concern locator, observation-unit, row-key, and source-value identity.
Read the full abstract

Tuesday 29 September

Digital Libraries · Cross-listed from cs.CL

Doc2LoRA Provides Decodable Representations of Scientific Ideas

Chand Sahil Mansuri, Joel Zachariah and Sadamori Kojaku

Representing scientific papers as points in a space lets us search for similar papers and inquire about how fields relate to one another and drive innovation. Beyond search, the vector space of papers invites generation: mixing papers through simple vector operations creates new points, mirroring combinatorial novelty, the recombination of existing ideas into new ones. However, a mixed point often represents an idea no paper has yet realized, with no papers nearby to identify the idea. We propose representing each paper by a LoRA adapter generated by the Doc-to-LoRA hypernetwork. Every point in the space, including mixtures, thus represents a large language model (LLM) open to questions and instructions in natural language. On papers from the American Physical Society (APS), we instruct the LLM at the average of each subfield to name the field in a few words and obtain labels closer to the official names than the labels of five baselines, as judged by word overlap and a panel of five LLM judges. We also ask the LLMs at points between two APS papers to write an abstract and obtain descriptions shifting from one paper to the other in step with the mixing weight. While Doc-to-LoRA is trained for generation, a small invertible transform makes the embeddings competitive for search, on par with SPECTER2 and EmbeddingGemma and close to SBERT. Because the transform is invertible, every point in the transformed space still maps back to an LLM. The embeddings thus serve both search and generation, enabling researchers to question the idea at any point in the space as a starting point for generating new ideas.
Read the full abstract

Digital Libraries

Which reported inputs govern molecular docking reproducibility? A benchmark from reporting audit to independent re-execution

Giap Duc Ha

Computational docking results can be re-executed only when the molecular system and protocol are specified, yet reporting checklists do not quantify how strongly individual method fields affect the reported score. We combined a span-verified audit of 50 open-access papers with controlled one-factor perturbations, a Vinardo scoring-function check and a Vina cross-docking extension covering 12 targets in 7 protein families, and re-execution of 37 published claims on local and independent cloud infrastructure. Search-related fields were sparsely reported, but perturbing box centre, box size, exhaustiveness or random seed produced median absolute score changes of no more than 0.08 kcal mol$^{-1}$; ligand protonation and identity produced median changes of 0.19 and 0.29 kcal mol$^{-1}$, respectively, whereas receptor structure produced the largest change (1.02 kcal mol$^{-1}$; n = 85; P < 0.001), and 22% of 116 unique reported ligand strings remained unresolved or ambiguous after deterministic name normalization. These measurements yielded empirical field weights for the NewScience Evidence score and identified exact receptor structure, machine-resolvable ligand identity and protonation state as primary reporting elements; 29 of 37 local and 27 of 37 cloud re-executions were within 2.0 kcal mol$^{-1}$ of the reported score, but these descriptive rates do not estimate experimental replication or a calibrated probability of reproduction.
Read the full abstract

Monday 28 September

Digital Libraries · Cross-listed from cs.SE

The Uneven Decline of Collective Knowledge Production: Evidence from Stack Overflow After Generative AI

Myokyung Han, Taegyoon Kim, Jinhyuk Yun et al. (+1 more)

Generative AI (Gen AI) is reshaping how individuals learn and work, but its consequences for collective knowledge, the shared body of knowledge that online communities produce together, remain poorly understood. Prior work has documented an aggregate decline in participation on knowledge-sharing platforms, but it remains unclear which specific kinds of knowledge are being lost first. We study this question using Stack Overflow, one of the largest online communities for software engineering, treating the release of ChatGPT-3.5 as a natural shock. Analyzing over two million questions posted between 2020 and 2025, we track how two dimensions of collective knowledge, difficulty and data availability, change following Gen AI's release. Using diverse methods and robust checks, we find consistent patterns. Easy questions decline sharply while difficult questions become more common, a pattern corroborated by rising code complexity. Data-rich topics and tags lose share of questions, while data-scarce ones gain ground. The two dimensions also interact: the decline in easy questions is concentrated specifically within data-rich domains, while difficult questions increase regardless of data availability. This pattern extends beyond Python across programming languages, with more prevalent languages showing sharper shifts. Together, our findings reveal that Gen AI's impact on collective knowledge is uneven, eroding easy, accessible knowledge first while more complex, less common knowledge persists.
Read the full abstract

All authors Myokyung Han, Taegyoon Kim, Jinhyuk Yun, Lanu Kim

Sunday 27 September

Digital Libraries

Paying for Publication with Peer Review Design and Simulation of a Fee-Free, Credit-Based Open Access Journal for Information Technology

Murat Ozer

Open access publishing has moved much of the cost of scholarly communication from readers to authors, while the labor of peer review remains unpaid and unevenly shared. This paper describes the design of Quorum: An Open Journal of Information Technology (jquorum.org), a journal that charges neither readers nor authors and uses peer review as its currency instead. Every first complete review a scholar writes on a manuscript earns one credit, and one credit pays for one submission. Each manuscript receives three double-blind reviews from reviewers matched automatically by expertise, and editorial decisions follow published rules rather than editor discretion. The paper documents the credit ledger, a bootstrapping mechanism called the genesis block, a reviewer search that widens after seven days and opens the paper to volunteers, the decision rules, and the integrity safeguards. An agent-based simulation, run for two simulated years with 40 replications per scenario, evaluates these choices. Without the genesis block, no paper was submitted in any replication. With it, a community growing from 40 to about 330 members submitted about 603 papers, and papers waited a median of 7 days for a full review panel. The day-7 escalation had a modest effect in a large community but a decisive one in a small community of about 60 members, where removing it left 48.8% of papers without a panel after 60 days, compared with 4.6% with it. Unspent credits also accumulated to about 3.7 per member after two years, which points to a policy choice the journal will face as it grows.
Read the full abstract

Saturday 26 September

Digital Libraries

More of the same? Are scientific papers losing originality?

Ilan Doron-Arad and Elchanan Mossel

Scientific output is growing rapidly, but it is unclear whether the expanding literature remains original or is increasingly repeating itself. Originality has many dimensions. One dimension can now be measured directly: how textually distinct a paper is from the work that came before it. Using semantic language models, we screened more than 26,000 full-text papers across eight fields in the CORE database (2015–2025) for passages resembling recent literature. We compared every paper against a fixed-size sample of earlier work, so increases in resemblance are not a by-product of literature growth. We find that the share of a paper's text resembling recent work was stable before 2021, then rose sharply, more than tripling by 2025. The increase came not from a few papers borrowing more heavily but from resemblance spreading across the literature: more papers now contain passages that echo recent work, and almost none of this growth is explained by cited work. We also found an analogous increase in passages expressing the same scientific idea. The pattern held across robustness checks and replicated in a second, independent database. Along this one measurable axis of originality, scientific writing has become less distinct from the work preceding it.
Read the full abstract

Friday 25 September

Digital Libraries · Cross-listed from cs.IR

AgentRecommender: LLM Agents Enable Customizable Recommender Systems on the User Side

Ryoma Sato

Recommender systems have traditionally been developed for platforms. However, this has given rise to many phenomena that may be advantageous for platform lock-in but are a nuisance to users, such as clickbait, filter bubbles, and the spread of fake news. Recently, user-side recommender systems have been proposed as a new paradigm for solving this problem. If users deploy their own recommender systems, they are no longer at the mercy of the platform's interests. However, building a user-side recommender system is not trivial; in particular, customizing one for oneself requires additional data. We propose AgentRecommender, a method that leverages the investigation capability and internal knowledge of LLM agents to flexibly build user-side recommender systems without additional data. AgentRecommender allows users to easily create recommender systems tailored to their own preferences.
Read the full abstract

Thursday 24 September

Digital Libraries

A framework for linking literature-based knowledge integration and infrastructure-supported knowledge integration: Opportunities and challenges from a case study

Mahlet Degefu Awoke, Hadi Ghaemi, Lauren Synder et al. (+2 more)

Integrating knowledge across disciplines is central to sustainability research, yet most evidence-synthesis methods rely on findings as reported in publications, limiting verification and reuse of underlying data and workflows. We develop a conceptual framework linking literature-based and infrastructure-supported knowledge integration, using a systematic review case study to examine when integration can extend beyond reported findings. We reviewed 37 studies on climate change, violent conflict, and household food security. Literature-based synthesis enabled integration across all included studies, whereas access to reusable outputs was limited: over half provided no data availability statement, 27% reported availability upon request, but reusable data and workflows were available for only 8%. To explore infrastructure-supported integration, we used the TIB Knowledge Loom to represent studies with accessible data and code as machine-readable outputs, and produced a knowledge gap map (KGM) from manually extracted and Loom-derived data, comparing manual and infrastructure-supported synthesis. Where outputs were reusable, synthesis could be produced directly from data and workflows rather than from publications. These findings show that literature-based synthesis can be complemented by infrastructure-supported integration where outputs are accessible and usable, and that advancing knowledge integration depends not only on infrastructures but on making data, code, and workflows accessible, executable, and reusable.
Read the full abstract

All authors Mahlet Degefu Awoke, Hadi Ghaemi, Lauren Synder, Markus Stocker, Tilman Brück

Wednesday 23 September

Digital Libraries

Residential Electricity Consumption Dataset for Sri Lanka (RECON-SL)

Chanuka Algama, Merl Chandana, Isuruni Fernando et al. (+6 more)

This article describes the Residential Electricity Consumption Dataset for Sri Lanka (RECON-SL), a multi-source resource that integrates utility records, high-frequency smart meter data, and household surveys from the service area of Lanka Electricity Company (LECO) in Sri Lanka. The dataset captures electricity use for 4,063 households, collected over the period October 2022 to January 2025, with coverage at monthly, 6-hour, and 15-minute intervals. Monthly billing-cycle data provide complete coverage across all households, while smart meter data are available for a subsample of 1,438 households, offering both fine-grained 15-minute readings and coarser 6-hour records. Three survey waves complement these consumption measures, collecting information on demographics, housing, appliance ownership, and perceptions of energy security. Across its components, the dataset includes more than 50 million meter readings and detailed survey responses from over 11,000 household interviews. As the first dataset of its kind in Sri Lanka, and among the few worldwide to link smart meter records with longitudinal survey data at this scale, it provides a unique resource for energy, machine learning and policy research, with known gaps in smart meter coverage documented for users. The dataset can support diverse applications, including electricity load forecasting, socio-economic and behavioral energy research, distribution planning, and policy analysis in energy affordability and equity. Access is provided through an open repository, with accompanying documentation and scripts to facilitate reuse.
Read the full abstract

All authors Chanuka Algama, Merl Chandana, Isuruni Fernando, Dinithi Dissanayake, Amanda Ariyaratne, Nipuni Habaragamuwa, Pasindu Ranage, Jesús Ramos, Kasun Amarasinghe

Digital Libraries

PaperAtlas: an automatically constructed atlas of computational methods and software from 6.4 million open-access articles

Neeti Shah, Anagha Vippagunta and Yash Bhargava

Computational methods and software are dispersed across a literature that is increasingly difficult to review manually, while curated registries capture only a subset of available resources. We present PaperAtlas, an automatically constructed atlas derived from the PubMed Central open-access corpus. Of 6,446,741 abstracts screened, 1,074,191 were classified as computational, with schema-valid records extracted for 1,074,140. Among these, 267,893 papers describing an algorithm, software package or web server formed 1,438 clusters, with 1,000 retained after restriction to parent-level biomedical categories. In 296 clusters linked to at least five bio.tools entries, the mean concentration of the most frequent EDAM topic was 61.7%, compared with 26.4% under permutation. Of 31,180 distinct software and web-server names, 61.0% lacked a strict match in bio.tools, PyPI, CRAN, Bioconductor or Bioconda. Among 12,207 bio.tools entries with defining papers in the corpus, 56.6% were recovered end to end, increasing to 83.6% among the 8,265 entries whose defining publication entered the atlas. PaperAtlas is fully open source and available as a web server at bhargavaresearch.org/paperatlas.
Read the full abstract

Digital Libraries · Cross-listed from cs.CR

Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG

Zhonghao Sun, Zhiliang Tian, Xinyue Fang et al. (+4 more)

Deploying large language models (LLMs) on sensitive data via Retrieval-Augmented Generation (RAG) introduces severe privacy risks. Recent studies apply Differential Privacy (DP) to LLMs with RAG for formal privacy guarantees. However, existing DP-RAG frameworks rapidly exhaust the privacy budget. Although recent efforts attempt to save the budget by narrowing the retrieval scope or sparsifying private generation, these methods themselves cumulatively consume the budget, whereas they could actually rely merely on public information or at a negligible one-time privacy cost. This mismatch fails to align budget expenditure with the model's actual reliance on private data, causing substantial waste on operations that require no private access. To address this, we propose SparsePay-RAG, adopting "only pay what you must spend" as its core principle. Using public information as a zero-privacy prior, it charges the privacy budget only for the private increment. Specifically, SparsePay-RAG narrows the retrieval scope via public topic-guided clustering, adaptively controls private access frequency without privacy cost through isotonic cross-layer trajectory fitting, and compresses per-access budget via DP contrastive decoding. Under strong privacy constraints, experiments show SparsePay-RAG achieves superior privacy-utility trade-offs over baselines.
Read the full abstract

All authors Zhonghao Sun, Zhiliang Tian, Xinyue Fang, Shuo Ma, Juhua Zhang, Yiping Song, Dongsheng Li

Tuesday 22 September

Digital Libraries · Cross-listed from cs.CL

Layout-Guided Masking for GROBID: Lightweight Structural Gains in Large-Scale Scientific PDF Ingestion

Luca Foppiano, Sana Khamassi and Vipul Gupta

Transforming scholarly PDFs into machine-readable fulltext remains a bottleneck for large-scale information systems. Recent vision-based parsers improve accuracy, but need GPUs and may introduce noise into the extracted text. GROBID, a modular font-stream parser running on CPU, is the de-facto standard for structuring scientific articles and underpins several of the largest open scholarly corpora. We pair it with a lightweight CPU detector localising figure, table, and paratext (header, footer, page number) regions, encoded as typed-area masks whose tokens are routed to GROBID's specialised models or discarded. On two PMC corpora, Bioinformatics (1,926 articles) and Materials Science (2,595), scored against JATS with a section-aware structural protocol, our extension improves over plain GROBID on most metrics (NS $+0.025$/$+0.013$; $+0.086$ paragraph recall on Materials Science, $d_z{=}1.08$), and caption-linked figure recovery improves on both corpora. On the external Table-BRGM benchmark, table detection recovers F1 $0.16 \to 0.94$ and table structure follows (GriTS-Top $0.27 \to 0.78$, below the strongest GPU system). On body text, against four vision-based systems (Docling, MinerU, olmOCR, dots.ocr), it has the best paragraph precision on both corpora, the best section detection on Materials Science, and a character error rate within 0.004 of the best GPU parser. End-to-end on CPU, it costs $2.7$–$3.2\times$ less than the cheapest GPU system (Docling) and $10$–$14\times$ less than generative parsers.
Read the full abstract

Digital Libraries

GitScholar: A Dataset for Predicting AI Research Impact from GitHub Engagement

Emilien Guandalino, Lorenz K. Müller, Beatrice Alessandra Motetti et al. (+2 more)

With the rapid pace of AI research and the hundreds of daily new publications, staying up-to-date with the latest developments has become increasingly difficult. For researchers, quickly identifying impactful work is essential, yet manually reviewing each new publication is impractical. Automated impact prediction methods help address this challenge, usually by combining various information sources available, such as a paper's content or citation history. In this work, we propose using GitHub engagement as an additional source and demonstrate that it provides both a timely and accurate signal. To this end, we introduce GitScholar, a novel dataset that links GitHub activity from 444,000 repositories to over 558,000 AI arXiv papers. Our experiments show that GitHub reactions improve early prediction precision by up to 12% over a strong academic baseline. Additionally, we find that GitHub signal offers near-complete coverage of high-impact AI papers, and consistently correlates with future academic success. GitScholar is publicly available at https://huggingface.co/datasets/huawei-csl/GitScholar.
Read the full abstract

All authors Emilien Guandalino, Lorenz K. Müller, Beatrice Alessandra Motetti, Konstantin Berestizshevsky, Lukas Cavigelli

Monday 21 September

Digital Libraries

Open Science, Closed Models: How Funding Shapes AI in Science

Ana Trišović and Janakan Sivaloganathan

How funding shapes AI engagement in science is poorly understood despite its structural importance. We analyze 104,226 papers (2018-2025), linking funding acknowledgments to how each paper engages with foundation models: whether it extends a model (fine-tunes or builds on it), uses one without modification, or references models only peripherally. We report three findings. First, both private and public funding are associated with greater AI engagement. Public funding is associated with more off-the-shelf and open-weight use, while private funding is associated with greater model extension. Industry cloud credits are associated with less open-weight use, consistent with greater reliance on closed models. Second, industry involvement is associated with model extension, suggesting that corporate resources reach science through coauthorship, though this channel is concentrated in Computer Science and Global North collaborations. Third, the gap across research systems is not one of AI adoption but of control. Off-the-shelf use is rising across fields and country groups, and the least publicly funded group, the Global South, now has the highest share of API-only engagement (0.4% to 10.0% of engaged papers). China shows the opposite profile: the highest public-funding share and lowest API-only engagement, with extension remaining stable as it declines across the Global North. Standard capacity indicators show convergence in AI adoption even as dependence on proprietary infrastructure diverges.
Read the full abstract

Digital Libraries

Scoring Grant Applications with Large Language Models

Mike Thelwall, David Pride, Francesco Osborne et al. (+4 more)

Purpose: Assessing grant applications is time-consuming and difficult, adding to the overall burden of academic peer review. Whilst funders are exploring whether AI can help, there is no published research into the accuracy of Large Language Models (LLMs) for scoring contemporary grants. Design/methodology/approach: This study investigates whether six open-weight LLMs (Gemma 3 1B/4B/12B/27B, DeepSeek R1 32B, Qwen 3 32B) can give useful scores for 2267 recent UK Economic and Social Research Council (ESRC), and Engineering and Physical Sciences Research Council (EPSRC) grant applications, comparing them with scores from the original reviewers and funding panel members. Findings: Although the LLM scores are individually inaccurate, when averaged and converted to ranks they correlate positively with expert average scores. The best performing LLM, Gemma 3 27B (10 iterations with varied prompts), had moderate rank correlations with average reviewer scores (mean rho=0.26). Gemma 3 27B's average correlation with individual reviewers was 0.19, which is lower than the inter-reviewer mean correlation of 0.24, suggesting that it scores are slightly weaker than individual reviewer scores. Gemma 3 27B had weak rank correlations with average panel member scores (mean rho=0.17), with lower average correlations with individual panellists (mean rho=0.14), which is substantially lower than the inter-panellist correlation (mean rho=0.38). Whilst the correlations seem too weak to replace expert review at the final panel stage, LLM scores might help with the initial reviewing state, such as by helping identify the weakest proposals for fast-track desk rejections, to replace one human reviewer, or for triangulation to check for bias.
Read the full abstract

All authors Mike Thelwall, David Pride, Francesco Osborne, Angelo Salatino, Petr Knoth, Gloria Iyawa, Oluwaremilekun Yusuf

Digital Libraries

The illusion of neutrality in metric-based research evaluation

Fengyuan Liu, Hazem Ibrahim, Yasir Zaki et al. (+1 more)

Venue prestige and citation counts are two widely used, albeit imperfect, signals of research quality. When the two signals conflict, evaluators must decide how much weight to assign each. Yet, it remains unknown how researchers across disciplines trade off these two signals when evaluating research outcome. To fill this gap, we surveyed 869 researchers using paired choices between hypothetical departmental hiring rules that assigned different weights to venue prestige and citation counts, asking which would produce better science. Among 795 respondents whose choices were largely internally consistent, choices placed nearly equal aggregate weight on the two signals. This apparent balance concealed substantial individual heterogeneity: nearly two in five respondents occupied the most venue-heavy or citation-heavy intervals. Moreover, respondents on average chose more citation-heavy rules than they believed their departments used in hiring. By revealing the subjective judgments that arise when research indicators conflict, our findings reinforce calls for greater caution when quantitative indicators are used to evaluate research.
Read the full abstract

All authors Fengyuan Liu, Hazem Ibrahim, Yasir Zaki, Talal Rahwan

Sunday 20 September

Digital Libraries

Reading the Data Back: Enriching Variable-Level Metadata for Model-Data Consistency Checks

Eryk Kulikowski

Purpose: Research data repositories often lack variable-level metadata. Model-choice screening, checking whether a reported model suits the values of its outcome variable, also requires summaries of those values and links to the analyses that use them. We ask how repositories can represent and acquire them as metadata with evidence and review histories. Methods: We propose a metadata application profile compatible with the Data Documentation Initiative Cross-Domain Integration (DDI-CDI) model, linking versioned variables and empirical profiles to reported analyses, estimators, and analysis roles. Extraction provenance and review decisions are recorded separately. We evaluate streaming data profiling, deterministic extraction from Stata and R code, and language-model extraction of analysis records from papers. Results: A worked export of one replication package illustrates the profile: it passes shape and interchange checks and answers four queries, one of which returns the analyses that raise no alert. Screening 4,868 replication datasets from six political-science journals links outcomes to profiled columns in 1,440 deposits and flags 965, including 14 of the 21 deposits with a hand-verified linear model on a count, proportion, binary, or ordinal outcome. AI-assisted adjudication yields 55% precision on a near-balanced sample of 119 count and proportion candidates; screening rules were developed on the same corpus. Conclusion: The profile makes analysis-variable relationships queryable while preserving evidence and review history; screening yields review candidates with context. Code, metadata, and measurements are released.
Read the full abstract

Friday 18 September

Digital Libraries

From Code Archival to Knowledge Graph: Bridging Software Heritage, COAR Notify and Wikidata

Camillo Carlo Pellizzari di San Girolamo and Francesco Tosoni

Software is a first-class scientific object, yet validated links between source code and the scholarly record remain largely absent from the Linked Open Data (LOD) cloud, isolating archived artefacts from semantic discovery. This paper presents an end-to-end reconciliation pipeline that harvests, validates, and models publication-to-repository pairs from sources where the link between a paper and its source code is explicit and editorially verified: the software-centric journals JOSS, SoftwareX, and IPOL, together with the reproducibility reports of the SIGMOD Availability and Reproducibility Initiative (ARI). This yields a curated corpus of 4,397 $\langle$DOI, repository-URL$\rangle$ pairs. We design two distinct application profiles grounded in Wikidata classes (one for scholarly articles, one for software instances) aligned with the schema.org and CodeMeta vocabularies. This architectural separation enables rule-based reconciliation at two granularities: lightweight, inline publication references or standalone, first-class Wikidata software nodes equipped with SWHIDs, Software Heritage's content-addressed identifiers. A read-only lookup against Wikidata shows that only 82 of the harvested repositories were already modelled there; human-reviewed batches have since created 4{,}182 new software items cross-linked to their articles. We further show that payloads of the emerging COAR Notify protocol, an external effort we do not develop, map natively onto our input format, so the same backend could later serve a live enrichment stream. Our core contribution is a pair of application profiles that turn Wikidata into a connector between the scholarly record and archived source code; we openly release all code, application profiles, and harvested datasets.
Read the full abstract

Thursday 17 September

Digital Libraries · Cross-listed from cs.CV

Automated Goldsmith's Mark Retrieval in Silverware

Atmik Tiwari, Vincent Christlein, Mark Fichtner et al. (+5 more)

For art historians, goldsmith marks play a critical role in the identification and dating of artifacts. In practice, experts must manually compare a query mark against hundreds of documented examples, a process that is both tedious and highly dependent on specialist knowledge. To address this, we present an AI-assisted retrieval pipeline that combines mark localization with metric-learning fine-tuning across three backbone architectures: an ImageNet-pretrained ResNet-50, a supervised ViT-S/16, and a self-supervised DINOv2 ViT-S/14. We conduct a systematic evaluation of cropping strategies, where we measure the impact of no cropping, manual ground-truth cropping, and learned detection-based cropping, and assess their interaction with each backbone. Our strongest configuration, DINOv2 ViT-S/14 with manual crop and metric-learning fine-tuning, achieves an mAP of 62.63% and a Top-1 accuracy of 73.74%. Our experiments show that self-supervised pretraining and mark localization are the two most impactful factors, with learned cropping recovering the majority of the gain from manual cropping without requiring ground-truth annotations at inference time. To enable reproducibility and adoption in the digital humanities, we release our manually annotated dataset and codebase, and deploy the system via a public web interface.
Read the full abstract

All authors Atmik Tiwari, Vincent Christlein, Mark Fichtner, Freya Gohlke, Birgit Schübel, Theresa Witting, Heike Zech, Mathias Zinnen

Digital Libraries

greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI

Justin Payan, Bálint Gyevnár, Atoosa Kasirzadeh et al. (+1 more)

Conferences, journals, funders, schools, and universities are struggling with a surge of potentially AI-generated submissions from ostensibly human authors, who may not have exercised sufficient human oversight for their manuscripts. In turn, institutions evaluating submissions can no longer reliably credit expertise based solely on authors' names on submitted work. To address this problem, we propose greCAPTCHA, a proctored assessment approach that measures authors' understanding of research manuscripts via the construct of capacity to verify, which we define as the knowledge and reasoning required to critically assess the contents underlying one's contributions to a manuscript. greCAPTCHA generates questions assessing multiple levels of understanding and provides an evaluative report based on authors' responses. Using a prototype implementation, we conduct a user study and semi-structured interviews with $31$ researchers to evaluate greCAPTCHA. Its automated scores predict which papers were or were not authored by study participants with an AUC of $0.90$. Participants reported positive overall experiences with the system and remarked on the appropriate construct validity for author understanding, while also suggesting important changes to be made before deployment. Our results provide initial evidence that greCAPTCHA can assess manuscript-specific understanding under proctored conditions.
Read the full abstract

All authors Justin Payan, Bálint Gyevnár, Atoosa Kasirzadeh, Nihar B. Shah

Digital Libraries · Cross-listed from cs.CL

Improving Cross-Lingual Transfer for Sequential Sentence Classification in Research Papers via Structural Similarity

Kazuhiro Yamauchi and Marie Katsurai

Sequential sentence classification (SSC) is an essential task for structuring scientific publications, and extending SSC research to languages other than English can improve accessibility to scientific knowledge in multilingual digital libraries. Cross-lingual transfer is a promising approach to address the scarcity of training data in non-English languages. Prior work on other natural language processing tasks has shown the benefits of capturing linguistic similarity between source and target languages. However, SSC inherently depends on patterns at the discourse level, such as label sequences and positional regularities, which appear consistently across languages regardless of linguistic differences. To examine the factors that determine transfer success in SSC, we constructed a multilingual SSC dataset covering 13 non-English languages collected from five academic databases. Our cross-lingual transfer experiments, using both encoder-based and generative models, show that linguistic proximity has no consistent predictive power for transfer performance, whereas structural similarity in rhetorical organization shows a weak but consistent positive correlation across models. After controlling for source-language performance, the similarity of label distributions is the most consistent predictor. Building on this finding, we propose a set of three methods that explicitly leverage structural information using generative models. In the in-domain evaluation, the best combination reaches parity with the strongest encoder baselines, and in transfer to languages unseen during training, it outperforms the strongest encoder baseline.
Read the full abstract

Digital Libraries · Cross-listed from cs.CY

Funding the runners-up beats a golden ticket

Haining Wang

A golden ticket allows one reviewer to advance a proposal that the rest of a panel would reject. We assess its score-based rationale by comparing minority-support ranking with other rules for allocating the same number of places. We do so on 10,625 rejected submissions to a computer-science conference between 2017 and 2024, using later citation percentiles as the outcome. Minority-support ranking measured the gap between the two highest scores after adjustment for the panel mean and reviewer count. At a budget of 10% of the year's accepted count, panel-mean ranking selected submissions with a mean citation percentile 7.75 points higher than minority-support ranking (95% bootstrap interval 4.35 to 11.09). The difference was larger in the other sample periods and citation windows examined. At that budget, minority-support ranking was equivalent to a lottery within a margin of plus or minus 5 percentile points. Two further analyses, adjusted highest-score ranking and a matched comparison, found no clear citation advantage from minority support once the panel mean was accounted for. Ranking on score variance also outperformed minority-support ranking in a paired comparison. Estimates cover the 69.8% of eligible rejections whose later citations could be verified. These comparisons identify panel-mean ranking as a benchmark for evaluating whether golden tickets select work that would otherwise be overlooked.
Read the full abstract

Wednesday 16 September

Digital Libraries

Shifting Research Funding Priorities under Geopolitical Pressure: Evidence from Estonia

Yunfeng Gao and Yang Ding

This study examines how the 2014 Donbas-war breakpoint was associated with changes in the semantic composition of research funding recorded in Estonia, a geopolitically exposed country outside the belligerent states. It combines 17,952 research-funding records from the Estonian Research Information System (ETIS) for 2000 to 2019 with a field-by-year matched OpenAlex reference corpus and geocoded Donbas conflict records. Contrastive text projections distinguish explicit-war language from dual-use technological orientation and eight crisis-relevant capability channels, allowing shifts to be detected beyond projects that directly mention war or security. Segmented annual models show positive post-2014 slope changes in dual-use and maximum-capability measures, whereas explicit-war language follows a different trajectory. Financing-weighted estimates identify positive post-2014 level differences among larger recorded projects. Capability-channel comparisons indicate the strongest positive period differences in computer science, energy, and engineering, together with a temporary narrowing of the capability profile. The results suggest that a neighboring research system can register geopolitical pressure through changes in the technical and preparedness-oriented language of funded projects. More broadly, the study demonstrates how project-level semantic evidence can reveal shifts in public science-funding priorities that remain difficult to observe through disciplinary classifications or explicit conflict terminology alone.
Read the full abstract

Tuesday 15 September

Digital Libraries · Cross-listed from cs.CL

Is Luke the Author of a Gospel and the Acts of the Apostles?

Jacques Savoy

According to Christian tradition, Luke is credited with authoring a Gospel and the Acts of the Apostles, even if his name does not appear in either book, both originally written in Koine Greek. Several biblical scholars assume that both texts were written by a common author, while others deduce the presence of two authors. Different studies have been found to support either finding, some based on qualitative evaluation, while a few others consider the occurrence frequency differences between the two books. To propose an enhanced quantitative analysis, this study is grounded on two recent authorship attribution models. The Burrows' Delta, applied with eleven different feature sizes, demonstrates common authorship. An author verification model confirms this finding. The following experiments consider several stylistic representations, feature sizes, and distance functions to confirm that Luke is the true author of both books.
Read the full abstract

Digital Libraries · Cross-listed from cs.CL

Evolution of US Oral Political Language

Jacques Savoy

The analysis of US political language is usually based on the written form (e.g. presidential addresses) or posts broadcasted on various social networks. Oral production, however, which is even more frequent, can better reveal the style and mode of thinking of the speaker. This study covers this mode of linguistic communication by considering 19 candidates from the presidential elections between 1960 to 2024. Our main research objectives are to disclose the main trends hidden in those presidential debates. Do we observe a clear simplification of the US political language over time? Does Trump have poor language compared to the other candidates? Do unusual stylistic features occur only with a single, specific president? Moreover, can we detect a pattern explaining the success or failure of some nominees? Over time, this study demonstrates a significant reduction in political language complexity, a decrease of the mean sentence length, and a noteworthy decline of complex terms. Moreover, the emotional tone increases over the decades, while logical and rational thinking tends to lessen.
Read the full abstract

Digital Libraries

Quantifying the impact of clinical-academic collaborations

Mohamad Zeina, Nick McNally, Karl S. Peggs et al. (+1 more)

Academic collaboration is of self-evident value but requires a quantitative representation to be optimally guided by policy. No established methodological approach to such representation exists. Here we introduce a general framework of graphical and bibliometric analysis of open data for the task of quantifying the impact of academic networks, with NIHR Biomedical Research Centre (BRC) clinical-academic partnerships in England as the prototype. We define publication-level identities for the 20 English BRCs based on the conjunction of authors from each BRC's partner institutions. Drawing on bibliometric and administrative records, we characterise the graphical properties of each network, estimate what the university adds to the hospital's papers, what the partnership adds to the papers of relatively infrastructure-poor collaborating institutions, and how that gain depends on existing infrastructure. We apply our framework to the NIHR UCLH/UCL BRC as an exemplar. UCLH/UCL authored 20,985 network papers from April 2007, in collaboration with 9,868 distinct external partners over the whole record, forming the most central node of the graph of networks across England. University co-authored papers exhibited 1.6 times the field-weighted citation impact (FWCI) of hospital-only papers, and were 2.1 times as likely to be cited by a patent. Across 60 of the exemplar's most partnered with UK healthcare organisations, the benefit rose from 1.8 times where local NIHR infrastructure activity was densest to 3.4 times where it was sparsest, while impact without the exemplar varied little. Academic networks can be robustly identified from open data, enabling comparative analysis of collaborative impact. Applied to NIHR BRCs, the approach enables quantification of the impact across networks and reveals that benefit is most pronounced where infrastructure is least developed.
Read the full abstract

All authors Mohamad Zeina, Nick McNally, Karl S. Peggs, Parashkev Nachev

Digital Libraries

Testing Our Foundations: Citation Trends, Errors, and Emerging Hallucinations in the Computing Education Literature

Paul Denny, Gweneth Barbre, Musa Blake et al. (+5 more)

Accurate references are foundational to scholarly work, enabling verification, attribution, and systematic review. However, the rapid adoption of large language models has introduced a serious integrity concern: plausible-looking but fabricated citations. Although hallucinated references are widely discussed, their visibility within specific research communities remains unclear. We address this gap by examining reference integrity at key computing education venues using ACM Digital Library data. We analyze referencing trends across 24,751 computing education papers and compare them with the broader ACM corpus of more than 723,000 papers and 15 million references. We then examine reference lists from these venues, classify common bibliographic errors, and manually identify LLM-generated hallucinations containing verifiably false information, including impossible page ranges, invented titles, and misattributed authors. In 2025, hallucinated references appeared across five SIGCSE-sponsored or in-cooperation venues. At the Technical Symposium alone, verified hallucinated references increased from 3 in 2025 to 17 in 2026, appearing in 2.3% of 2026 proceedings papers. Although still relatively rare for now, this growth poses an integrity risk our community should not ignore.
Read the full abstract

All authors Paul Denny, Gweneth Barbre, Musa Blake, Yan Cathy Hua, Juho Leinonen, Andrew Luxton-Reilly, James Prather, Brent N. Reeves

Digital Libraries

Geospatial Metadata Improves Discoverability by Connecting Datasets Across Scientific Disciplines

Daniel Ebanks and Devika Jain

Research data repositories are essential infrastructure for scientific inquiry and for ensuring that datasets follow FAIR (Findable, Accessible, Interoperable, and Reusable) principles. However, repository reuse depends on the quality and completeness of geospatial and thematic metadata, which researchers generally provide voluntarily. Given limited curation resources, it is unsurprising that even Harvard Dataverse, the world's largest general-purpose research repository, contains many incomplete metadata records. Missing fields represent lost information and reduce interoperability. We find that datasets with more missing metadata receive fewer downstream citations and have fewer resolvable connections to other datasets. The implications are particularly important for geospatial datasets: only 0.3% of research datasets include a bounding box, and most represent archival points rather than complete geographic shapes. Our analysis shows that geospatial metadata helps connect concepts across disciplines. After embedding Harvard Dataverse datasets in a metadata knowledge graph, we find that datasets are twice as likely to connect across scientific disciplines through shared geospatial metadata as through keywords. This suggests that geographic metadata is a more reliable basis for cross-disciplinary interoperability than keyword vocabularies, which often remain discipline-specific. We train and fine-tune a small language model using datasets from Harvard Dataverse. Through geospatial metadata enrichment, we increase the share of datasets from different disciplines connected through metadata elements from 58.5% to 63.2%.
Read the full abstract