MétaCan
Menu
Back to cohort
Record W3208091411 · doi:10.5281/zenodo.4683794

Repository Features to Help Researchers: An invitation to a dialogue

2021· article· en· W3208091411 on OpenAlexaff
Chris Graf, Kiera McNeice, Wei Mun Chan, Sarah Callaghan, Ilaria Carnevale, Imogen Cranston, Scott Edmunds, Nicholas Everitt, Emma Ganley, Iain Hrynaszkiewicz, Varsha Khodiyar, Adam Leary, Thomas Lemberger, Catriona MacCallum, Hollydawn Murray, Kathryn Sharples, Marina Soares E Silva, Guillaume Wright, Peter McQuilton, Susanna‐Assunta Sansone

Bibliographic record

VenueZenodo (CERN European Organization for Nuclear Research) · 2021
Typearticle
Languageen
FieldComputer Science
TopicData Mining Algorithms and Applications
Canadian institutionsVictoria Park
Fundersnot available
KeywordsData scienceComputer scienceEpistemologyEngineering ethicsCognitive sciencePsychologyEngineeringPhilosophy

Abstract

fetched live from OpenAlex

A group of publishers came together to discuss how we could reduce the complexity and inconsistency provided in publisher's advice to researchers when selecting an appropriate data repository. It is a shared goal among publishers and other stakeholders to increase repository use – which remains far from optimal – and we assume that helping researchers find a suitable repository more easily will help achieve this. To address this a list of features has been created and it is intended only as a framework within which publishers can make recommendations to researchers, not as a way to restrict which repositories researchers may choose for their data. Our intention is that the features we highlight will act to initiate engagement and collaboration among publishers, repositories and the RPOs, government and funders that ultimately make the policies around Open Research. As we start this conversation, it is important that we act together with other stakeholders to raise awareness of the challenges involved around FAIR data and to prevent any perverse consequences. From the RDA FAIRsharing WG point of view, the ultimate objective is to map repository features across all existing initiatives, and to identify a common core set of metadata fields that all stakeholders want to see in registry of repositories. The FAIRsharing registry in particular is agnostic as to the selection process of standards, repositories and policies, as part of its commitment to working with and for all stakeholder groups.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.107
metaresearch head score (Gemma)0.145
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesScholarly communication
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Empirical · Consensus signal: none
Teacher disagreement score0.966
Threshold uncertainty score0.568

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.1070.145
Meta-epidemiology (narrow)0.0030.003
Meta-epidemiology (broad)0.0030.005
Bibliometrics0.0020.002
Science and technology studies0.0340.018
Scholarly communication0.0340.043
Open science0.0080.047
Research integrity0.0860.101
Insufficient payload (model declined to judge)0.0150.006

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.057
GPT teacher head0.288
Teacher spread0.231 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

Study designNot applicable
Domainnot available
GenreEmpirical

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations1
Published2021
Admission routes1
Has abstractyes

Explore more

Same venueZenodo (CERN European Organization for Nuclear Research)Same topicData Mining Algorithms and ApplicationsFrench-language works237,207