Investigating Document Type Discrepancies between OpenAlex and the Web of Science
Bibliographic record
Abstract
Bibliometrics, whether used for research or research evaluation, relies on large multidisciplinary databases of research outputs and citation indices. The Web of Science (WoS) was the main supporting infrastructure of the field for more than 30 years until several new competitors emerged. OpenAlex, launched in 2022, stands out for its openness and extensive coverage. While OpenAlex may reduce or eliminate barriers to accessing bibliometric data, one of the concerns that hinder its broader adoption for research and research evaluation is the quality of its metadata. This study aims to assess the metadata quality of works in OpenAlex and WoS, focusing on document type accuracy. We observe that over 4% of the publications indexed in both OpenAlex and WoS appear to be misclassified as research articles or reviews, and that the vast majority (about 97%) of these errors occur in OpenAlex. By addressing discrepancies and misattributions in document types this research seeks to enhance awareness of data quality issues that could impact bibliometric research and evaluation outcomes. Enquête sur les divergences de types de documents entre OpenAlex et the Web of Science RésuméLa bibliométrie, qu’elle soit utilisée pour la recherche ou pour l’évaluation de la recherche, repose sur de vastes bases de données multidisciplinaires regroupant des publications scientifiques et indices de citation. The Web of Science (WoS) était la principale infrastructure dans le domaine pendant plus 30 ans jusqu’à l’émergence de plusieurs nouveaux concurrents. OpenAlex, lancée en 2022, se démarque pour sa transparence et sa couverture étendue. Pendant qu’OpenAlex pouvait réduire voire éliminer les barrières pour l’accès aux données bibliométriques, une des préoccupations qui entravait sa large adoption pour la recherche et l'évaluation de la recherche est la qualité de ses métadonnées. Cette étude a pour but d’évaluer la qualité du travail des métadonnées dans OpenAlex et WoS, en se focalisant sur la précision du type de document. On observe que plus de 4% des publications indexées, à la fois dans OpenAlex et dans WoS, semblent être mal classées en tant qu’articles ou revues de recherche, et que la grande majorité (environ 97%) de ces erreurs se présentent dans OpenAlex. En relevant ces divergences et erreurs d’attribution dans les types de documents, cette recherche a pour but d’améliorer l’attention portée aux problèmes de qualité des données qui pourraient impacter les recherches bibliométriques et les résultats de l’évaluation. Mots-clésBibliométrie; évaluation de la recherche; OpenAlex; Web of Science; Bibliothèque et Science de l’Information; données ouvertes, métadonnées, type de document
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.021 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.005 |
| Scholarly communication | 0.002 | 0.006 |
| Open science | 0.004 | 0.002 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".