A re-examination of online journal quality and investigation of the possible impact of poor electronic surrogate quality on researchers
Bibliographic record
Abstract
Objective – This study re-examines the findings of a paper (Ladd, 2010) that investigated whether evidence indicated print equivalent journal collections needed to be preserved, based on the quality of their electronic surrogates. The current study investigates whether: 1) electronic surrogate articles that failed (i.e., the print equivalent article needed to be consulted to view all the content/information) in the first study had improved in quality; and 2) there was evidence that poor-quality electronic surrogates could impact on research if the print equivalent articles did not exist. Methods – Each of the 198 PDF documents identified in the 2010 study as failing were re-examined to assess whether any change in quality had occurred. To assess the possible impact for researchers if they needed to rely solely on poor-quality electronic journal surrogates, citation data were collected for each of the failed scholarly PDFs using Web of Science and Scopus, and usage count data were collected from Web of Science. Results – Across the electronic journal backfiles/archives examined, there were 13.6% fewer failures of electronic surrogates for all PDF documents than in the original study, while for scholarly PDF documents (e.g., research papers) there were 13.8% fewer failures. One electronic journal archive accounted for 91.7% of the improvement for scholarly PDF documents. A second archive accounted for all the observed improvement for non-scholarly PDF documents. The study found that for the failed scholarly PDF documents from the original study, 58.7% had been cited or had Web of Science usage counts from 2010 onward. Conclusion – The study demonstrates a continued need for retaining print equivalent journal titles for the foreseeable future, while poor-quality electronic surrogates are being replaced and digitally preserved. There are still poor-quality images, poor-quality scans of text-only articles, missing pages, and even content of PDF documents that could not be explained (e.g., incorrect text for images when compared to the print). While it is known that not all researchers will consult each of the papers that they cite, although it is best practice to do so, the extent of citations of the failed scholarly PDF documents indicate that having to rely solely on electronic surrogates could pose a problem for researchers.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.312 | 0.624 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.012 | 0.016 |
| Science and technology studies | 0.003 | 0.007 |
| Scholarly communication | 0.014 | 0.016 |
| Open science | 0.004 | 0.008 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.007 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".