Cross-sectional analysis of bibliometrics and altmetrics: comparing the impact of qualitative and quantitative articles in the <i>British Medical Journal</i>
Bibliographic record
Abstract
OBJECTIVES: In comparison to quantitative research, the impact of qualitative articles in the medical literature has been questioned by the BMJ; to explore this, we compared the impact of quantitative and qualitative articles published in BMJ. DESIGN: Cross-sectional survey. SETTING: Articles published in the BMJ between 2007 and 2017. MAIN OUTCOME MEASURES: Bibliometric and altmetric measures of research impact were collected using Web of Science, Google Scholar, Scopus, Plum Analytics and ProQuest Altmetric. Bibliometric measures consisted of citation numbers, field weighted citation impact and citation percentile. Altmetric measures consisted of article usage, captures, mentions, readers, altmetric attention score and score percentile. Scores were compared using the Wilcoxon Rank-sum test. RESULTS: We screened a total of 7777 articles and identified 42 qualitative articles. Each qualitative article was matched to 3 quantitative articles published during the same year (126 quantitative articles). Citation numbers were not statistically different between the two research types; the median number of citations (google scholar) per quantitative article was 62 (IQR 38-111) versus 58 (IQR 36-85) per qualitative article (p=0.47). Using Plum Analytics, qualitative articles were found to have a significantly higher usage, with a median of 984 (IQR 581-1351) versus 379 (IQR 177-763) for quantitative (p<0.001). The Altmetric Attention Score was higher for quantitative articles at 16 (IQR 7-37) versus qualitative articles at 9 (IQR 5-23, p=0.05), as was the Altmetric Score percentile 93 (IQR 87-96) versus 88 (IQR 76-95; p=0.02). CONCLUSION: Qualitative and quantitative articles published in the BMJ between 2007 and 2017 both have a high impact. No article type was consistently superior in terms of bibliometric or altmetric measures, suggesting that type of article is not the major driver of impact.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.041 | 0.177 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.026 | 0.032 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.004 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".