How do Alternative and Traditional Dissemination Metrics Compare in Medical Education Scholarship?
Bibliographic record
Abstract
Purpose Medical educators need to demonstrate the dissemination of scholarship for academic promotion. Dissemination impact has traditionally been measured using citation‐based metrics. In recent years, other platforms (such as social media tools like Twitter and Facebook) have provided new avenues for authors to promote their work to both scholarly and non‐scholarly audiences. While alternative metrics (altmetrics) can capture non‐traditional dissemination data such as attention generated on social media, the academic value of altmetrics in medical education is unclear. The aim of this study was to determine which altmetrics are indices of access counts and citations in a highly cited, general interest medical education journal. Methods A database study was performed (August 2015) for all Medical Education papers in 2012 (n=236) and 2013 (n=246). Citation, altmetric and access (HTML views + PDF downloads) data were obtained from Scopus, the Altmetric Bookmarklet Tool, and the journal Medical Education , respectively. Correlation coefficients (r values) between variables were determined and statistical significance was calculated. Results Facebook, Google+, Reddit, and LinkedIn were each only used in the dissemination of a small fraction of papers and were not characterized further. Twitter and Mendeley were the only Altmetric‐tracked platforms utilized in the dissemination of greater than 50% of articles. For access counts versus Mendeley downloads, Twitter mentions, and Altmetric Scores, the correlation coefficients were r = 0.79, 0.56, and 0.52 for 2012 papers and 0.65, 0.32, and 0.35 for 2013 papers, respectively. For citation counts versus access counts, Mendeley downloads, Twitter mentions and Altmetric Scores, the correlation coefficients were r = 0.77, 0.81, 0.57, and 0.58 for 2012 papers and 0.62, 0.61, 0.18, and. 0.22 for 2013 papers, respectively. All correlations were statistically significant (P<0.01). Conclusions Mendeley downloads appear to be the strongest altmetric index of both readership and citations for articles in the high‐impact, general interest journal Medical Education . We conclude that access‐based indices (i.e. views and downloads) may be the best Altmetric tool to use in conjunction with citations for the purposes of measuring the impact of medical education scholarship.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.139 | 0.526 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.040 | 0.065 |
| Science and technology studies | 0.001 | 0.003 |
| Scholarly communication | 0.015 | 0.019 |
| Open science | 0.002 | 0.006 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".