Number of Publications on New Clinical Prediction Models: A Bibliometric Review
Bibliographic record
Abstract
Background: Concerns have been expressed about the abundance of new clinical prediction models (CPMs) proposed in the literature. However, the extent of this proliferation in prediction research remains unclear. Objective: This study aimed to estimate the total and annual number of CPM development-related publications available across all medical fields. Methods: Using a validated search strategy, we conducted a systematic search of literature for prediction model studies published in Pubmed and Embase between 1995 and the end of 2020. By taking random samples for each year, we identified eligible studies that developed a multivariable model (ie, diagnostic or prognostic) for individual-level prediction of a health outcome across all medical fields. Exclusion criteria included development of models with a single predictor, studies not involving humans, methodological studies, conference abstracts, articles with unavailable full text, and those not available in English. We estimated the total and annual number of published regression-based multivariable CPM development articles, based on the total number of publications, proportion of included articles, and the search sensitivity. Furthermore, we used an adjusted Poisson regression to extrapolate our results to the period 1950-2024. Additionally, we estimated the number of articles that developed CPMs using techniques other than regression (eg, machine learning). Results: From a random sample of 10,660 articles published between 1995 and 2020, 109 regression-based CPM development articles were included. We estimated that 82,772 (95% CI 65,313-100,231) CPM development articles using regression were published, with an acceleration in model development from 2010 onward. With the addition of articles that developed non-regression-based CPMs, the number increased to 147,714 (95% CI 125,201-170,226). After extrapolation to the years 1950-2024, the number of articles increased to 156,673 and 248,431 for regression-based models and total CPMs, respectively. Conclusions: Based on a representative sample of publications from the literature, we estimated that nearly 250,000 articles reporting the development of CPMs across all medical fields were published until 2024. CPM development-related publications continue to increase in number. To prevent research waste and close the gap between research and clinical practice, focus should shift away from developing new CPMs to facilitating model validation and impact assessment of the plethora of existing CPMs. Limitations of this study include restriction of search to articles available in English and development of the validated search strategy prior to the popularity of artificial intelligence and machine learning models.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.035 | 0.193 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.008 | 0.010 |
| Bibliometrics | 0.150 | 0.151 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.007 | 0.008 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.007 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".