Publication and Impact of Preprints Included in the First 100 Editions of the CDC COVID-19 Science Update: Content Analysis
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: Preprints are publicly available manuscripts posted to various servers that have not been peer reviewed. Although preprints have existed since 1961, they have gained increased popularity during the COVID-19 pandemic due to the need for immediate, relevant information. OBJECTIVE: The aim of this study is to evaluate the publication rate and impact of preprints included in the Centers for Disease Control and Prevention (CDC) COVID-19 Science Update and assess the performance of the COVID-19 Science Update team in selecting impactful preprints. METHODS: All preprints in the first 100 editions (April 1, 2020, to July 30, 2021) of the Science Update were included in the study. Preprints that were not published were categorized as "unpublished preprints." Preprints that were subsequently published exist in 2 versions (in a peer-reviewed journal and on the original preprint server), which were analyzed separately and referred to as "peer-reviewed preprint" and "original preprint," respectively. Time to publish was the time interval between the date on which a preprint was first posted and the date on which it was first available as a peer-reviewed article. Impact was quantified by Altmetric Attention Score and citation count for all available manuscripts on August 6, 2021. Preprints were analyzed by publication status, publication rate, preprint server, and time to publication. RESULTS: Of the 275 preprints included in the CDC COVID-19 Science Update during the study period, most came from three servers: medRxiv (n=201, 73.1%), bioRxiv (n=41, 14.9%), and SSRN (n=25, 9.1%), with 8 (2.9%) coming from other sources. Additionally, 152 (55.3%) were eventually published. The median time to publish was 2.3 (IQR 1.4-3.7). When preprints posted in the last 2.3 months were excluded (to account for the time to publish), the publication rate was 67.8%. Moreover, 76 journals published at least one preprint from the CDC COVID-19 Science Update, and 18 journals published at least three. The median Altmetric Attention Score for unpublished preprints (n=123, 44.7%) was 146 (IQR 22-552) with a median citation count of 2 (IQR 0-8); for original preprints (n=152, 55.2%), these values were 212 (IQR 22-1164) and 14 (IQR 2-40), respectively; for peer-review preprints, these values were 265 (IQR 29-1896) and 19 (IQR 3-101), respectively. CONCLUSIONS: Prior studies of COVID-19 preprints found publication rates between 5.4% and 21.1%. Preprints included in the CDC COVID-19 Science Update were published at a higher rate than overall COVID-19 preprints, and those that were ultimately published were published within months and received higher attention scores than unpublished preprints. These findings indicate that the Science Update process for selecting preprints had a high fidelity in terms of their likelihood to be published and their impact. The incorporation of high-quality preprints into the CDC COVID-19 Science Update improves this activity's capacity to inform meaningful public health decision-making.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.035 | 0.019 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.009 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.003 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it