MétaCan
Menu
Back to cohort
Record W3209159738 · doi:10.1101/2021.11.04.21265886

The United States COVID-19 Forecast Hub dataset

2021· preprint· en· W3209159738 on OpenAlex
Estee Y. Cramer, Yuxin Huang, Yijin Wang, Evan L Ray, Matthew Cornell, Johannes Bracher, Andrea Brennen, Alvaro J Castero Rivadeneira, Aaron Gerding, Katie House, Abdul Hannan Kanji, Ayush Khandelwal, Khoa Le, Jarad Niemi, Ariane Stark, Apurv Shah, Nutcha Wattanchit, Martha Zorn, Nicholas G Reich

Why this work is in the frame

A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.

fundA Canadian funder is recorded on the work.
no affNo Canadian affiliation: this work is invisible to an affiliation-only frame.
No Canadian affiliation. An affiliation-only frame, the usual design, would never have seen this work. It is one of the works that make the case for inverting the frame.

Bibliographic record

VenuemedRxiv · 2021
Typepreprint
Languageen
FieldMathematics
TopicCOVID-19 epidemiological studies
Canadian institutionsnot available
FundersOak Ridge National LaboratoryNatural Sciences and Engineering Research Council of CanadaQuest for Intelligence, Massachusetts Institute of TechnologyPlant Sciences Institute, Iowa State UniversityAdvanced Research Projects AgencyNational Institute of General Medical SciencesUniversity of Massachusetts AmherstJohns Hopkins Bloomberg School of Public HealthUniversity of California, San DiegoIowa State UniversityNorth Carolina State UniversityBundesministerium für Bildung und ForschungUniversity of California, Santa BarbaraDefense Advanced Research Projects AgencyNational Institutes of HealthCenters for Disease Control and PreventionKlaus Tschira StiftungSan Diego Supercomputer CenterHôpitaux Universitaires de GenèveDivision of Materials ResearchCenter for Emerging Infectious Diseases, University of IowaDefense Threat Reduction AgencyCouncil of State and Territorial EpidemiologistsInstitute for Health Metrics and EvaluationWellcome TrustIndiana University-Purdue University IndianapolisUniversity of MichiganNational Institute of Diabetes and Digestive and Kidney DiseasesCalifornia Institute of TechnologyBill and Melinda Gates FoundationLos Alamos National LaboratoryJohns Hopkins UniversityGordon and Betty Moore FoundationU.S. Department of Homeland SecurityLaboratory Directed Research and DevelopmentNational Science Foundation
KeywordsLeverage (statistics)Coronavirus disease 2019 (COVID-19)DownloadGovernment (linguistics)PandemicDisease controlComputer scienceScale (ratio)Consensus forecastData scienceBusinessEconometricsGeographyInfectious disease (medical specialty)EconomicsMachine learningWorld Wide WebEnvironmental healthMedicine

Abstract

fetched live from OpenAlex

Abstract Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident hospitalizations, incident cases, incident deaths, and cumulative deaths due to COVID-19 at national, state, and county levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.004
metaresearch head score (Gemma)0.058
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Meta-epidemiology (narrow)
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Empirical · Consensus signal: Empirical
Teacher disagreement score0.204
Threshold uncertainty score1.000

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0040.058
Meta-epidemiology (narrow)0.0010.000
Meta-epidemiology (broad)0.0010.000
Bibliometrics0.0000.000
Science and technology studies0.0010.000
Scholarly communication0.0000.000
Open science0.0010.005
Research integrity0.0000.001
Insufficient payload (model declined to judge)0.0000.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.400
GPT teacher head0.465
Teacher spread0.065 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it