Comparison of the Oxford COVID‐19 Government Response Tracker and the ECDC‐JRC Response Measures Database for nonpharmaceutical interventions
Bibliographic record
Abstract
During the COVID-19 pandemic, governments implemented different public health measures and interventions to control COVID-19. These wide-ranging public health interventions, also known as non-pharmaceutical interventions (NPIs), have been documented in Europe in two different publicly available databases. In the context of an EU-funded research project aimed at Preparing for Respiratory Syncytial Virus (RSV) Immunisation and Surveillance in Europe (PROMISE),1 we will use these databases to assess the impact of NPIs (e.g., school closures) on the seasonality of RSV. In this letter, we will focus on the comparison of the NPI databases, which we will later use for our analyses. The first database is the Oxford COVID-19 Government Response Tracker (OxCGRT)2 which collects policy measures implemented from 01 January 2020 to 31 December 2022 in 185 countries and contains 25 indicators that are recorded on an ordinal scale that represents the level of strictness of the policy. The second database is the European Centre for Disease Prevention and Control (ECDC) and the Joint Research Centre (JRC) Response Measures Database (ECDC-JRC RMD)3 which is an archive of NPIs introduced by 30 countries in the EU and EEA from 01 January 2020 to 30 September 2022. We compared five NPI measures related to RSV transmission in the OxCGRT and ECDC-JRC RMD databases and found important differences (see Figure 1). We chose five measures with similar definitions: workplace measures, public gathering restrictions, closure of public spaces, closure of educational institutions and protective mask use. We chose the measures that were most comparable across both databases and chose the strictest measure to define whether an intervention was implemented or not (e.g., we chose full implementation over partial implementation to assess whether a measure was introduced). The four countries in Figure 1 were selected to capture different regions in Europe, ranging from Portugal in the West to the Czech Republic in the East and Denmark and the Netherlands in Northern Europe. The figure shows that the measures in the two databases are often similar, but differences between the start and end dates for each NPI are clearly observed. For example, in Denmark, public gathering restrictions started in March 2020 and ended around August 2021 according to the OxCGRT, while the ECDC-JRC RMD shows approximately the same start and end date but with multiple weeks where public gatherings restrictions were not applied. It is difficult to observe clear patterns in the differences between the two databases, and it is not possible to say which database is more conservative or strict (i.e., one database consistently indicates shorter intervention periods). Another study compared international border restrictions in four countries (Morocco, New Zealand, South Korea and the United States) across five NPI databases, including OxCGRT and also found discrepancies between the timing of interventions.5 The variation between the databases might be explained by differences in definitions, the methodology regarding how the databases are constructed or the way the data were collected. It is also possible that one database is better for certain indicators, while the other is better for other indicators. Considering these points, it is not possible for us to say which NPI database is best and it may be advisable for researchers to run their analyses on both databases (separately). Our assessment finds that the OxCGRT and ECDC-JRC RMD databases are valuable for research purposes; they are comprehensive, freely available and easily accessible. However, despite the extensive documentation provided, we encountered challenges in synchronising the databases and we observed many disparities. This means it is difficult for researchers to select a suitable NPI database for research purposes, including for our modelling study. In summary, we found important differences between the two databases regarding NPIs related to RSV and we would recommend that an evaluation of the databases (e.g., accuracy and completeness) is initiated to support other researchers wanting to use these databases for research purposes. SH and JP have contributed to the conception and design of the study. SH was responsible for data analysis and interpretation of the data. SH wrote the letter, and JP revised all versions. JP was involved until one of the final versions of the letter; after his passing, only small textual changes occurred, with no substantive alterations taking place. PS and HN critically reviewed the manuscript, provided comments and approved this manuscript. JP declares that Nivel has received unrestricted grants from the World Health Organization, Sanofi and the Foundation for Influenza Epidemiology outside the submitted work. HN reports grants from the World Health Organization, the National Institute for Health Research, Pfizer and Icosavax and personal fees from the Bill & Melinda Gates Foundation, Pfizer, GSK, Merck, AbbVie, Janssen, Icosavax, Sanofi, Novavax, outside the submitted work.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.020 | 0.099 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.007 | 0.013 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.007 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".