A rapid qualitative methods assessment and reporting tool for epidemic response as the outcome of a rapid review and expert consultation
Bibliographic record
Abstract
During the first year of the COVID-19 pandemic, the Methods Sub-Group of the WHO COVID-19 Social Science Research Roadmap Working Group conducted a rapid evidence review of rapid qualitative methods (RQMs) used during epidemics. The rapid review objectives were to (1) synthesize the development, implementation, and uses of RQMs, including the data collection tools, research questions, research capacities, analytical approaches, and strategies used to speed up data collection and analysis in their specific epidemic and institutional contexts; and (2) propose a tool for assessing and reporting RQMs in epidemics emergencies. The rapid review covered published RQMs used in articles and unpublished reports produced between 2015 and 2021 in five languages (English, Mandarin, French, Portuguese, and Spanish). We searched multiple databases in these five languages between December 2020 and January 31, 2021. Sources employing "rapid" (under 6 months from conception to reporting of results) qualitative methods for research related to epidemic emergencies were included. We included 126 published and unpublished sources, which were reviewed, coded, and classified by the research team. Intercoder reliability was found to be acceptable (Krippendorff's α = 0.709). We employed thematic analysis to identify categories characterizing RQMs in epidemic emergencies. The review protocol was registered at PROSPERO (no. CRD42020223283) and Research Registry (no. reviewregistry1044). We developed an assessment and reporting tool of 13 criteria in three domains, to document RQMs used in response to epidemic emergencies. These include I. Design and Development (i. time frame, ii. Training, iii. Applicability to other populations, iv. Applicability to low resource settings, v. community engagement, vi. Available resources, vii. Ethical approvals, viii. Vulnerability, ix. Tool selection); II. Data Collection and Analysis (x. concurrent data collection and analysis, xi. Targeted populations and recruitment procedures); III. Restitution and Dissemination (xii. Restitution and dissemination of findings, xiii. Impact). Our rapid review and evaluation found a wide range of feasible and highly effective tools, analytical approaches and timely operational insights and recommendations during epidemic emergencies.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.053 | 0.074 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.004 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".