An Evaluation of the Text Illness Monitoring (TIM) Platform for COVID-19: Cross-sectional Online Survey of Public Health Users
Bibliographic record
Abstract
BACKGROUND: The US public health response to the COVID-19 pandemic has required contact tracing and symptom monitoring at an unprecedented scale. The US Centers for Disease Control and Prevention and several partners created the Text Illness Monitoring (TIM) platform in 2015 to assist US public health jurisdictions with symptom monitoring for potential novel influenza virus outbreaks. Since May 2020, 142 federal, state, and local public health agencies have deployed TIM for COVID-19 symptom monitoring. OBJECTIVE: The aim of this study was to evaluate the utility, benefits, and challenges of TIM to help guide decision-making for improvements and expansion to support future public health emergency response efforts. METHODS: We conducted a brief online survey of previous and current TIM administrative users (admin users) from November 28 through December 21, 2020. Closed- and open-ended questions inquired about the onboarding process, decision to use TIM, groups monitored with TIM, comparison of TIM to other symptom monitoring systems, technical challenges and satisfaction with TIM, and user support. A total of 1479 admin users were invited to participate. RESULTS: A total of 97 admin users from 43 agencies responded to the survey. Most admin users represented the Indian Health Service (35/97, 36%), state health departments (26/97, 27%), and local or county health departments (18/97, 19%), and almost all were current users of TIM (85/94, 90%). Among the 43 agencies represented, 11 (26%) used TIM for monitoring staff exclusively, 13 (30%) monitored community members exclusively, and 19 (44%) monitored both staff and community members. Agencies most frequently used TIM to monitor symptom development in contacts of cases among community members (28/43, 65%), followed by symptom development among staff (27/43, 63%) and among staff contacts of cases (24/43, 56%). Agencies also reported using TIM to monitor patients with COVID-19 for the worsening of symptoms among staff (21/43, 49%) and community members (18/43, 42%). When asked to compare TIM to previous monitoring systems, 78% (40/51) of respondents rated TIM more favorably than their previous monitoring system, 20% (10/51) said there was no difference, and 2% (1/51) rated the previous monitoring system more favorably than TIM. Most respondents found TIM favorable in terms of time burden, staff burden, timeliness of the data, and the ability to monitor large population sizes. TIM compared negatively to other systems in terms of effort to enroll participants (ie, persons TIM monitors) and accuracy of the data. Most respondents (76/85, 89%) reported that they would highly or somewhat recommend TIM to others for symptom monitoring. CONCLUSIONS: This evaluation of TIM showed that agencies used TIM for a variety of purposes and rated TIM favorably compared to previously used monitoring systems. We also identified opportunities to improve TIM; for example, enhancing the flexibility of alert deliveries would better meet admin users' varying needs. We also suggest continuous program evaluation practices to assess and respond to implementation gaps.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.012 | 0.025 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.004 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".