An Evaluation of the Text Illness Monitoring (TIM) Platform for COVID-19: Cross-sectional Online Survey of Public Health Users
Bibliographic record
Abstract
BACKGROUND: The US public health response to the COVID-19 pandemic has required contact tracing and symptom monitoring at an unprecedented scale. The US Centers for Disease Control and Prevention and several partners created the Text Illness Monitoring (TIM) platform in 2015 to assist US public health jurisdictions with symptom monitoring for potential novel influenza virus outbreaks. Since May 2020, 142 federal, state, and local public health agencies have deployed TIM for COVID-19 symptom monitoring. OBJECTIVE: The aim of this study was to evaluate the utility, benefits, and challenges of TIM to help guide decision-making for improvements and expansion to support future public health emergency response efforts. METHODS: We conducted a brief online survey of previous and current TIM administrative users (admin users) from November 28 through December 21, 2020. Closed- and open-ended questions inquired about the onboarding process, decision to use TIM, groups monitored with TIM, comparison of TIM to other symptom monitoring systems, technical challenges and satisfaction with TIM, and user support. A total of 1479 admin users were invited to participate. RESULTS: A total of 97 admin users from 43 agencies responded to the survey. Most admin users represented the Indian Health Service (35/97, 36%), state health departments (26/97, 27%), and local or county health departments (18/97, 19%), and almost all were current users of TIM (85/94, 90%). Among the 43 agencies represented, 11 (26%) used TIM for monitoring staff exclusively, 13 (30%) monitored community members exclusively, and 19 (44%) monitored both staff and community members. Agencies most frequently used TIM to monitor symptom development in contacts of cases among community members (28/43, 65%), followed by symptom development among staff (27/43, 63%) and among staff contacts of cases (24/43, 56%). Agencies also reported using TIM to monitor patients with COVID-19 for the worsening of symptoms among staff (21/43, 49%) and community members (18/43, 42%). When asked to compare TIM to previous monitoring systems, 78% (40/51) of respondents rated TIM more favorably than their previous monitoring system, 20% (10/51) said there was no difference, and 2% (1/51) rated the previous monitoring system more favorably than TIM. Most respondents found TIM favorable in terms of time burden, staff burden, timeliness of the data, and the ability to monitor large population sizes. TIM compared negatively to other systems in terms of effort to enroll participants (ie, persons TIM monitors) and accuracy of the data. Most respondents (76/85, 89%) reported that they would highly or somewhat recommend TIM to others for symptom monitoring. CONCLUSIONS: This evaluation of TIM showed that agencies used TIM for a variety of purposes and rated TIM favorably compared to previously used monitoring systems. We also identified opportunities to improve TIM; for example, enhancing the flexibility of alert deliveries would better meet admin users' varying needs. We also suggest continuous program evaluation practices to assess and respond to implementation gaps.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.015 | 0.011 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".