Using Ontario's "Telehealth" health telephone helpline as an early-warning system: a study protocol
Bibliographic record
Abstract
BACKGROUND: The science of syndromic surveillance is still very much in its infancy. While a number of syndromic surveillance systems are being evaluated in the US, very few have had success thus far in predicting an infectious disease event. Furthermore, to date, the majority of syndromic surveillance systems have been based primarily in emergency department settings, with varying levels of enhancement from other data sources. While research has been done on the value of telephone helplines on health care use and patient satisfaction, very few projects have looked at using a telephone helpline as a source of data for syndromic surveillance, and none have been attempted in Canada. The notable exception to this statement has been in the UK where research using the national NHS Direct system as a syndromic surveillance tool has been conducted. METHODS/DESIGN: The purpose of our proposed study is to evaluate the effectiveness of Ontario's telephone nursing helpline system as a real-time syndromic surveillance system, and how its implementation, if successful, would have an impact on outbreak event detection in Ontario. Using data collected retrospectively, all "reasons for call" and assigned algorithms will be linked to a syndrome category. Using different analytic methods, normal thresholds for the different syndromes will be ascertained. This will allow for the evaluation of the system's sensitivity, specificity and positive predictive value. The next step will include the prospective monitoring of syndromic activity, both temporally and spatially. DISCUSSION: As this is a study protocol, there are currently no results to report. However, this study has been granted ethical approval, and is now being implemented. It is our hope that this syndromic surveillance system will display high sensitivity and specificity in detecting true outbreaks within Ontario, before they are detected by conventional surveillance systems. Future results will be published in peer-reviewed journals so as to contribute to the growing body of evidence on syndromic surveillance, while also providing an non US-centric perspective.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.043 | 0.024 |
| Meta-epidemiology (narrow) | 0.003 | 0.003 |
| Meta-epidemiology (broad) | 0.003 | 0.003 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.007 | 0.003 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.005 | 0.002 |
| Research integrity | 0.005 | 0.004 |
| Insufficient payload (model declined to judge) | 0.032 | 0.009 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".