Validation of Physician Diagnosis of Bordetella pertussis in Alberta, Canada (2004–2014)
Bibliographic record
Abstract
Background. Pertussis outbreaks are occurring throughout North America. To assess the utility of physician diagnosis in estimating vaccine effectiveness, we sought to validate physician diagnosis of Bordetella pertussis (BP) against laboratory confirmed cases. Alberta has a universal publicly funded health care insurance system. The Alberta Ministry of Health (AH) captures the ICD-9 coded diagnoses for all physician billing records and hospitalizations. All lab positive and epidemiologically linked cases of notifiable diseases, including BP, are also forwarded to public health within AH. All records include a Unique Lifetime Identifier (ULI) for the patient Methods. We used PCR positive cases notified to public health as the gold standard (public health diagnosed). We extracted records with ICD-9 codes of 033, 033.0, 033.1, 033.8, and 033.9 from physician billing and hospitalization databases (physician diagnosed BP). Public health records were deterministically linked with billing and hospitalization data using ULIs. Numbers and proportions of true positive (TP: public health and physician diagnosed), false positive (FP: only physician diagnosed) and false negative (FN: only public health diagnosed) cases were estimated. Sensitivity (SE) was calculated as TP/TP + FN; positive predictive value (PPV) as TP/TP + FP. Physician inaccuracy was calculated as FN/TP + FN. Analysis is ongoing as data are received for negative predictive value and specificity. Results. From public health OR physician diagnosis 7711 cases of BP were identified. Of these, 13% were TP, 27% FN, and 60% were FP. SE of physician diagnosis was 32.5% (95% CI: 30.9%-34.1%) and PPV was 18.0% (95% CI: 17.3%–19.4%). Among 5587 physician diagnosed cases, ICD-9 code 033 (whooping cough) was used in TP and ICD-9 code 033.0 (B. pertussis) was used in FP (86% each). Of the 2124 FN cases 71% were physician diagnosed as “acute respiratory infection” (ICD-9 codes 460-466). Physicians incorrectly diagnosed 67% of public health diagnosed BP cases. Conclusion. Our study demonstrates low sensitivity of physician diagnosis for BP. It also shows that 67% of Alberta physician diagnosed cases of BP are misdiagnoses. Additional lab negative data is essential to estimate negative predictive value and specificity. Disclosures. All authors: No reported disclosures.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.014 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.003 | 0.004 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.002 | 0.000 |
| Open science | 0.003 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".