CASE 11: Evaluating a Public Health Program for Continuous Quality Improvement: Options and Methods in a Time of Pandemic
Bibliographic record
Abstract
In 2018, a review of the public health inspection program by the Middlesex-London Health Unit (MLHU) identified a gap in how its Environmental Health and Infectious Disease division evaluated mandatory public health services. To meet the requirements of the Ontario Public Health Standards, the MLHU must implement important recommendations from the review to ensure the most effective and efficient service model is delivered. One recommendation under consideration is to initiate activities that will entrench quality assurance and promote continuous quality improvement through monitoring and integration of findings from client experience into MLHU’s mandatory public health services. Rex Paul, the Manager of the Food Safety and Healthy Environment team at the MLHU, wants to initiate an evaluation framework for mandatory public health inspection programs. He is interested in exploring best practices and/or approaches for evaluating experiences of public pool and spa operators. This will serve as a pilot to assess client experiences with other mandatory public health services. Additionally, Rex wants to know the best data collection methods for the assessment of mandatory public health services in the context of COVID-19 pandemic. This will take a different approach as it is focused on improving the performance of staff who conduct regulatory work that enforces rules as opposed to previous client experience surveys where clients seek services from Public Health (vaccines, sexual health checks, smoking cessation, etc.).\nThis case outlines the procedures and approaches for the evaluation of a public health program. It discusses relevant tools including logic models useful for clarifying the purpose of evaluation, mapping out an evaluation plan, identifying data collection tools, and collecting important information to address evaluation needs. The case also describes the importance of organizational standards and stakeholder consultations towards effective data collection and analysis for overall improvement in the program outcome.\nThis process will help Paul’s team understand best approaches for collecting client feedback and incorporating findings to improve delivery of mandatory public health services. It will eventually provide baseline information for the planned monitoring of all regulated clients’ experiences of the MLHU’s mandatory programs.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.065 | 0.070 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.014 | 0.009 |
| Scholarly communication | 0.010 | 0.009 |
| Open science | 0.005 | 0.007 |
| Research integrity | 0.012 | 0.009 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".