Performance measurement and evaluation of health practitioner regulation: A scoping review protocol
Bibliographic record
Abstract
Health practitioner regulation plays a fundamental role in public protection by overseeing and governing healthcare professionals to ensure they deliver safe health services. It also serves as a strategic lever to strengthen broader health system goals such as improving the accessibility of services, the sustainability of health workforces, and health system resilience. Although the goals of health practitioner regulation are easily articulated, achieving and evaluating these goals are far more challenging. Performance measurement and evaluation of professional regulators and regulatory systems are critical to improving regulatory processes and functions. This is especially important where there is rising government, public, and professional skepticism and mistrust of the effectiveness and efficiency of regulators across global jurisdictions. Although there is evidence that some health practitioner regulators and regulatory systems engage in performance measurement and evaluation, the similarities and differences remain unclear. The objective of this scoping review is to explore the nature, extent, and range of scholarship related to health practitioner regulatory performance measurement and evaluation. It will explore existing performance measurement and evaluation frameworks; the key principles and areas of focus of these frameworks; and the indicators, metrics and outcomes used to evaluate performance. The review will be conducted in accordance with the JBI guidelines for scoping reviews and will be reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews. Database searches will include Ovid MEDLINE, Ovid EMBASE, CINAHL, Scopus, and Web of Science Core Collection. Gray literature will be identified through leading regulatory organizations, consortiums, and think tanks. Two independent reviewers will screen titles and abstracts followed by full-text and disagreements will be resolved by a third reviewer. Data will be analyzed using descriptive statistics and conventional content analysis. Results will be presented using evidence tables and a narrative summary. Open Science Framework Registration: https://doi.org/10.17605/OSF.IO/WABTF.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.227 | 0.221 |
| Meta-epidemiology (narrow) | 0.006 | 0.007 |
| Meta-epidemiology (broad) | 0.018 | 0.016 |
| Bibliometrics | 0.027 | 0.025 |
| Science and technology studies | 0.007 | 0.008 |
| Scholarly communication | 0.013 | 0.013 |
| Open science | 0.008 | 0.008 |
| Research integrity | 0.012 | 0.010 |
| Insufficient payload (model declined to judge) | 0.084 | 0.020 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".