Coverage with evidence development: The Ontario experience
Bibliographic record
Abstract
BACKGROUND: For non-drug technologies, there is often residual uncertainty following systematic review, mainly due to inadequate evidence of efficacy. The unwillingness to make decisions in the presence of uncertainty may lead to passive diffusion and intuitive decision making with or without public pressure. This may affect health system sustainability. There is increasing interest in post-market evaluation through processes that include coverage with evidence development (CED) to address residual uncertainty regarding effectiveness and cost-effectiveness. Global experience of CED has been slow to develop despite their potential contribution to decision making. METHODS: Ontario's field evaluation program to better inform decision making represents a collaboration between physicians, policy decision makers and academic centers. We report results of the first ten CEDs from this program to assess whether they achieved their objective of influencing policy by addressing residual uncertainty following systematic review. RESULTS: Since 2003, nineteen field evaluation studies to resolve residual uncertainty following systematic review have been completed, ten of which met the criteria of CED and are the focus of this report. There was more than one patient subgroup or intervention in three of the CEDs. This provided the basis for evaluating thirteen outcomes. In each case, the CED addressed the uncertainty and led to a decision based on the systematic review and CED result. The CEDs led to adoption of the technology in six instances, modified adoption in three instances and withdrawal in four instances. CONCLUSIONS: CED makes an important contribution to translating evidence to decision making. Methodologies are needed to increase the scope and reduce timelines for CEDs, such as the use of linked comprehensive and robust data sets and collaborative studies with other jurisdictions. CED before making long-term funding decisions, especially where there is uncertainty of effectiveness, safety or cost-effectiveness, should be increasingly funded by health systems.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.008 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.002 | 0.000 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".