Mobilizing the Use of Policy-Relevant Documents in Evidence-Informed Health Policymaking: The Development and Contents of an Online Repository of Policy-Relevant Documents Addressing Healthcare Renewal in Canada
Bibliographic record
Abstract
Research objectives: 1) Develop an online repository of policy-relevant documents addressing healthcare renewal in Canada; and 2) describe the general contents of policy-relevant documents addressing healthcare renewal in Canada. Methods: The methods for this study were iteratively developed using an approach similar to a scoping review. Documents were identified through website hand-searches and sixteen Canadian health organizations that contributed to the development of the online repository. The majority of organizations are government health ministries/departments or government-supported health organizations. The focus of the analysis was to calculate general descriptive frequencies of the distribution of documents included in the online repository, specifically: 1) the general characteristics of the documents, such as document type, publication year and jurisdictional focus; 2) document themes by national priority areas; 3) document themes by health system topics; and 4) contributing organizations. Results: A total of 304 documents were coded for inclusion in the online repository (http://eihrportal.org). The Health Council of Canada contributed the largest amount of documents (n=60, 19%). The top three types of documents are health and health system data (n=75, 25%), situation analysis (n=72, 24%) and jurisdictional review (n=49, 16%). The top three national priority areas addressed in the documents are health human resources (n=270, 89%), quality as a performance indicator (n=210, 69%) and information technology (n=183, 60%). The least commonly addressed national priority areas are technology assessment (n=19, 6%), prescription drug coverage (n=68, 22%) and Aboriginal health (n=87, 29%). Conclusion: The process of developing a systematic method for identifying policy-relevant documents and retrieving useful information from these documents can be reproduced by anyone interested in using this type of evidence to inform their health policymaking. A number of implications exist for policy and research, both in Canada and in low- and middle-income countries, which have to be considered in relation to the unique nature of this type of evidence.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.174 | 0.328 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.037 | 0.042 |
| Science and technology studies | 0.019 | 0.011 |
| Scholarly communication | 0.031 | 0.012 |
| Open science | 0.007 | 0.020 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".