Assessing healthcare organizations’ readiness to implement a learning health system: questionnaire validation using a Delphi method
Bibliographic record
Abstract
INTRODUCTION: Adopting a learning health system (LHS) approach holds promise for bridging knowledge between policymakers, health professionals, managers, researchers, and patients and their families to collaboratively improve health care. Organizational readiness assessments exist in the quality improvement literature, but may not consider LHS components. This study aimed to establish the content validity of a new LHS readiness questionnaire. METHODS: A three-round Delphi study was conducted to establish consensus on the importance, relevance, clarity, and comprehensiveness of the domains and items included in this questionnaire. Purposive sampling was used to recruit participants with expertise in LHS who are involved in healthcare organizations across Canada and internationally (n = 41). A minimum of 70% agreement represented consensus. A steering committee reviewed findings and refined items for clarity. Modified items were re-tested in subsequent rounds. RESULTS: In Round 1, 85 items were tested, of which 41 achieved consensus, 7 were removed, 21 underwent major modification, 16 were clarified and retested, and 11 new items were proposed. Round 2 tested 36 items (25 revised, 11 new). 18 items achieved consensus, 8 were removed, and 10 were modified. In Round 3, 10 items were tested, 5 achieved consensus, 1 was removed, and 4 were clarified and included post-expert panel discussion. Overall, 41 items were retained in their original form, 20 were modified, and 7 new items were added. The final measure includes 68 items reflected by four domains: (1) performance to data (n = 13 items), (2) data to knowledge (n = 13 items), (3) knowledge to performance (n = 22 items), and (4) LHS core values (n = 20 items). CONCLUSION: The proposed new measure can help establish organizational readiness for change. Future research should seek to test the psychometric properties of this tool and explore potential barriers to its adoption amongst interested parties.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.117 | 0.086 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.002 | 0.005 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".