Engaging Primary Care using participatory evaluation:  Approaches to uncovering end-user needs of an Integrated Medical Record.
Bibliographic record
Abstract
Given the unique role Primary Care plays in the health journeys of their patients capturing their perspectives on harmonized electronic medical records is critical. In Ontario, Canada, our multi-disciplinary, Primary Care led working group evaluated clinicians’ needs in the context of a “One Person, One Record” health information system. Previous quantitative phases of our work indicated that primary care clinicians had interest in supporting it but concerns that required attention such as data considerations, information exchange, practice support, costs, privacy, governance, and practice efficiency and autonomy. We grouped this feedback into three discovery areas; (1)system features, (2) change management, and (3) test of change. In summary, voices of clinicians pointed to a clear need for robust engagement. The question remained how to accomplish this task. Workshop Component(30 minutes) After introducing the audience to the context, the audience will be guided in a group, case-study, problem-solving exercise. The objective of the exercise is to co-design an evaluation-based, primary-care engagement approach within a specific set of parameters (i.e. timeline, funding/ budget, human resources, partners, health teams.) Discovery Component (15 minutes) The tables (groups) will be asked to briefly share their participatory evaluation approaches. Our Approach and Methods(10 minutes) During this component of the workshop, we will walk the audience through the actual method that we used, while drawing upon connections from their insights. In 2023, we undertook the socialization phase of our work using qualitative methods to foster in-depth dialogues on the central themes. Our working group co-designed our engagement approach using diffusion of innovation theory and a complimentary participatory evaluation approach. Primary Care clinicians and administrative personnel (n=54) were engaged in mini focus groups on these topics in the form of a World Café series. The change management themes (and corresponding subsets of probing questions stemming from them) were grouped in to three distinct breakout discussions framed by the following over-arching research questions. SYSTEM NEEDS What system features would be necessary for clinicians to adopt a new integrated EMR? CHANGE MANAGEMENT What actions and supports are necessary for the successful diffusion and uptake of an integrated EMR? TEST OF CHANGE What are the core elements of a primary care pilot that will lead to spread and scale of an integrated EMR? Findings(10 minutes) This panel will share its, tools, and best practices with audience members on this primary care engagement approach for spread and scale purposes, in addition to the qualitative data representing primary cares’ change management needs. Discussion (15 minutes) Conclusion, Recommendations and Contributions: Recently, the discussion of primary care adopting a Health Information System (HIS) for their practices has been trending across the world. To get there, an in-depth understanding of primary care’s current systems, support and test of change needs is required. Components are not just a one-time assessment; they are ongoing as service levels and requirements mature. This panel will share insights on how to explore key components using a method that has been tested, endorsed, and delivered by primary care.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.217 | 0.158 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.013 | 0.015 |
| Scholarly communication | 0.011 | 0.009 |
| Open science | 0.003 | 0.024 |
| Research integrity | 0.003 | 0.004 |
| Insufficient payload (model declined to judge) | 0.008 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".