Participatory Evaluation of the OECD PaRIS Dashboards in Saskatchewan: Grounded in Relationships
Bibliographic record
Abstract
Content The Organisation for Economic Co-operation and Development (OECD)’s Patient-Reported Indicator Surveys (PaRIS) Project was implemented in Saskatchewan in 2023 to center the voices of patients and providers using a standardized collection of patient-reported experience and outcome measures (PREMS and PROMS). Objective To evaluate clinic-level engagement with the PaRIS Dashboards and explore perspectives of the partnering clinics on data interpretation, relevance, and opportunities for improvement. Design The project was informed by a framework of participatory evaluation and transformative learning, focusing on the strengths and opportunities for improvement in the Dashboard Reports returned to partnering clinics. Setting and Participants Ten primary care clinics in urban and rural settings across Saskatchewan which included primary care providers, clinic managers, Medical Office Assistants, and people with lived experience (PWLE). Intervention The PaRIS Dashboards, which summarized local and provincial-level PREMs and PROMs survey results. Results Our partners across all ten clinics responded positively to the Dashboards citing them as tools for patient/provider advocacy and internal quality improvement. They also valued the relational and transparent nature of the processes, reporting that it had increased their trust and willingness to engage with future surveys. Several clinics (7/10) noted how the data affirmed their existing efforts and identified gaps in care delivery, particularly around access and provider-patient communication. Identified challenges with the Dashboards included difficulty interpreting complex data visualizations (e.g., bar graphs on mobile devices) and embargoes on comparative data that delayed fuller analysis. Areas for improvement identified included, presenting results in more accessible visual formats, integrating real-time data collection (e.g., iPads in waiting rooms), and tailoring engagement to facilitate digitally underserved populations. Nonetheless, the participatory process fostered a shared sense of ownership and optimism for future learning environments. Conclusions The participatory evaluation of the PaRIS Dashboards in Saskatchewan demonstrated that relationships enhanced the relevance and impacted health data informing primary care. Our results also supported the integration of participatory evaluation into future data initiatives to strengthen healthcare system transformation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.127 | 0.081 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.003 |
| Science and technology studies | 0.011 | 0.007 |
| Scholarly communication | 0.006 | 0.003 |
| Open science | 0.003 | 0.012 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.007 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".