How to mobilise users' experiential knowledge in the evaluation of advanced technologies and practices in Quebec? The example of the permanent users' and relatives' panel
Bibliographic record
Abstract
INTRODUCTION: With the purpose of supporting scientific professionals and helping them to better integrate the expertise of users in their work, a users' and relatives' panel (URP) was set up at the National Institute for Excellence in Health and Social Services in Quebec (INESSS), Canada for the social services and mental health directorate. URPs are advisory structures that mobilise the experiential knowledge of people affected by various issues. OBJECTIVES: The objective of this study is to assess from a diverse stakeholders' perceptions: (1) the experience of developing and implementing the URP within the context of an Agencies for Health Technology Assessment and Assessment of Social Services (AHTAASS), (2) the contribution of such a URP, (3) the challenges encountered and (4) the perspectives of improvement for the following years. METHODOLOGY: We conducted a qualitative descriptive evaluation study. Nineteen interviews were conducted: six with URP members and 13 with staff representatives. The documents related to the creation of the panel, the URP minutes summarising the discussions and the reports published during that period were collected and analysed. Following a preliminary round of data analysis, a debriefing meeting was conducted with a few participants to validate the results. RESULTS: The panel was set up as part of the INESSS' desire to better integrate experiential knowledge into its recommendations. Twelve projects were presented to the panel on various themes. The URP enabled health professionals to consider dimensions they had not identified, to better integrate the experiential data collected from users into their work and to develop recommendations that made more sense to users. Panel members and INESSS professionals learned to work together, moving the working methods from consultation to collaboration and even coconstruction. Based on the panel's significant contribution, the INESSS decided to maintain it and to strengthen its place in its system to better integrate the experiential knowledge of users into its work. CONCLUSION: This research illustrates how AHTAASS can set up a URP composed exclusively of users, and how it can contribute and be evaluated. It shows that URPs are structures that value the sharing of experiential knowledge of its members, humanise decision-making and give meaning to the work done by scientific professionals. PATIENT OR PUBLIC CONTRIBUTION: One patient-researcher has contributed to the preparation and writing of this manuscript.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.030 | 0.024 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.018 | 0.010 |
| Scholarly communication | 0.007 | 0.004 |
| Open science | 0.002 | 0.005 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".