Developing an evaluation framework for assessing the impact of recovery colleges: protocol for a participatory stakeholder engagement process and cocreated scoping review
Bibliographic record
Abstract
INTRODUCTION: Recovery colleges (RCs) are mental health centres aimed at equipping people with skills to live a meaningful life despite the presence of mental distress. Unique to them is the aspect of cocreation; RCs are designed collaboratively with people of lived experiences of mental health and addictions and care providers. Despite established benefits, there remains a lack of empirical evidence on how RCs work and on their impact. AIMS: We aim to address this gap by designing a cocreated evaluation framework for RCs. This will be accomplished by engaging RC student/facilitators to provide perspectives on RCs/RC evaluation and cocreate a scoping review identifying evaluation gaps in the literature. Themes identified through these processes will form the evaluation framework. METHODS AND ANALYSIS: Two methodologies will be used to explore RC evaluation: student/facilitator engagement and a scoping review of current published and grey literature on RC evaluation. Engagement will be achieved using a participatory action research approach consisting of informant interviews of ~25 RC students/facilitators across Canada, which will be thematically analysed. The scoping review will follow methodology described by Arksey and O'Malley modified to support cocreation. Concurrent conducting of the engagement process and scoping review will allow RC students and peer facilitators the opportunity to shape RC evaluations, address gaps in the literature and codesign an evaluation framework focused on recovery-oriented processes and outcomes mattering most to RCs students/facilitators. ETHICS AND DISSEMINATION: Ethics approval was received for the RC student/facilitator engagement component from the Centre for Addictions and Mental Health Research Ethics Board (#042-2020) and Ontario Shores Centre for Mental Health Sciences (#20-013-B). Scoping review results will be copresented through national and international medical education conferences and published in open-access peer-reviewed journals. Furthermore, a dissemination strategy on evaluation for the national RC community will be created.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.396 | 0.318 |
| Meta-epidemiology (narrow) | 0.006 | 0.006 |
| Meta-epidemiology (broad) | 0.008 | 0.013 |
| Bibliometrics | 0.017 | 0.016 |
| Science and technology studies | 0.010 | 0.009 |
| Scholarly communication | 0.013 | 0.010 |
| Open science | 0.009 | 0.013 |
| Research integrity | 0.012 | 0.010 |
| Insufficient payload (model declined to judge) | 0.051 | 0.013 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".