Engaging people with lived experience in peer review: Expanding the role of citizen reviewers
Bibliographic record
Abstract
Abstract Background Research shows that including the voices of people with lived experience (PWLE) in the full research process ensures that the research is both relevant and meaningful in real‐life contexts. However, operationalizing this in a meaningful way can be challenging, particularly in the context of dementia. Since 2014, the Alzheimer Society of Canada (ASC) has included PWLE in the Alzheimer Society Research Program’s (ASRP) peer review process through the role of the citizen reviewer, however, this has been limited in scope. In 2019, a pilot study expanded this role to a broader panel of citizen reviewers who are equal participants in the peer review process, scoring applications and providing valuable contextual feedback. In 2020, this process was further formalized, allowing greater participation and enhanced engagement. Method Qualitative feedback was collected from panelists following the 2019 pilot which demonstrated a collective desire to improve overall engagement while formalizing a systematic process. The data was thematically assessed and used to create several resources to recruit, maintain, and train citizen reviewers as equal and valued participants in the peer review process. This was conducted within the context of the pandemic, in which the full process was adapted to a virtual peer review process. Result Data from the 2019 pilot resulted in a four‐step process for recruitment, orientation, and training of citizen reviewers. Semi‐structured interviews were conducted with potential candidates, followed by a comprehensive orientation package, and a virtual orientation session designed for PWLE that included role playing and case studies. Lastly, technical support meetings were created to support the virtual nature of the process. Twenty‐five citizen reviewers were recruited, an increase of 44% participation from the previous year. This, in turn, has led to the creation of a new resource guide for engaging PWLE in peer review, which demonstrates how to operationalize meaningfully engaging and valuing the experiences of people living with dementia. Conclusion The pilot and subsequent expansion of the citizen reviewer role has created a new way to operationalize lived experiences within the research funding process. While requiring significant upfront work, the value of real‐life experience in prioritizing research is unparalleled.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.241 | 0.383 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.002 |
| Science and technology studies | 0.014 | 0.014 |
| Scholarly communication | 0.021 | 0.018 |
| Open science | 0.006 | 0.032 |
| Research integrity | 0.005 | 0.005 |
| Insufficient payload (model declined to judge) | 0.008 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".