Capacity development in patient-oriented research: programme evaluation and impact analysis
Bibliographic record
Abstract
BACKGROUND: National and provincial funding was invested to increase the quantity and quality of patient-oriented research (POR) across Canada. Capacity development became a priority to ensure all stakeholders were prepared to engage in POR. In part, this need was met through an annual Studentship competition in the province of Alberta, providing funding to students whose research incorporated principles of POR. However, despite efforts to build capacity in the health research trainee population, little is known about the outcomes of these programmes. This evaluation study examined the outcomes of a POR capacity development programme for health research trainees. METHODS: Final impact narrative reports were submitted by the 21 Studentship programme awardees for 2015 and 2016 who represent a variety of health disciplines across three major research universities. The reports describe the programme outcomes as well as the overall impact on individual, project and professional development as POR trainees. A synthesis of structured and categorised report data was conducted, along with additional qualitative analyses as new themes emerged that were not apparent in the competency framework utilised in the programme design. RESULTS: Awardee reports detailed the impact of the Studentship programme on the key themes of increased knowledge and skill, relationship building, confidence and leadership, as well as project and career impact. The impacts felt most profoundly by the awardees were not reflective of the competencies that guided programme design. The outcomes were then re-examined using a health research capacity development framework to gain a more comprehensive view of programme impact. CONCLUSION: The Studentship programme narratives provided insight into the rarely tracked capacity development outcomes of POR research trainees. Awardee narratives indicated significant development beyond the intended competencies and suggested a need to revisit the competency framework for POR in Alberta. While competencies were useful in guiding the design of the initial programme, a more comprehensive capacity development framework was required to capture the broader impacts on trainee development. Future capacity development programmes may benefit from these early programme insights, specifically the need for more robust competencies for POR. Further exploration of evaluation methods for short-term awards and sustainability of capacity development programmes is warranted.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | Metaresearch Domain: Evaluation · Genre: Empirical About the Canadian research system: no · About a Canadian topic: no | Observational | low |
| gpt | Metaresearch Domain: Evaluation · Genre: Empirical About the Canadian research system: no · About a Canadian topic: no | Observational | high |
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.038 | 0.003 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.002 | 0.004 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.002 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedLabeled directly by 2 models reading the full record.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".