Use of portfolios for assessment of global health residents: qualitative evaluation of design and implementation
Bibliographic record
Abstract
BACKGROUND: When the Global Health training program was created at the University of Calgary, residents were encouraged to seek learning experiences that met their career goals and individualized objectives. An assessment tool was sought that could be reliable, valid, yet flexible. A portfolio process was chosen, but research was necessary to determine whether it was robust. METHODS: A qualitative study was conducted with academic experts in Canadian residency training, as well as directors and residents involved in Global Health study in order to assess the validity and benefit of such a tool. Through an online survey, interviews, and focus groups, views on the portfolio and intended content were collected and coded thematically. RESULTS: Multiple themes emerged from the content analysis. Overall, all stakeholders (residents and faculty) were supportive of the use of portfolios for summative assessment, mentioning authentic and varying assessments, reflective and narrative components, and mentor interaction as positive attributes, but they did have many recommendations. CONCLUSION: This qualitative evaluation validated the use of portfolios for this cohort of students while yielding comments and suggestions that will further enhance the interactive and flexible nature of this seldom used assessment tool. These findings contribute to the understanding of how Global Health assessment can remain individualized yet rigorous.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.160 | 0.142 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.005 | 0.006 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.003 | 0.007 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".