Developing Culturally Competent Situational Judgement Tests for Indigenous Mentorship
Bibliographic record
Abstract
Background: The disparities in health outcomes between Indigenous and non-Indigenous populations in Canada are well-documented and persistent. One effective strategy to address these disparities is by increasing the number of Indigenous healthcare professionals, which can be facilitated through mentorship programs tailored to Indigenous students. This thesis proposes the development of culturally competent Situational Judgment Tests (SJTs) specifically designed to evaluate Indigenous Mentorship (IM) training modules that are being developed. Utilizing input from IM experts, the study aims to create SJTs that reflect the unique cultural contexts and needs of Indigenous mentees. Aim: To develop an effective evaluation tool designed to assess IM practices. Methods: The creation and refinement of SJT scenarios were conducted through focus groups with Subject Matter Experts (SMEs) in IM. Multiple phases of scenario development, response option generation, and consensus building were used to ensure cultural relevance and validity. After the focus groups, SME reactions were captured through a survey to gather insights on the evaluation development process and the tool’s utility. Results: Six SJTs were created, comprising 19 scenarios across six domains: Foster Indigenous Identity (3 scenarios), Abide by Indigenous Ethics (3 scenarios), Utilize Mentee-Centered Focus (3 scenarios), Advocacy (4 scenarios), Practice Relationalism (3 scenarios), and Imbue Criticality (3 scenarios). Following the development process, four out of six SMEs provided feedback. Most found the process enjoyable, noting the collaborative environment and the SJTs practical utility for IM. They also deemed it culturally appropriate for Indigenous stakeholders, though one SME highlighted challenges in aligning Indigenous concepts with Western frameworks. Conclusion: The study successfully created SJTs that align with core principles of IM. Survey feedback provided insights into the development process, emphasizing its appropriateness, practicality, and alignment with participants’ values. However, challenges in integrating Western methodologies with Indigenous contexts persist, highlighting the need for thoughtful adaptation in future efforts. The study concludes with a discussion of implications and potential directions for advancing IM through the use of SJTs.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.051 | 0.154 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.002 | 0.004 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".