Development and validation of a situational judgment test for assessing interprofessional education facilitators: a design-based research
Bibliographic record
Abstract
This study developed and validated a situational judgment test (SJT) designed to assess the teaching abilities of interprofessional education (IPE) teachers, with specific alignment to the Canadian Interprofessional Health Collaborative (CIHC) framework. Using a design-based research (DBR) approach, I implemented iterative cycles of design, feedback, and refinement, engaging IPE experts and teachers to ensure content validity and practical relevance. The study addressed how effectively the SJT could evaluate IPE teachers' teaching abilities while maintaining alignment with CIHC framework principles.Through a systematic literature review, expert consultation, cognitive interviews, and think-aloud protocols, we developed scenarios that captured authentic interprofessional challenges and educational priorities. The iterative design process significantly contributed to the SJT's content validity, ensuring scenarios and response options closely aligned with CIHC competencies essential for effective IPE facilitation. Analysis of cognitive interviews and think-aloud protocols confirmed that the scenarios accurately reflected teachers' decision-making processes in complex interprofessional situations, effectively representing real-world teaching challenges.The findings revealed three primary outcomes. First, the iterative design process enhanced the SJT's content validity, creating strong alignment between assessment items and CIHC competencies. Second, validation through think aloud protocols and cognitive interviews demonstrated the scenarios' effectiveness in capturing teachers' decision-making processes when addressing complex interprofessional challenges. Third, feedback from IPE teachers indicated high acceptability of the SJT, with participants affirming its potential as both an evaluative and developmental tool for enhancing IPE facilitation skills.Despite these strengths, the study encountered limitations regarding sample size, cultural context variability, and the complexity of developing nuanced response options. These challenges highlighted opportunities for future research, particularly in adapting the SJT for diverse cultural contexts and expanding its application across different healthcare disciplines.The study's implications extend beyond immediate assessment applications, contributing to the broader field of interprofessional health education. The SJT represents a significant advance in faculty development, offering a structured method to evaluate and enhance teaching abilities within interprofessional contexts. Its alignment with the CIHC framework emphasizes core competencies essential for effective teamwork and collaboration in healthcare, supporting the global movement toward integrated, patient-centered care delivery.This research lays the groundwork for future developments in IPE teachers evolution and training, suggesting several directions for continued investigation, including cross-professional adaptations, cultural sensitivity considerations, and longitudinal impact studies. The SJT's potential for standardizing teacher assessment across institutions contributes to international efforts in establishing consistent criteria for evaluating IPE facilitation, ultimately supporting the development of healthcare professionals better prepared for collaborative practice.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.098 | 0.188 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.004 | 0.002 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.003 | 0.003 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".