A Communication-Based Assessment Station as an Alternative to the OSCE

To the Editor: The objective structured clinical examination (OSCE) is the gold standard in clinical performance examination, but its administration is expensive, requiring training of standardized patients and evaluators. 1 With recent centralization of clinical performance assessment at our institution into a high-stakes “integrated OSCE,” the Department of Psychiatry modified its psychiatry-specific examination by replacing two OSCE stations with a communicator skills assessment (ComSA) station. In ComSA stations, the examiner role-plays scenarios such as answering a family member’s questions or consulting an allied health professional. The ComSA station appears to be a favorable option, given its lower cost and logistical simplicity. Yet, literature on examiner roleplay in performance-based assessments is limited. 2 In 2014, we sought evidence for our department’s use of a ComSA station in place of an OSCE station. Using 3 random iterations of our revised psychiatry clerkship examination, we examined 126 examinees’ scores on four examination components: process scores (communication skills), content scores (station-specific knowledge), post-encounter probe (PEP) for OSCE stations, and the psychiatry written examination. We conducted a paired-samples test of equivalence to assess whether performance on ComSA stations was equivalent to that on OSCE stations. Our findings demonstrate that the performances on the knowledge domains (content scores, PEPs, and the written exam) are equivalent. The process domains on the ComSA and OSCE are also equivalent; however, the ComSA process score and the written examination score are not equivalent measurements. Using Cronbach’s alpha omitted-variable analysis, we determined the impact of the replacement on overall examination performance. 3 The internal consistency of two OSCE stations and one ComSA station was similar to and often higher than that of three OSCE stations. This finding strongly suggests that a ComSA station can replace an OSCE station without adversely impacting the exam’s internal reliability. Overall, our results demonstrate that our ComSA station and OSCE stations assessed learner performance similarly and that the former can be integrated into an exam without decreasing the exam’s validity. Study limitations include possible curricular variations between iterations. Furthermore, the equivalence of the ComSA station to OSCE stations and the former’s acceptability in other specialities, such as surgery, should be examined. The realities of medical education require that we consider not only a method’s rigor but also its feasibility and sustainability. Our findings are promising, as they suggest that lower-cost alternatives can be validated and easily implemented into existing assessment frameworks without reducing assessment quality. Acknowledgments: The authors would like to thank Tammy Mok for her assistance in data extraction.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.