Theories of Expertise and Measures of Competence: Cognitive and Interactional Perspectives

Theories of Expertise and Measures of Competence: Cognitive and Interactional Perspectives Participants Carl H. Frederiksen Timothy Koschmann Dept. of Educ. & Counseling Psychology McGill University Carl.Frederiksen@mcgill.ca Dept. of Medical Education Southern Illinois University tkoschmann@siumed.edu Brian MacWhinney Colleen Seifert Dept. of Psychology Carnegie-Mellon University macw@cmu.edu Dept. of Psychology University of Michigan seifert@umich.edu Discussant Edward H. Shortliffe Dept. of biomedical Informatics Arizona State University edward.shortliffe@asu.edu solving. In these tests, applicants work up a series of clinical cases portrayed, for the purposes of the exam, by “standardized patients” (SPs). Applicants interview these patients, perform physical examinations, and document their findings in simulated chart notes. They are evaluated using a combination of behavioral checklists and categorical grades assigned by trained observers (NBME, 2005). Competence, then, is defined instrumentally as that which the assessments measure—little light is shed on its underlying cognitive and interactional processes. Further, behavioral checklists have been found to imperfectly reflect changing levels of expertise (Hodges et al., 1999). Such checklists would appear, therefore, to lack validity as a metric for professional competence. Though expert evaluators may be able to recognize expert performance when they see it, there are also problems with categorical ratings. Such ratings seek to break down the components of competent performance (e.g., “Questioning skills,” “Information-sharing,” “Professional manner and rapport”), but in the process, may lose the phenomenon of interest. In the clinical exam, the cognitive skills and the interactional work of gathering pertinent information, forming an appropriate clinical picture and communicating findings (both to the patient and fellow healthcare workers) are irremediably interdependent. The papers to be presented here are all based on a corpus of graded performance samples. These samples are modeled after the practical testing protocols used in high- stakes licensure exams. Each sample includes a video recording of the subject interviewing the SP and conducting a physical exam, the subject’s associated chart note, and a recording of a detailed debriefing interview. These structured samples allow comparisons of performance across different levels of clinical training. Drawing on diverse disciplinary backgrounds, the four presentations Abstract This symposium explores the relationship between expertise and competence, two terms used to describe skilled performance. Expertise has long been a foundational topic of inquiry in Cognitive Science (Chi, Glaser, & Farr, 1988; Ericsson, Charness, Feltovich & Hoffman, 2006). Research in this area has produced a much better understanding of how expertise develops at the highest levels of performance (Ericsson et al., 2006). It might seem reasonable that theories of expertise might inform the methods used to assess competence, but it is not clear that this has necessarily been the case. In contradistinction to expertise as studied in Cognitive Science, professional competency is a regulatory matter. Minimal standards of performance are established by certifying bodies and, in professions such as law and medicine, enforced by statute. The relationship between theories of expertise and what measures of competence actually measure has received relatively little attention in the past. Given its inherent complexity and vital importance to society, much of the research on expertise has been carried out within the domain of medicine (e.g., Patel, Kaufman, & Magder, 1996; Ericsson et al., 2006). Medicine is a highly regulated field and one in which there is a vital need to establish and maintain high standards of practice. State medical boards were established in the U.S. over a century ago to provide “the public a way to enforce basic standards of competence and ethical behavior in their physicians, and physicians a way to protect the integrity of their profession” (FSMB, n.d.). All medical boards require passing scores on the components of a nationally-administered licensing exam. Though most of this exam is based on conventional written tests, one component involves practical problem-

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.