A reflective learning framework to evaluate CME effects on practice reflection
Bibliographic record
Abstract
INTRODUCTION: The importance of reflective practice is recognized by the adoption of a reflective learning model in continuing medical education (CME), but little is known about how to evaluate reflective learning in CME. Reflective learning seldom is defined in terms of specific cognitive processes or observable performances. Competency-based evaluation rarely is used for evaluating CME effects. To bridge this gap, reflective learning was defined operationally in a reflective learning framework (RLF). The operationalization supports observations, documentation, and evaluation of reflective learning performances in CME, and in clinical practice. In this study, the RLF was refined and validated as physician performance was evaluated in a CME e-learning activity. METHODS: Qualitative multiple-case study wherein 473 practicing family physicians commented on research-based synopses after reading and rating them as an on-line CME learning activity. These comments formed 2029 cases from which cognitive tasks were extracted as defined by the RLF with the use of a thematic analysis. Frequencies of cognitive tasks were compared in a cross-case analysis. RESULTS: Four RLF cognitive processes and 12 tasks were supported. Reflective learning was defined as 4 interrelated cognitive processes: Interpretation, Validation, Generalization, and Change, which were specified by 3 observable cognitive tasks, respectively. These 12 tasks and related characteristics were described in an RLF codebook for future use. DISCUSSION: Reflective learning performances of family physicians were evaluated. The RLF and its codebook can be used for integrating reflective learning into CME curricula and for developing competency-based assessment. Future research on potential uses of the RLF should involve participation of CME stakeholders.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.095 | 0.127 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.004 |
| Bibliometrics | 0.014 | 0.007 |
| Science and technology studies | 0.003 | 0.012 |
| Scholarly communication | 0.008 | 0.009 |
| Open science | 0.003 | 0.008 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".