Competency assessors’ cognitive map of practice when assessing practice based encounters
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Introduction: There is growing evidence that inconsistencies exist in how competencies are conceptualised and assessed. Aim: This study aimed to determine the reliability of pharmacist assessors when observing practice-based encounters and to compare and contrast assessors’ cognitive map of practice with the guiding competency framework. Methods: This was a qualitative study with verbal protocol analysis. A total of 25 assessors were recruited to score and verbalise their assessments for three videos depicting practice-based encounters. Verbalisations were coded according to the professional competency framework. Results: Protocols from 24 participants were included. Interrater reliability of scoring was excellent. Greater than 75% of assessment verbalisations were focused on 3 of the 27 competencies: communicate effectively, consults with the patient, and provide patient counselling. Conclusion: Findings support the notion that assessment completed within practice could be largely informed by a single component of the interaction or more specifically, what ‘catches the eye’ of the assessor.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.015 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it