A structured communication adolescent guide (SCAG): assessment of reliability and validity
Bibliographic record
Abstract
PURPOSE: To assess the reliability and validity of a Structured Communication Adolescent Guide (SCAG) in an undergraduate medical education setting using trained adolescent raters. METHOD: The SCAG is a 49-item, 6-section (A-F), protocol derived from the Calgary Cambridge Observation Guide that incorporates adolescent psychosocial data collection with the physician's communication skills. Four trained female adolescents scored 42 videotaped adolescent clinical encounters using the SCAG; a trained psychologist's rating for each videotape was used as the gold standard. RESULTS: Agreement among adolescent raters was determined by calculating intraclass correlation coefficients (ICC). The individual SCAG item scores, combined with the global ratings for each section, resulted in an overall ICC value of 0.93, indicating very strong agreement among the 4 raters. The global rating scores for the sections ('initiating the session', 'initiating separation', 'once adolescent is alone--lifestyle' and 'closure') produced an ICC range of 0.58-0.93. However, the ICC values for the 2 remaining sections ('how was information collected' and 'gathering information') global rating scores were below 0.30, signifying low agreement. Overall agreement between the adolescent raters and the gold standard resulted in an ICC value of 0.78. This is evidence of the SCAG's criterion validity. CONCLUSION: The SCAG is a reasonably valid tool for use in guiding an encounter with an adolescent patient. However, 2 sections require modifications to improve their reliability and thus the SCAG's overall performance. Our results suggest that the SCAG shows promise as a potentially useful teaching resource in undergraduate medical education in adolescent medical interviewing.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.004 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".