The Inter-Tester and Test-Retest Reliability of the Off-Field SCAT6 Assessment Tool In An Adult Population
Bibliographic record
Abstract
OBJECTIVE: No previously published repeatability and reliability data for The Sports Concussion Assessment Tool-6 (SCAT6) exists. We aimed to evaluate inter/intra-tester reliability of the off-field SCAT6 in a non-concussed adult population. DESIGN: Inter-rater and Intra-rater reliability study design. SETTING: Single university site. PARTICIPANTS: Twenty active adults (mean age: 27.55 ± 5.59 years) with no recent history of concussion (Concussive injury within past year). INTERVENTIONS: Participants completed 3 SCAT6 tests on the same day, with 3 testers (Inter-rater testing). The same participants returned at 2 further time points to complete the remaining 2 SCAT6 tests with 1 tester (Intra-rater testing). Participants complete a total of 5 SCAT6 assessments in total across testers and time. Rater Background: Those completing the SCAT6 testing, our study rater team, comprised of 1 senior physiotherapist and PhD candidate, and 2 MSc Physiotherapy students. All raters were from Scotland, and had significant training in completing SCAT6 assessments. MAIN OUTCOME MEASURES: Off-field SCAT6 Domain scores. ANALYSIS: ICCs were used to establish inter and intra-rater reliability for continuous, ration and ordinal data components of the SCAT6. For nominal data sets, Fleiss's kappa was calculated. Kendall's W was used for non-parametric data. Percentage error scores were calculated for SCAT6 domains. RESULTS: Inter-tester : Symptom number, severity, and dual-task scoring demonstrated excellent reliability (ICC = 0.981; 0.984; 0.913, respectively). Total concentration score was found to have good reliability (0.827). Dual-task errors (0.398), Total mBESS (0.199), and Month recall all returned poor scores (k = 0.191). Intra-tester : Dual tasking was the only domain to report excellent reliability (ICC = 0.943). Symptom number (0.868), severity (0.831), total concentration (0.787), total mBESS (0.813), and time tandem gait (0.834) yielded good reliability scores. Dual-task error testing returned poor reliability scores (Kendall's W = 0.001). All remaining domains yielded moderate reliability. Percentage error rates ranges from 3% to 100%, demonstrating the variability between scores yielded for non-concussed individuals completing the same SCAT6 domain tests. CONCLUSION: SCAT6 ICC results reported good-excellent reliability for 4 and 6 domains, out of 13 domains, for inter-tester and intra-tester reliability, respectively. Notably, the domains which relied on tester error scoring yielded poor reliability results. Percentage error highlighted the failure of the SCAT6 to provide consistent domain score results in this population.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.018 | 0.033 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".