Development and Evaluation of an Item Pool of “Movement-Related Body Functions in the Context of Task Performance”
Bibliographic record
Abstract
OBJECTIVE: The study aimed to describe the development and evaluation of an item pool for a new performance-based clinical outcome assessment (COA), the Spinal Cord Injury Movement Index (SCI-MovIn). DESIGN: Iterative focus groups, one-on-one consultations and field-testing sessions were used to develop the conceptual model for the SCI-MovIn measurement construct, establish an item pool consisting of candidate items with 5 response categories, and create standardized guidelines for set-up, administration, and scoring. Trained therapists administered 61 items to individuals with spinal cord injury (SCI) for inter- and intrarater reliability testing. SETTING: Academic institution in an urban area. PARTICIPANTS: Professionals with measurement/SCI expertise participated in focus groups. Individuals with SCI engaged in field-testing and reliability testing sessions. MAIN OUTCOME MEASURE(S): Total percent exact agreement between paired raters was calculated. Cronbach's alpha (α) and intraclass correlation coefficient (ICC) with 95% confidence interval (CI) were used to examine internal consistency. Intra- and interrater reliability were measured using ICCs. RESULTS: Through iterative focus groups with 18 SCI/measurement professionals and 24 field-testing sessions with 10 individuals with SCI, 226 items were developed, 132 of which were eliminated. The reliability testing sample consisted of 33 adults who sustained SCI an average of 13.2 years before participation. Of the 1215 paired scores from administration of 61 SCI-MovIn items, raters' scores were identical for 833 (68.6%). Internal consistency was high (α=0.948). Reliability of repeated SCI-MovIn scores was high for both intrarater (ICC=0.992; CI=0.983-0.996) and interrater (ICC=0.887; CI=0.873-0.899) reliability. CONCLUSION: The next step is a large-scale study to transform the item pool into a calibrated item bank from which tailored short forms can be developed.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.013 | 0.022 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".