Student and Faculty Perspectives on the Usefulness and Usability of a Digital Health Educational Tool to Teach Standardized Assessment of Persons After Stroke: Mixed Methods Study
Bibliographic record
Abstract
BACKGROUND: The VSTEP Examination Suite is a collection of evidence-based standardized assessments for persons after stroke. It was developed by an interdisciplinary team in collaboration with clinician users. It consists of 5 standardized assessments: 2 performance-based tests using the Kinect camera (Microsoft Corp) to collect kinematics (5-Time Sit-to-Stand and 4-Square Test); 2 additional performance-based tests (10-Meter Walk Test and 6-Minute Walk Test); and 1 patient-reported outcome measure, the Activities-Specific Balance Confidence Scale. OBJECTIVE: This study aimed to describe the development of the VSTEP Examination Suite and its evaluation as an educational tool by physical therapy students and faculty to determine its usefulness and usability. METHODS: A total of 6 students from a Doctor of Physical Therapy program in the United States and 6 faculty members who teach standardized assessments in different physical therapy programs from the United States and Israel were recruited by convenience sampling to participate in the study. They interacted with the system using a talk-aloud procedure either in pairs or individually. The transcripts of the sessions were coded deductively (by 3 investigators) with a priori categories of usability and usefulness, and comments were labeled as negative or positive. The frequencies of the deductive themes of usefulness and usability were tested for differences between faculty and students using a Wilcoxon rank sum test. A second round of inductive coding was performed by 3 investigators guided by theories of technology adoption, clinical reasoning, and education. RESULTS: The faculty members' and students' positive useful comments ranged from 83% (10/12) to 100%. There were no significant differences in usefulness comments between students and faculty. Regarding usability, faculty and students had the lowest frequency of positive comments for the 10-Meter Walk Test (5/10, 50%). Students also reported a high frequency of negative comments on the 4-Square Test (9/21, 43%). Students had a statistically significantly higher number of negative usability comments compared with faculty (W=5.7; P=.02), specifically for the 5-Time Sit-to-Stand (W=5.3; P=.02). Themes emerged related to variable knowledge about the standardized tests, value as a teaching and learning tool, technology being consistent with clinical reasoning in addition to ensuring reliability, expert-to-novice clinical reasoning (students), and usability. CONCLUSIONS: The VSTEP Examination Suite was found to be useful by both faculty and students. Reasons for perceived usefulness had some overlap, but there were also differences based on role and experience. Usability testing revealed opportunities for technology refinement. The development of the technology by interdisciplinary teams and testing with multiple types of users may increase adoption.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.050 | 0.075 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.004 | 0.003 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.005 | 0.003 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".