Evaluation of a Mobile Device Survey System for Behavioral Risk Factors (SHAPE): App Development and Usability Study
Bibliographic record
Abstract
BACKGROUND: Risk factors, including limited exercise, poor sleep, smoking, and alcohol and drug use, if mitigated early, can improve long-term health. Risk prevalence has traditionally been measured using methods that now have diminished participation rates. With >75% of American citizens owning smartphones, new data collection methods using mobile apps can be evaluated. OBJECTIVE: The objective of our study was to describe the development, implementation, and evaluation of a mobile device-based survey system for behavioral risk assessment. Specifically, we evaluated its feasibility, usability, acceptability, and validity. METHODS: We enrolled 536 students from 3 Vermont State Colleges. Iterative mobile app development incorporated focus groups, extensive testing, and the following 4 app versions: iOS standard, iOS gamified, Android standard, and Android gamified. We aimed to capture survey data, paradata, and ambient data such as geolocation. Using 3 separate surveys, we asked a total of 27 questions that included demographic characteristics, behavioral health, and questions regarding the app's usability and survey process. RESULTS: Planned enrollment was exceeded in just a few days. There were 1392 "hits" to the landing page where the app could be downloaded. Excluding known project testers and others not part of the study population, 670 participants downloadeded the SHAPE app. Of those, 94.9% of participants (636/670) agreed to participate by providing in-app consent. Of the 636 who provided consent, 84.3% (536/636) were deemed eligible for the study. The majority of eligible respondents completed the initial survey (459/536, 85.6%), whereas 29.9% (160/536) completed the second survey and 28.5% (153/536) completed the third survey. The SHAPE survey obtained 414 participants on the behavioral risk items in survey 1, which is nearly double the 209 participants who completed the traditional Vermont College Health Survey in 2014. SHAPE survey responses were consistent with the traditionally collected Vermont College Health Survey data. CONCLUSIONS: This study provides data highlighting the potential for mobile apps to improve population-based health, including an assessment of recruitment methods, burden and response rapidity, and future adaptations. Although gamification and monetary rewards were relatively unimportant to this study population, item response theory may be technologically feasible to reduce individual survey burden. Additional data collected by smartphones, such as geolocation, could be important in additional analysis, such as neighborhood characteristics and their impact on behavioral risk factors. Mobile tools that offer rapid adaptation for specific populations may improve research data collection for primary prevention and could be used to improve engagement and health outcomes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.018 | 0.026 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".