Evaluating sport-for-development outcome measures used in a living lab setting: Process, improvements, and insights
Bibliographic record
Abstract
BACKGROUND: Sport-for-development (SFD) is an innovative approach utilizing sport to foster positive physical, mental, and social outcomes among children and youth, particularly those from underserved backgrounds. Living labs, which emphasize participant-centered research conducted in natural, real-world environments, present unique challenges for outcome measurement, including reduced control over conditions, variability in participant engagement, and logistical issues that complicate standardized data collection. Further, there are few outcome measures that are developed for SFD measurement in living lab settings. For these reasons, outcome measurement in a living lab setting remains challenging. OBJECTIVE: Our objective was to evaluate a set of outcome measures that have been administered in a living lab setting to better understand their performance, reliability, and areas for improvement. METHODS: SFD programming was delivered in a living lab setting at a large facility located in an urban center in Toronto, Canada. We evaluated 11, self-reported, Likert-style outcome measures against 8 key metrics used in Classical Test Theory to understand (for example) floor-and-ceiling effects, inter-item correlations, internal consistency, and test-retest reliability. Data were collected from 2019 to 2024 across multiple cohorts aged 6-29 years, involving diverse SFD programs. RESULTS: Our analysis of 2656 questionnaire completions demonstrated strengths in data collection, including complete response rates with minimal missing data (91 % of outcome measures met missingness thresholds), yet also highlighted issues primarily related to single-item-endorsement and inter-item correlations (with 38 % and 19 % of outcome measures meeting these thresholds, respectively). These insights prompted iterative improvements to the evaluation tools, such as modifying Likert scale response formats to include more response categories (and thereby reducing the impact of binning of responses). CONCLUSIONS: Evaluating our outcome measures provided insight into how they can be improved for administration in a living-lab setting. The results emphasize the need for context-appropriate tools to effectively capture nuanced SFD program impacts and underscore the importance of ongoing validation to improve both research quality and practical implementation in living lab environments.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.074 | 0.076 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".