Validation of the SPARK Tool to collect demographic and social needs data in healthcare
Bibliographic record
Abstract
Context: Despite evidence that social determinants (e.g. income, housing) affect health, there is no standardized tool or approach in Canada for the routine collection of demographic and social information in healthcare settings. Objective: To validate the SPARK Tool using concurrent validity by assessing agreement between the SPARK Tool and well-established national survey questions (e.g. census) that are currently used for policy decisions and to identify health inequities. Study Design and Analysis: We distributed an online survey across Canada that included the SPARK Tool and a post-survey questionnaire. We used contingency tables to assess agreement. Setting or Dataset: The survey was distributed to Canadians in all provinces and territories. Population Studies: The survey was distributed from June 2023 to October 2023 using Qualtrics, a company that maintains a large panel of individuals to engage in survey research, and through social media. We ensured representativeness of the sample to the Canadian population. Intervention/Instrument: The SPARK Tool contains 18 questions including demographic questions and social needs questions, and the post-survey questionnaire contains 11 questions. Outcome Measures: We assessed associations between the SPARK Tool and post-survey questionnaire using sensitivity, specificity, Positive Predictive Value and Negative Predictive Value. We assessed correct classification represented by combined True Positives and True Negatives. Results: There were 2,222 participants that completed the SPARK Tool and post-survey questionnaire. Participants represented all provinces and territories, with the majority in Ontario (40.5%). The SPARK Tool correctly classified 74.3% of participants with or without a need. In total, 1,610 participants (72.3%) had ≥1 access or social need identified, with the majority (64.9%, n=1443) indicating at least one social need. Agreement varied within the domains of the SPARK Tool. Conclusions: The SPARK Tool performed reasonably well with a large diverse sample, demonstrating comparable classification with well-established national survey questions. The SPARK Tool presents a validated tool which can serve as a standard for the systematic and routine collection of demographic and social needs data in health care.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.096 | 0.137 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.005 | 0.005 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.003 | 0.005 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.007 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".