Feasibility of a Reinforcement Learning–Enabled Digital Health Intervention to Promote Mammograms: Retrospective, Single-Arm, Observational Study
Bibliographic record
Abstract
BACKGROUND: Preventive screenings such as mammograms promote health and detect disease. However, mammogram attendance lags clinical guidelines, with roughly one-quarter of women not completing their recommended mammograms. A scalable digital health intervention leveraging behavioral science and reinforcement learning and delivered via email was implemented in a US health system to promote uptake of recommended mammograms among patients who were 1 or more years overdue for the screening (ie, 2 or more years from last mammogram). OBJECTIVE: The aim of this study was to establish the feasibility of a reinforcement learning-enabled mammography digital health intervention delivered via email. The research aims included understanding the intervention's reach and ability to elicit behavioral outcomes of scheduling and attending mammograms, as well as understanding reach and behavioral outcomes for women of different ages, races, educational attainment levels, and household incomes. METHODS: The digital health intervention was implemented in a large Catholic health system in the Midwestern United States and targeted the system's existing patients who had not received a recommended mammogram in 2 or more years. From August 2020 to July 2022, 139,164 eligible women received behavioral science-based email messages assembled and delivered by a reinforcement learning model to encourage clinically recommended mammograms. Target outcome behaviors included scheduling and ultimately attending the mammogram appointment. RESULTS: In total, 139,164 women received at least one intervention email during the study period, and 81.52% engaged with at least one email. Deliverability of emails exceeded 98%. Among message recipients, 24.99% scheduled mammograms and 22.02% attended mammograms (88.08% attendance rate among women who scheduled appointments). Results indicate no practical differences in the frequency at which people engage with the intervention or take action following a message based on their age, race, educational attainment, or household income, suggesting the intervention may equitably drive mammography across diverse populations. CONCLUSIONS: The reinforcement learning-enabled email intervention is feasible to implement in a health system to engage patients who are overdue for their mammograms to schedule and attend a recommended screening. In this feasibility study, the intervention was associated with scheduling and attending mammograms for patients who were significantly overdue for recommended screening. Moreover, the intervention showed proportionate reach across demographic subpopulations. This suggests that the intervention may be effective at engaging patients of many different backgrounds who are overdue for screening. Future research will establish the effectiveness of this type of intervention compared to typical health system outreach to patients who have not had recommended screenings as well as identify ways to enhance its reach and impact.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".