Impact of Social Determinants of Health and Demographics on Refill Requests by Medicare Patients Using a Conversational Artificial Intelligence Text Messaging Solution: Cross-Sectional Study
Bibliographic record
Abstract
Background Nonadherence among patients with chronic disease continues to be a significant concern, and the use of text message refill reminders has been effective in improving adherence. However, questions remain about how differences in patient characteristics and demographics might influence the likelihood of refill using this channel. Objective The aim of this study was to evaluate the efficacy of an SMS-based refill reminder solution using conversational artificial intelligence (AI; an automated system that mimics human conversations) with a large Medicare patient population and to explore the association and impact of patient demographics (age, gender, race/ethnicity, language) and social determinants of health on successful engagement with the solution to improve refill adherence. Methods The study targeted 99,217 patients with chronic disease, median age of 71 years, for medication refill using the mPulse Mobile interactive SMS text messaging solution from December 2016 to February 2019. All patients were partially adherent or nonadherent Medicare Part D members of Kaiser Permanente, Southern California, a large integrated health plan. Patients received SMS reminders in English or Spanish and used simple numeric or text responses to validate their identity, view their medication, and complete a refill request. The refill requests were processed by Kaiser Permanente pharmacists and support staff, and refills were picked up at the pharmacy or mailed to patients. Descriptive statistics and predictive analytics were used to examine the patient population and their refill behavior. Qualitative text analysis was used to evaluate quality of conversational AI. Results Over the course of the study, 273,356 refill reminders requests were sent to 99,217 patients, resulting in 47,552 refill requests (17.40%). This was consistent with earlier pilot study findings. Of those who requested a refill, 54.81% (26,062/47,552) did so within 2 hours of the reminder. There was a strong inverse relationship (r10=−0.93) between social determinants of health and refill requests. Spanish speakers (5149/48,156, 10.69%) had significantly lower refill request rates compared with English speakers (42,389/225,060, 18.83%; X21 [n=273,216]=1829.2; P<.001). There were also significantly different rates of refill requests by age band (X26 [n=268,793]=1460.3; P<.001), with younger patients requesting refills at a higher rate. Finally, the vast majority (284,598/307,484, 92.23%) of patient responses were handled using conversational AI. Conclusions Multiple factors impacted refill request rates, including a strong association between social determinants of health and refill rates. The findings suggest that higher refill requests are linked to language, race/ethnicity, age, and social determinants of health, and that English speakers, whites, those younger than 75 years, and those with lower social determinants of health barriers are significantly more likely to request a refill via SMS. A neural network–based predictive model with an accuracy level of 78% was used to identify patients who might benefit from additional outreach to narrow identified gaps based on demographic and socioeconomic factors.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | no category Domain: not available · Genre: Empirical About the Canadian research system: no · About a Canadian topic: no | Observational | low |
| gpt | no category Domain: not available · Genre: Empirical About the Canadian research system: no · About a Canadian topic: no | Observational | low |
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.008 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedLabeled directly by 2 models reading the full record.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".