Adaptation and Evaluation of a Symptom-Monitoring Digital Health Intervention for Patients With Relapsed and Refractory Multiple Myeloma: Pilot Mixed-Methods Implementation Study
Bibliographic record
Abstract
BACKGROUND: Relapsed and refractory multiple myeloma (RRMM) is a bone marrow cancer that requires systemic treatment, which often results in severe symptom burden. Recent studies have found that electronic patient-reported outcome (ePRO) interventions implemented in the clinic setting have had positive outcomes for other oncology populations. Evidence of the efficacy of a similar approach is lacking for patients with RRMM. OBJECTIVE: Recent recommendations for digital health interventions call for the publication of descriptions of iterative development processes in order to improve reproducibility and comparability. This study is an implementation pilot aiming to evaluate the acceptability and appropriateness of an ePRO intervention for patients with RRMM and to explore its impact on clinic workflow. METHODS: A total of 11 patients with RRMM were recruited from the John Theurer Cancer Center in Hackensack, New Jersey. Patients used a mobile app to report on 17 symptoms at 4 sessions, each a week apart. Patients could also report symptoms ad hoc. When reports met predefined thresholds, the clinic was alerted and patients received automated guidance. Study end points were assessed using qualitative and quantitative methods. RESULTS: A total of 9 patients (mean age 69.7 years) completed the study. Overall, 83% (30/36) of weekly sessions were completed. Patients found the frequency and time required to complete reporting acceptable. All patients agreed that the app was easy to use and understand. Providers felt the alerts they received required refinement. Patients and providers agreed it would be beneficial for patients to report for longer than 4 weeks. Patients felt that the training they received was adequate but contained too much information for a single session. All patients found the symptoms tracked to be appropriate; providers suggested shortening the list. All patients understood how to use the app for weekly reporting but had confusion about using it ad hoc. Providers felt the ad hoc feature could be removed. Neither patients nor providers viewed the in-app data reports but agreed on their potential value. Patients reported benefitting from symptom reporting through increased awareness of their symptoms. Clinic staff reported that app alerts were too numerous and redundant. They had difficulty responding to alerts within their existing workflow, partially because the data were not integrated into the electronic medical record system. CONCLUSIONS: Overall, the intervention was found to be acceptable and appropriate for patients with RRMM. Points of friction integrating the intervention into the clinic workflow were identified. Clinic staff provided recommendations for addressing these issues. Once such modifications are implemented, ePRO data from patients with RRMM could be used to inform and improve clinical research and care. This study underlines the importance of an iterative approach to implementation that includes all stakeholders in order to ensure successful adoption.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.026 | 0.025 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".