Tailoring Rheumatoid Arthritis Visit Timing Based on mHealth App Data: Mixed Methods Assessment of Implementation and Usability
Bibliographic record
Abstract
Background: Visits to medical subspecialists are common, with follow-up timing often based on heuristics rather than evidence. Unnecessary visits contribute to long wait times for new patients. Specialists could enhance visit timing and reduce frequency by systematically monitoring patients' symptoms between visits, especially for symptom-driven conditions like rheumatoid arthritis (RA). We previously designed an intervention using a mobile health (mHealth) app to collect patient-reported outcomes (PRO). One of several aims of the app was to assist rheumatologists in determining visit timing for patients with RA. The intervention did not reduce visit frequency. Objective: To explore possible reasons for the lack of association between the intervention and visit frequency, we describe app usage, assess usability, and identify barriers and facilitators for using between-visit PRO data to reduce visits when patients' symptoms are stable. Methods: We analyzed patients' use of the app by reporting adherence (percent of PRO questionnaires completed during the 12-month study) and retention (use in the last month of the study). To examine rheumatologists' experiences, we summarized views of the electronic health record (EHR)-embedded PRO dashboard and EHR inbox messages suggesting early or deferred visits. We assessed app usability using the interactive mHealth App Usability Questionnaire for Ease of Use and Usefulness for patients and the System Usability Scale for rheumatologists. We assessed rheumatologist-level effects of intervention usage using Kruskal-Wallis rank sum and equality of proportion tests. We identified barriers and facilitators through interviews and surveys. Results: The analysis included 150 patients with RA and their 11 rheumatologists. Patients answered a median of 53.3% (IQR 34.1%-69.2%) of PRO questionnaires; this proportion varied by rheumatologist (range 40.7%-67%). Over half of the patients used the app during the final month of the study (56%, range 51%-65%, by rheumatologists); the median number of months of use was 12 (IQR 9-12). Rheumatologists viewed the dashboard 78 times (17.6% of 443 visits) with significant differences in viewing rates by rheumatologist (range 10%-66%; P<.01). There were 108 generated messages sent to rheumatologists suggesting a deferred visit (24.4% of 443 visits) with significant differences in message counts received per visit by rheumatologist (range 10.8%-22.6%; P=.03). Rheumatologists' reported barriers to offering visit deferrals included already scheduling as far out as they were comfortable and rescheduling complexities for staff. Based on 39 patient interviews and 44 surveys, patients reported 2 main barriers to app usage: questionnaire frequency not being tailored to them and reduced motivation after not discussing PRO data with their rheumatologist. A total of 5 interviewed patients received the option to defer their visits, of which 3 elected to defer the appointment and 2 chose to keep it. Conclusions: While an mHealth app for reporting RA PROs was used frequently by patients, using these data to reduce the frequency of unneeded visits was not straightforward. Better engagement of clinicians may improve the use of PRO data.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.119 | 0.124 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.004 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".