The WorkingWell Mobile Phone App for Individuals With Serious Mental Illnesses: Proof-of-Concept, Mixed-Methods Feasibility Study
Bibliographic record
Abstract
BACKGROUND: The disparities in employment for individuals with serious mental illnesses have been well documented, as have the benefits of work. Mobile technology can provide accessible in-the-moment support for these individuals. The WorkingWell mobile app was developed to meet the need for accessible follow-along supports for individuals with serious mental illnesses in the workplace. OBJECTIVE: We explore the usability, usage, usefulness, and overall feasibility of the WorkingWell mobile app with individuals with serious mental illnesses who are actively employed and receiving community-based services. METHODS: In this proof-of-concept, mixed-methods, 2-month feasibility study (N=40), employed individuals with serious mental illnesses were recruited in mental health agencies. Participants completed surveys regarding background characteristics and cellphone use at enrollment and responded to interview items regarding app usability, usage, and usefulness in technical assistance calls at 1, 2, 4, and 6 weeks of participation and in the exit interview at 8 weeks. Data on the frequency of app usage were downloaded on a daily basis. A version of the System Usability Scale (SUS) was administered in the exit interview. Overall feasibility was determined by the percent of users completing the study, responses to an interview item regarding continued use, and findings on usability, usage, and usefulness. General impressions were obtained from users regarding user support materials, technical assistance, and study procedures. RESULTS: Most participants were male (60%, 24/40), aged 55 or younger (70%, 28/40), white (80%, 32/40), had less than a 4-year college education (78%, 31/40), were employed part-time (98%, 39/40), had been working more than 6 months (60%, 24/40), and indicated a diagnosis of bipolar, schizoaffective, or depressive disorder (84%, 16/25). The majority of participants owned cellphones (95%, 38/40) and used them multiple times per day (83%, 33/40). Their average rating on SUS usability items was 3.93 (SD 0.77, range 1.57-5.00), reflecting positive responses. In general, participants indicated WorkingWell was "very easy," "straightforward," "simple," and "user friendly." Usability challenges were related to personal issues (eg, memory) or to difficulties with the phone or app. Data on app usage varied considerably. The most frequent navigations were to the home screen, followed by Rate My Day and My Progress, and then by Manage the Moment and Remind Me. The app was described as useful by most participants; 86% (30/35) agreed the app would help them manage better on the job. Of the 40 original participants, 35 (87%) completed the study. CONCLUSIONS: The WorkingWell app is a feasible approach to providing accessible, as-needed employment support for individuals with serious mental illnesses. The app would benefit from modifications to address recommendations from feasibility testing. Controlled research with larger samples, more diverse in individual characteristics and workplace settings, is essential to demonstrating the effectiveness of the app.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.034 | 0.020 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".