Developing and Implementing an Electronic Patient-Reported Outcomes Measurement Using REDCap in Usual Care Psychiatric Settings
Bibliographic record
Abstract
Background: Finding ways to feasibly and cost-efficiently measure patient-reported outcomes in psychiatric routine care settings is critical to facilitate patient engagement in care, improve patient outcomes, and assess the performance of our treatment programs. McLean Hospital is a free-standing psychiatric hospital that is part of the Partners HealthCare network and offers a full continuum of psychiatric care (inpatient, residential, partial hospital, and outpatient services) located on multiple campuses. We began an electronic patient-reported outcomes measurement program, the Clinical Measurement Initiative (CMI), in November 2010. Objective: To describe qualitatively and quantitatively the build and implementation experience of the McLean CMI program. Methods: The CMI aims to inform individual patient care, assist with quality assessment of clinical programs, and facilitate clinical research. On admission, discharge, and interim points, patients complete computerized self-assessments of validated clinical measures. The CMI uses Research Electronic Database Capture (REDCap), a free (but not open-source) secure Web application for building and managing online surveys. We designed a custom reporting module for individual patient reports to be available immediately at the point of care and an online aggregate reporting tool for clinical teams to track survey completion rates and outcomes. Clinical teams work closely with the CMI team to develop their CMI program/survey tools but receive no additional resources to accomplish survey administrations. We calculated descriptive statistics of admission and discharge survey administration and developed qualitative information about implementation “lessons learned” based on discussions with clinical teams that have implemented the CMI as part of the ongoing evaluation and monitoring of the program. Results: Over 20 programs representing 11 clinical psychiatric subpopulations have implemented the CMI to date; over 9000 episodes of care have been completed. Two programs (inpatient units) were unable to sustain the CMI in a way they found useful for clinical care and discontinued. Across active programs, 92% of admissions had admission assessments completed within 3 days of admission; 61% of discharges included a survey administration within 3 days of discharge. Clinical programs varied in the ability to successfully implement and sustain the CMI (eg, 69% of active CMI programs had >70% of admissions with admission surveys completed; over half accomplished >90% of admission surveys). Having at least one clinical champion at each program was a key driver for successful implementation. Champions served several needs: problem-solving successful new workflows, generating team enthusiasm, and setting team expectations for the importance of integrating the CMI information into clinical care. Teams that integrated the CMI into their clinical care with patients were also more successful in sustaining the CMI program. Conclusions: Achieving electronic patient reported outcomes measurement in intensive treatment psychiatric settings using REDCap and custom reporting tools is feasible but more easily accomplished in residential and partial hospital levels of care (compared to inpatient), where patient acuity is high but less severe and lengths of stay are longer. Clinical champions play critical roles in successful implementation and maintenance of electronic patient reported outcomes measurement and can be successful independent of program level of care or patient acuity.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.092 | 0.097 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.004 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.004 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".