Achieving reliable patient reported outcomes collection to measure health care improvement in a learning health network: lessons from pediatric rheumatology care and outcomes improvement network
Bibliographic record
Abstract
Introduction: Data from the Pediatric Rheumatology Care and Outcomes Improvement Network (PR-COIN) registry suggests that reliable collection of patient-reported outcomes (PROs) varies across sites. The objective of this study was to better understand the practices of collecting PROs at PR-COIN sites. Methods: A REDCap survey was sent to the lead representative for each PR-COIN site. Registry data were analyzed to better understand the completion rates of PROs. Interviews of physician leaders of high performing sites were conducted by videoconference, audiotranscribed and themes were summarized. Quantitative data were analyzed using descriptive statistics and qualitative data were thematically analyzed. Results: All 23 PR-COIN sites responded to the survey. PROs were collected by 21/23 (91%) sites. Arthritis-related pain intensity, morning stiffness, and physical function were the top three collected PROs ( Supplementary 3 and 4). PROs were collected using paper, electronically or in combination, with most sites collecting PROs only on paper. PROs were manually scored at most sites. Among sites with electronic PRO collection, 42% did not have automatic transfer of scores into the electronic medical record. Facilitators to successful collection of PROs included availability of staff, training, and culture. Barriers to PRO collection cited were limited time, lack of infrastructure, and lack of staff. Completion rates of PROs in the registry in top 4 performing centers for morning stiffness was 100%, overall well-being and pain intensity scores ranged from 93%-98%, and for physical function 69%-94%. Interviews with physician leaders indicated that their site overcame barriers through: integration of PRO collection into workflow, gaining buy-in of stakeholders (clinicians and patients), and automating PRO collection. Interviewees endorsed automation of data collection (e.g., self-completion on tablets) and automated transfer to electronic medical record (EMR) as key components enabling reliable PRO collection. Conclusions: Through understanding our current ability to systematically collect PROs across all sites in PR-COIN and exploring successful implementation of PRO collection both within and outside our learning health network, we share lessons learned and identify the most influential factors for successful PRO collection in pediatric rheumatology.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.113 | 0.176 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.005 | 0.008 |
| Open science | 0.004 | 0.008 |
| Research integrity | 0.001 | 0.006 |
| Insufficient payload (model declined to judge) | 0.005 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".