Patient Rating Sites for Daily Supervision by Healthcare Inspectorates: Implementation into Practice
Bibliographic record
Abstract
Background: Social media and especially patient rating sites (PRS’s) have shown to be an interesting new source of information about quality of care from the patient’s perspective. Several studies indicate a relationship between information on social media and quality of health care including patients’ experiences, mortality ratio’s, readmission rates and infection rates. Therefore, information on PRS’s could have added value in supervision the quality for care by official supervising bodies such as a healthcare inspectorate. Objective: To implement a system in which relevant information about the quality of care is efficiently identified and extracted from PRS’s and presented to inspectors by adding it to the risk judgment system for day-to-day supervision. Methods: The study consisted of three parts: (1) Exploration based on expert opinion by supervision experts of the Dutch Healthcare Inspectorate (DHI) of the added value for individual reviews with a poor rating (< 7 on a scale from 1-10) by making use of pre-developed scales. (2) Investigation of the opportunities for preselecting information by DHI researchers by scoring reviews in duos in order to test interrater agreement. (3) Designing a process description with all relevant stakeholders to create a realistic implementation path. Results: For 72 of 116 cases in supervision of long-term elderly care on four major risk themes (medication safety, hygiene, expertise and restriction of freedom) information was considered to be relevant. Preselecting information from PRS’s showed acceptable agreement for four out of five researchers. Based on these results we designed a process description of adopting PRS data into the risk database of the DHI for long-term elderly care by using a File Transfer Protocol, extracting data from the PRS. Starting from June 1st 2015 the DHI inspectors will receive information about long-term elderly care organisations of the major Dutch PRS, next to other quality and safety indicators. Conclusions: The results show that PRS’s could be used to include the patient’s perspective in day-to-day supervision. Important conditions are sufficient number of reviews and enthusiastic inspectors. These findings indicate that PRS’s may enable supervisory bodies to include the patients’ perspective in an efficient way. Future research should explore the opportunities of other healthcare sectors and other social media such as Twitter en Facebook.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.157 | 0.167 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.004 | 0.005 |
| Open science | 0.004 | 0.007 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".