Health Care Providers and the Public Reporting of Nursing Home Quality in the United States Department of Veterans Affairs: Protocol for a Mixed Methods Pilot Study
Bibliographic record
Abstract
BACKGROUND: In June 2018, the United States Department of Veterans Affairs (VA) began the public reporting of its 134 Community Living Centers' (CLCs) overall quality by using a 5-star rating system based on data from the national quality measures captured in CLC Compare. Given the private sector's positive experience with report cards, this is a seminal moment for stimulating measurable quality improvements in CLCs. However, the public reporting of CLC Compare data raises substantial and immediate implications for CLCs. The report cards, for example, facilitate comparisons between CLCs and community nursing homes in which CLCs generally fare worse. This may lead to staff anxiety and potentially unintended consequences. Additionally, CLC Compare is designed to spur improvement, yet the motivating aspects of the report cards are unknown. Understanding staff attitudes and early responses is a critical first step in building the capacity for public reporting to spur quality. OBJECTIVE: We will adapt an existing community nursing home public reporting survey to reveal important leverage points and support CLCs' quality improvement efforts. Our work will be grounded in a conceptual framework of strategic orientation. We have 2 aims. First, we will qualitatively examine CLC staff reactions to CLC Compare. Second, we will adapt and expand upon an extant community nursing home survey to capture a broad range of responses and then pilot the adapted survey in CLCs. METHODS: We will conduct interviews with staff at 3 CLCs (1 1-star CLC, 1 3-star CLC, and 1 5-star CLC) to identify staff actions taken in response to their CLCs' public data; staff's commitment to or difficulties with using CLC Compare; and factors that motivate staff to improve CLC quality. We will integrate these findings with our conceptual framework to adapt and expand a community nursing home survey to the current CLC environment. We will conduct cognitive interviews with staff in 1 CLC to refine survey items. We will then pilot the survey in 6 CLCs (2 1-star CLCs, 2 3-star CLCs, and 2 5-star CLCs) to assess the survey's feasibility, acceptability, and preliminary psychometric properties. RESULTS: We will develop a brief survey for use in a future national administration to identify system-wide responses to CLC Compare; evaluate the impact of CLC Compare on veterans' clinical outcomes and satisfaction; and develop, test, and disseminate interventions to support the meaningful use of CLC Compare for quality improvement. CONCLUSIONS: The knowledge gained from this pilot study and from future work will help VA refine how CLC Compare is used, ensure that CLC staff understand and are motivated to use its quality data, and implement concrete actions to improve clinical quality. The products from this pilot study will also facilitate studies on the effects of public reporting in other critical VA clinical areas. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/23516.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.076 | 0.053 |
| Meta-epidemiology (narrow) | 0.003 | 0.004 |
| Meta-epidemiology (broad) | 0.004 | 0.004 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.006 | 0.003 |
| Scholarly communication | 0.005 | 0.004 |
| Open science | 0.003 | 0.004 |
| Research integrity | 0.004 | 0.008 |
| Insufficient payload (model declined to judge) | 0.060 | 0.013 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".