Quality indicators for improved cardiovascular care: learnings from the National Institute for Health and Care Excellence
Bibliographic record
Abstract
The necessity for improvement in the quality of care has led to increased attention to the measurement and reporting of processes of care and patient outcomes. The European Society of Cardiology (ESC) recognizes this and has developed quality indicators (QIs) to evaluate the management of a range of common cardiovascular conditions,1–11 based upon a standardized methodology.12,13 This work built upon international experiences including the UK’s National Health Service (NHS), which implemented the Quality and Outcomes Framework (QOF) in 2004. The QOF is a voluntary annual reward and incentive programme for general practices, and forms part of practices’ basic income. At the time of its introduction, the QOF was the largest healthcare-related pay‐for‐performance (P4P) scheme in the world,14 and in 2023 the total value of the QOF was £768 million.15 Over time the scale and scope of financial incentives in the QOF has reduced,16 and Scotland dismantled the framework altogether in 2016. The involvement of the National Institute for Health and Care Excellence (NICE) in the QOF started in 2009 when they took on responsibility for indicator development. Twenty years of the QOF provides an opportunity to reflect on what has been learnt that may inform future quality improvement work by the ESC. Pay‐for‐performance is broadly defined as a payment model that rewards healthcare providers for meeting pre‐defined targets for specific indicators.17 Ultimately, the goal of clinical guideline developers including NICE and the ESC is to improve patient care and outcomes through the implementation of high-quality evidence-based guidelines. Pay‐for‐performance can help achieve this goal, and is now used within healthcare systems in England, Germany, France, Australia, New Zealand, North America, South America, and Canada.18,19 A methodologically robust and transparent approach, accompanied by careful testing of QIs intended for use in P4P frameworks, mitigates the risk of unintended consequences, a risk that may be higher where performance-related payment is implicated. The benefits and limitations of the QOF have been extensively discussed within the literature (Table 1). Benefits include a more structured and systematic approach to the care for a wide range of chronic conditions. However, this is potentially at the cost of being disease focused, protocolized, data-orientated and with an attendant risk of unintended consequences.14,16,20–26 Benefits and limitations of the Quality and Outcomes Framework The NICE took on responsibility for QOF indicator development in 2009. Previously, QOF indicators were developed by a group of appointed primary care experts (the ‘Expert Panel’) supported by a group of clinicians.27 As an independent organization, the transfer of this responsibility to NICE allowed a more neutral approach, insulated from political pressures.27 The transfer to NICE also fostered greater transparency, as committee meetings are open to public observers, and committee papers are available online (https://www.nice.org.uk/get-involved/meetings-in-public/indicator-advisory-committee). Under NICE, potential QOF indicators are frequently piloted in ∼30 representative general practices in the UK. This process helps to remove the risk of post-implementation problems with indicators,14 but may result in longer indicator development times. The first QOF indicators developed by NICE went live in April 2011. The NICE indicator development programme operates according to the core NICE principles, with a transparent and robust methodology available in an ‘indicator process guide’.28 The key principles of P4P indicator development include evidence base, an independent advisory committee, user input, transparency, consultation and regular review. The NICE P4P indicators measure processes of medical care that are linked to improved outcomes (such as blood pressure monitoring) or intermediate clinical outcomes (such as blood pressure control). Process indicators are evidence-based and underpinned by NICE quality standards, NICE guidance or other sources of high-quality evidence such as guidance from the UK’s National Screening Committee. Since 2004 the QOF in England has contained a small number of outcome indicators, although over time it has become apparent that indicators with small denominators can be subject to random variation in achievement, and therefore payment may not accurately reflect patient care.26 Therefore, although an indicator may have a strong link to the evidence base, challenges associated with small denominators at individual provider (practice level) may make it unsuitable for use in the QOF. The development of NICE indicators is led by the Indicator Advisory Committee, who are volunteers independent of employment by NICE (https://www.nice.org.uk/get-involved/meetings-in-public/indicator-advisory-committee-iac/iac-members). The committee membership comprises a range of backgrounds including GPs, commissioners (payers), primary and secondary care health professionals, lay members, researchers, public health specialists and quality improvement experts.29 Additional topic experts (often including guideline developers) are invited to attend the committee to advise members on a topic-by-topic basis and to assist in discussions. The standing committee reviews all topics, and the diverse membership ensures that generalists and specialists work together to develop robust evidence-based and clinically useful indicators, supported by experts from within NICE. Since the start of NICE's involvement in developing indicators for use in the QOF, all committee meetings have been open to public observers to allow transparency of the evidence that is under consideration and the decision-making process. This has removed some of the previous mystique around the indicator development process.27 The COVID-19 pandemic was a catalyst to enabling the meetings to be available via an online platform, which has increased the accessibility to public observers. A wide range of stakeholders are consulted for their views on draft indicators, which is felt to improve the content, generalizability and acceptability of the final product. Indicators are reviewed by NICE QI staff on a regular rota, and when an associated guideline is updated. A summary of NICE evaluation is presented to the Independent Advisory Committee to consider whether an indicator remains applicable, requires re-consideration, or should be withdrawn. The latest update to NICE indicator process guide was published in December 2019, and included the addition of a framework to help appraise the validity of indicators based on work by MacLean et al. (Table 2).13 The framework supports NICE advisory committee to consistently and systematically assess a range of indicators, whilst also providing assurance to external users that the indicators are meaningful and likely to result in improved care. National Institute for Health and Care Excellence criteria to appraise the validity of pay‐for‐performance indicators Historically, NICE has published indicators following guideline publication. However, in December 2023 the update of NICE's guideline on cardiovascular disease risk assessment and lipid modification included a new indicator that could be used to support quality improvement in managing cholesterol, which was published within the clinical guideline.30,31 This new NICE indicator was included in the 2024/25 QOF with financial incentives attached (Box 1). NICE guideline: Cardiovascular disease: risk assessment and reduction, including lipid modification NICE guideline recommendation: For secondary prevention of cardiovascular disease, aim for LDL cholesterol levels of ≤2.0 mmol/L, or non-HDL cholesterol levels of ≤2.6 mmol/L. NICE P4P indicator: The percentage of patients with cardiovascular disease in whom the last recorded LDL cholesterol level (measured in the preceding 12 months) is ≤2.0 mmol/L, or last recorded non-HDL cholesterol level (measured in the preceding 12 months) is ≤2.6 mmol/L, if LDL cholesterol is not recorded. The ESC tends to avoid providing P4P specifications for their QIs at the development stage. For instance, the ESC QI for preventive cardiology,32 which accompanied the respective clinical practice guidelines,33 did not detail P4P implementation (Box 2). This is important given that the ESC QIs are designed to have international applicability across different healthcare systems. Local regulatory bodies and commissioning organizations may then be interested in creating a country-specific P4P strategy based upon some, or all, of the relevant ESC QIs based upon the available resources and health infrastructure locally. Main QI (6.1): Proportion of patients with established or high risk for atherosclerotic cardiovascular disease (ASCVD) who have LDL cholesterol (LDL-C) levels at or below that recommended for their estimated cardiovascular risk.Numerator: Patients with established or high risk for ASCVD who have LDL-C levels at or below that recommended for their estimated cardiovascular risk.a Denominator: Patients with established or high risk for ASCVD who have no contraindication, refusal or history of intolerance to statins, ezetimibe and proprotein convertase subtilisin/kexin type 9 (PCSK9) inhibitors. A criticism of the QOF is that it does not incentivize person-centred care for patients with more complex conditions who may require greater individualized care planning.34 For example, applying universal glycated haemoglobin (HbA1c) targets to all patients with diabetes regardless of their comorbidities may risk incentivizing under- or overtreatment.35,36 However, to avoid the potential of disadvantaging patients or practices, healthcare providers may record that a patient should not be considered as part of the denominator for an indicator, for example, because they have declined the healthcare intervention, or it is contraindicated. Some indicators may explicitly exclude a population where benefit and risk may differ, for example: the percentage of patients with diabetes without moderate or severe frailty in whom the last HbA1c was 58 mmol/mol or less in the preceding 12 months.37 The mission of the ESC is to reduce the burden of cardiovascular disease. It is apparent that in Europe the evidence–practice gap for some cardiovascular diseases and interventions is wide and geographically variable. This in itself suggests that the second translational gap for cardiovascular disease is modifiable. New interventions, behaviours and skills are now required to address this; learnings can be sought from others, including NICE. Whilst the political approach to healthcare improvement is, in part, dependent upon local government, the ESC QIs can be used by healthcare professionals to advocate for quality improvement initiatives at scale. The main goal for any future initiatives in this space may aim to unify the methods by which quality of care is measured for common cardiovascular diseases and interventions. Such a global and multisocietal collaboration creates an opportunity to standardize QIs and thus facilitates comparative analyses across borders. Harmonizing the quality measures in various clinical settings and healthcare systems helps understand the specific challenges for their implementation, and thus drives the development of tailored approaches to overcome these challenges in practice. The widespread use of QIs has the potential to improve the provision of high-quality, evidence-based care across a wide range of conditions, and so improve patient outcomes and health system performance. With 15 years’ experience of developing QIs, NICE has significant expertise in the development of QIs. This internal capability is supplemented and amplified through NICE working with an independent advisory committee. The transparent approach taken by NICE has allowed international groups including the ESC to build on this work, which in turn has allowed NICE to learn from the approaches taken by others.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.115 | 0.170 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.004 | 0.002 |
| Bibliometrics | 0.007 | 0.007 |
| Science and technology studies | 0.002 | 0.007 |
| Scholarly communication | 0.011 | 0.017 |
| Open science | 0.003 | 0.009 |
| Research integrity | 0.009 | 0.028 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".