Towards case-mix-adjusted international renal registry comparisons: how can we improve data collection practice?

Renal registries are an integral part of national quality control processes for renal replacement services and provide a tool for benchmarking of clinical outcomes within and between countries. A seminal international comparison examined dialysis patient survival in Europe, Japan and the United States of America (USA) [1]. One limitation of this study, the absence of comorbidity data to adjust for case-mix differences between countries, was addressed in a subsequent study restricted to the Lombardy Dialysis and Transplant Registry and the US Renal Data System (USRDS) Case-Mix Severity study [2]. While case-mix adjustment led to attenuation of the survival differences between Italy and the USA, other limitations, such as ascertainment differences, poor data completeness and variation in methods of data collection between registries [3], remained. To overcome such problems, the Dialysis Outcomes and Practice Patterns Study (DOPPS) was initiated in the USA, later spreading to include countries on different continents around the world [4]. This prospective, longitudinal, observational study of representative samples of prevalent haemodialysis patients has the principal aim of identifying the practices that are associated with the best outcomes. However, DOPPS excludes patients on both home haemodialysis and peritoneal dialysis; such patients make up a significant proportion (up to 55%) of prevalent dialysis patients in Australia, New Zealand and the United Kingdom (UK) [5,6], and these patients tend to have less comorbidity [7]. Differential transplantation rates between countries lead to further potential for selection bias. Reporting data on all renal replacement therapy (RRT) patients to one organisation according to an internationally agreed standardized data collection methodology must be a long-term goal [8,9]. At least 12 regional or national renal registries attempt the collection of comorbidity data [10], but it remains a challenge [7]. The main aim of this review is to identify and share good practice in the collection of comorbidity data between four of those registries with a view to improving data completeness rates for countries already collecting such data and giving guidance to those considering doing so. A secondary aim is to examine the current comparability of the four renal registries’ population coverage, definitions and data completeness to assess the feasibility of initiating collaborative international comparison work. After development of the research questions, researchers at the Renal Association United Kingdom Renal Registry (UKRR) approached senior colleagues at three other national renal registries—the Australia and New Zealand Dialysis and Transplant (ANZDATA) registry, the Canadian Organ Replacement Register (CORR) and the USRDS—to establish collaboration. Annual reports and web-based resources relating to the data collection methods of the four national renal registries were then reviewed for background information on the registry systems and processes and to identify areas requiring focussed discussions. Seven broad areas were identified as follows: Generalizability of registry data—to establish whether incident RRT patients reported to the registry are generalizable to incident RRT patients in the country as a whole Definitions of RRT—to establish whether the criteria for inclusion of incident patients in the renal registry are similar in different registries Definitions of comorbidity and comorbidity data collection methods Incentives and disincentives to providing comorbidity data Evidence of validity of comorbidity data—to identify any studies that have been undertaken to determine the validity of the comorbidity data collected by the registry Evidence that adjusting for comorbidity is worthwhile, and Evidence of the relative importance of comorbidity items. A questionnaire was developed to form the basis for the semi-structured interviews. This questionnaire was piloted in an interview with a UK nephrologist experienced in national and international renal registry work and revised accordingly. Semi-structured interviews were then conducted with a senior representative of the participating renal registries. The questionnaire provided the structure for the interview while allowing discussions to explore any relevant issues that had not been anticipated. The interviews with representatives from the UKRR, USRDS and ANZDATA were conducted face-to-face; the interview with a representative of CORR was conducted by telephone conference. Two of the authors (L.K. and F.C.) participated in all interviews. Interviews took place between November 2007 and January 2008. All four renal registries collect individual patient level data on >90% of the dialysis population coverage—100% for the USRDS and ANZDATA, 99% for the UKRR and 95% for CORR. These population coverage estimates are based on dialysis centres covered but similar general population coverage estimates can be calculated for the UKRR, USRDS and ANZDATA. Apart from the UKRR, which has seen an increase in population coverage from 22% in 1998, the other three registries have observed a significant change in coverage in the preceding 10 years. In countries with <100% coverage, no systematic differences in centre characteristics (public/private, academic/non-academic, large/small) were recognized to exist between reporting and non-reporting centres. All registries believed that they reliably captured patients receiving a renal transplant whether pre-emptively or within 90 days of starting RRT. No important differences were identified in the definitions of RRT that should lead to reporting to the renal registries, with all registries expecting patients to be reported at the time of, or shortly after, their first chronic dialysis session and where the intention is that treatment will be chronic (Table 1). Patients with acute or acute-on-chronic renal failure are not expected to be reported to any registry unless they fail to recover renal function and remain on dialysis; for such cases, the date of first RRT is backdated in all registries to the date of their first ever treatment. ANZDATA, CORR and the USRDS rely on paper or web-based reporting whereas the UKRR extracts data electronically from clinical renal information technology systems in the dialysis centres. In Australia, New Zealand and the United States, it is intended that incident RRT patients are reported as soon as possible after their first RRT, and in the UK, the electronic approach captures all patients who have received RRT between census dates. CORR, which collects data on a census date basis, identified a potential for patients commencing RRT and dying between census dates to be overlooked. The United States is the only country in which notification of new RRT patients is compulsory by law. Variation in reporting of patients to registries and recording of comorbidity between the four national renal registries DOH = Department of Health. aAbsence of a tick means that the comorbid condition is absent. bCollected continuously on dialysis centre IT systems but downloaded automatically each quarter on census dates. Variation in reporting of patients to registries and recording of comorbidity between the four national renal registries DOH = Department of Health. aAbsence of a tick means that the comorbid condition is absent. bCollected continuously on dialysis centre IT systems but downloaded automatically each quarter on census dates. Although there are variations in the comorbidity items collected, all four registries collect data for ischaemic heart disease, peripheral vascular disease, cerebrovascular disease, diabetes and smoking (Table 2). Details of comorbidity data collected by the four renal registries MI = myocardial infarction; CABG = coronary artery bypass grafting; COPD = chronic obstructive pulmonary disease. aCoronary artery disease. bAngina, MI, pulmonary oedema and previous CABG. cAngina, previous MI within 3 months prior to start of RRT and previous CABG or coronary angioplasty. dAtherosclerotic heart disease or other cardiac disease, and congestive heart failure. eClaudication, amputation for vascular disease, angioplasty, ischaemic or neuropathic ulcers, vascular graft and aneurysm. fAbsent foot pulses, prior typical claudication, amputation for vascular disease, gangrene and aortic aneurysm. gInsulin dependant, non insulin requiring and insulin requiring. hType 1 or type 2. iOn insulin, on oral medication, without medication, diabetic retinopathy. jToxic nephropathy, alcohol dependence, illicit drug dependence, inability to ambulate, inability to transfer, needs assistance with daily activities, institutionalized (assisted living, nursing home, other institution), non-renal congenital abnormality. kOther serious illness (a disease not falling into one of the previously listed categories that is expected to greatly reduce 5-year survival, e.g. liver disease, dementia or HIV to enable use of the Charlson score). Details of comorbidity data collected by the four renal registries MI = myocardial infarction; CABG = coronary artery bypass grafting; COPD = chronic obstructive pulmonary disease. aCoronary artery disease. bAngina, MI, pulmonary oedema and previous CABG. cAngina, previous MI within 3 months prior to start of RRT and previous CABG or coronary angioplasty. dAtherosclerotic heart disease or other cardiac disease, and congestive heart failure. eClaudication, amputation for vascular disease, angioplasty, ischaemic or neuropathic ulcers, vascular graft and aneurysm. fAbsent foot pulses, prior typical claudication, amputation for vascular disease, gangrene and aortic aneurysm. gInsulin dependant, non insulin requiring and insulin requiring. hType 1 or type 2. iOn insulin, on oral medication, without medication, diabetic retinopathy. jToxic nephropathy, alcohol dependence, illicit drug dependence, inability to ambulate, inability to transfer, needs assistance with daily activities, institutionalized (assisted living, nursing home, other institution), non-renal congenital abnormality. kOther serious illness (a disease not falling into one of the previously listed categories that is expected to greatly reduce 5-year survival, e.g. liver disease, dementia or HIV to enable use of the Charlson score). The ANZDATA registry collects data from all dialysis and transplant units in Australia and New Zealand by means of completion of a standard form shortly after the initiation of RRT. The form includes a section for comorbid conditions that is updated each year for existing patients. For existing patients, comorbidities reported previously are pre-populated on the form by the registry to facilitate reporting by the centre. Forms that do not contain complete data on comorbidity are sent back to the dialysis centre for completion. The definitions of the comorbid conditions are opinion based. The CORR collects data by means of standardized paper forms submitted throughout the year by dialysis centres and transplant units. Provinces vary in terms of the extent of missing data on comorbid conditions. Forms with missing comorbidity items are returned to the dialysis facility for completion. The UKRR extracts data electronically from dialysis centres’ renal IT systems at quarterly census dates. All IT systems used in the UK renal centres have yes/no fields to indicate the presence or absence of the number of comorbid conditions at inception and annually thereafter; however, only the comorbidity at inception is routinely completed. The definitions of comorbidity items are given in an appendix of the UKRR annual report [11]. The USRDS identifies comorbid conditions using the Centers for Medicare and Medicaid Services (CMS) Medical Evidence Form 2728. This form, which must be completed at baseline for all new RRT patients regardless of Medicare eligibility, is sent electronically or by paper to CMS through the appropriate end-stage renal disease (ESRD) network. Although completion of the CMS Medical Evidence Form 2728 is required by law for reimbursement, the section relating to comorbid illnesses is voluntary but must be completed by a doctor. One simple strategy currently successfully employed by ANZDATA and CORR is the return of any incomplete forms to the dialysis centres. This applies not only to the baseline assessment but also annual assessment of comorbidities, which ANZDATA further facilitates by pre-populating fields with the previous years’ returns. CORR also has individuals in each dialysis centre who have declared an interest in reporting to the registry and have attended teaching sessions on data entry and data handling. Such solutions are limited by the available financial and manpower resources of the registries and dialysis centres but may be given greater priority if dialysis centres are reimbursed on a diagnosis- or health-related group basis. One registry, CORR, allows nurses and administrators to record comorbidity, and another, ANZDATA, does not specify who should enter these data (although in practice they are entered by or on behalf of physicians in the majority of cases). Optimizing data completeness through the use of trained non-medical staff is a well-recognized and accepted approach [12] and has been examined in the Hemodialysis (HEMO) study with an inter-rater agreement of 84% between the nurses trained in data extraction and physicians [13]. With ongoing training and regular assessment of their data abstraction accuracy, administrative staff have been shown to collect reliable, valid and quality data from medical records [14] and this may be more likely to prove cost-effective. The USRDS collects comorbidity data using the CMS Medical Evidence Form 2728 and although completion of this form is required to establish Medicare eligibility, reporting of comorbidity is voluntary and is known to result in underreporting of comorbidities [15]. In a comparison of the different methods used to overcome non-random missing registry comorbidity data in observational health care studies, Norris et al. found that when cases with missing data were excluded, the prevalence of certain comorbidities was overestimated, but when they assumed that missing data indicated absence of comorbidities, an opposite effect on the prevalence of other comorbidities was observed [16]. The UKRR has also examined selection bias in comorbidity reporting from renal units with a high percentage of completed comorbidity returns and this demonstrated that after adjustment for age, patients with comorbidity data returned (as either present or absent) had lower death rates than those in whom comorbidity was not returned [17]. The validity of merging registry and administrative data has been well established. For some time now the USRDS has been regularly updating its database using the Medicare Enrolment Database and the Medicare inpatient and outpatient claims databases in order to overcome the problem of incomplete data for comorbidity items. Quan et al. has shown that International Classification of Disease, 9th Version, Clinical Modification (ICD-9-CM) administrative data generally agrees with patient chart data although the kappa measure of agreement across a number of studies varies from 0.87 for metastatic solid tumours to 0.34 for peripheral vascular disease [18]. When Norris et al. proceeded to merge the registry data discussed above with ICD-9-CM administrative data, the ‘enhanced’ data were superior in predicting 1-year mortality [16]. Learning from this, the UKRR is currently exploring links with a number of existing databases including the Hospital Episodes Statistics dataset in England. The ‘urgent need for standardized information technology for automated collection and transmission of clinical performance indicators from electronic patient management systems’ is also recognized by the European Renal Association-European Dialysis and Transplantation Association (ERA-EDTA) Registry Quality European Studies (QUEST) initiative [9]. While the different comorbidity data items collected by the four registries studied in this work can be condensed to cover the same main themes—ischaemic heart disease, peripheral vascular disease, cancer, etc.—and thus allow international comparison, the potential value of harmonizing definitions is clear to see. Standardized data collection methods, including those for recording comorbid conditions and their severity, have long been recognized as important [19] and are another central component of the ERA-EDTA Registry QUEST initiative [9]. Once comorbidity definitions are harmonized, registries will be able to facilitate important international projects such as the International Quotidian Dialysis Registry, which had to merge often quite heterogeneous national datasets to study the effectiveness of frequent haemodialysis [20]. ANZDATA, CORR and the UKRR have begun to explore the possibility of using comorbidity data in case-mix adjustment for centre comparisons. The UKRR has used comorbidity data to adjust for differences in case mix when publicly reporting differences in survival between dialysis centres [21]. In the USA, the Dialysis Facility Compare website provides the public with data on quality measures, such as patient survival, that are adjusted for age, race, gender, diabetes, duration of ESRD, body mass index (BMI) and patient comorbidities recorded at incidence [22]. Such opportunities for public scrutiny should provide centres with considerable motivation to report the comorbidity of their patients. The risk of such a system, however, is that it might also create an environment that encourages up-coding, which is exaggerated reporting of comorbidity in order to achieve better case-mix adjusted outcomes, as has been reported for coronary artery bypass grafting [23]. Use of case-mix-adjusted reimbursement rates could provide an additional incentive to improve collection of comorbidity data. In the USA, Hirth et al. found that the introduction of a proposed expanded bundle for dialysis services and quality incentive payments [24] without adjusting for more than the current age and BMI could have serious financial implications for dialysis facilities [15]. The implementation of such performance-linked payments could therefore act as a financial incentive for dialysis facilities to improve their reporting of comorbidities. In an unpublished data validity exercise, ANZDATA confirmed that comorbid illnesses were accurately reported to the registry, although without a ‘gold-standard’ it was accepted that conclusions were inevitably opinion-based. CORR is currently conducting a validity study of comorbidity data reported between 2005 and 2006. In 2005, the UKRR undertook a data validation exercise in all five Welsh renal units using case notes as the of information and found that comorbidity data were and valid in the USA, a validation study using data from the study found that the of the Medical Evidence Form across the comorbidity items of the registries have undertaken studies to examine the relative importance of comorbidities when adjusting for variation in different health outcomes, e.g. and quality of is well that of comorbidity outcomes in dialysis and the of this to international where there is greater in practices and therefore greater to challenge the has been by the DOPPS While adjustment for case-mix reduce some of the observed mortality differences between patients in Lombardy and the there to be a on of the variation can be with case-mix international comparison of RRT survival has that adjusting for comorbidity and above age, gender, dialysis and renal disease more of the in RRT survival between countries At a centre a survival study of RRT patients in centres across five European countries found that case-mix adjustment but not survival differences between centres of the research however, work to have been to the of comorbidity adjustment in centre comparisons. In its 2007 Annual the UKRR and then renal diagnosis- and 1-year survival rates for centres [21]. While the effect on the survival was for a is likely to be in the of such comparisons. for comorbidity is also now routinely into the Dialysis Facility Compare website for patients in the the of case-mix adjustment for centre are by physicians and registries are likely to to with comorbidity returns that additional data entry by individuals with the additional variation in RRT outcomes that can be by comorbidity the initiation of RRT needs to be and the additional reporting on dialysis at least these data can be from existing secondary to improve data returns for comorbidity be to reduce the items collected to a of those to be associated with patient outcomes, without For a systematic review of the relating to comorbidity and survival on dialysis found to study only the of age, diabetes, heart disease and peripheral vascular disease which comorbidity items are and which are physicians need to additional variation in survival will be by each additional comorbidity they are to are also outcomes other than survival that are Although it could not have been it does that the same comorbidities are relevant to all health-related age, diabetes and comorbidities such as vascular disease, relevant to predicting have also been found to quality of rates and research is to establish the importance of adjusting for case-mix when outcomes using national renal registry data if reporting is to to and identify the comorbidities that are important in variation in outcomes between centres and countries across RRT and at different time International of RRT and outcomes provide an important for benchmarking between heterogeneous systems and must therefore be developed as a part of any national quality of internationally agreed quality of care measures, such as the Outcomes will provide further opportunities for quality Renal registries can provide these data in to international studies such as additional For such to be however, international differences in case-mix must be into to improve completeness of comorbidity datasets include financial or other to the use of trained staff to collect data and the of a number of comorbidities with value that are collected by a data validation as undertaken to some extent by all four registries, need to In to a relating to the population coverage, inclusion criteria and comorbidity definitions of the four registries, this exercise has been that international the merging of national renal datasets are now The authors for to the of this work and for the of interest by and data collection by renal Transplant

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.