Epidemiology and Health Administrative Data

We thank you for your insightful comments regarding our assessment of the epidemiological trends of inflammatory bowel disease (IBD) in Ontario.1 Your editorial raises many important points of methodological interest in the rapidly evolving field of research using health administrative data. The importance of the predictive values is another methodological issue not raised in your editorial. Even slight uncertainty in the predictive values could greatly affect the reliability of a low-prevalence disease cohort.2 Predictive values depend on the prevalence of the disease in a population. Our previous review of algorithm-validation studies emphasized the importance of accurately estimating positive predictive value and negative predictive value by validating the algorithm in a reference cohort with similar disease prevalence as the general population.3 In our pediatric4 and adult5 algorithm-validation studies, we attempted to address this issue. The validation study of the Alberta algorithm6 (which was used by the Quebec cohort) used a validation cohort with a 9% prevalence of IBD, much higher than the prevalence of IBD in the Canadian population, potentially inflating the positive predictive value. In fact, in our adult algorithm-validation study, we determined that the Alberta algorithm was not as accurate to identify patients with IBD in 2 Ontario validation cohorts, compared with the algorithms currently in use by the Ontario Crohn's and Colitis Cohort.5 However, we determined that a variation of a previously validated Manitoba algorithm7 was quite accurate in Ontario adults. Therefore, algorithms to identify patients from within health administrative data should be validated in the jurisdiction to which they are applied to ensure the most effective disease surveillance. Nevertheless, our algorithm-validation studies had weaknesses as well. Imperfect reference standard populations may have resulted in falsely reduced PPV. For example, in the pediatric study,4 we assumed that all children with IBD were seen in a single regional pediatric hospital and therefore contained within the hospital database. However, we discovered that the algorithm functioned better in younger children, with lower false-positive rates. The older “false positives” were likely diagnosed with IBD but were treated by adult gastroenterologists outside of the pediatric hospital. Therefore, the PPV appeared decreased due to an imperfect reference standard population and not due to an inaccurate algorithm. Imperfections in algorithm-validation studies are indicative of the imperfections in all studies using health administrative data. However, the methods used in this research field are rapidly evolving, reflecting the increasing availability of the large databases themselves. Health administrative data (defined as data collected for the purpose of administration of health care system8) are examples of routinely collected health data. Other examples include databases of electronic medical records, clinical registries, disease registries (such as cancer databases), and clinical research databases collected at the bedside, such as the Clinical Practice Research Datalink. Increased use of such routinely collected health data has prompted the efforts to improve reporting of such research, which will improve transparency of methods, results, strengths, and biases. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement will expand the STROBE guidelines to observational studies using such data.9 We have obtained the input of over 100 international stakeholders and held a working committee meeting in Lausanne, Switzerland (October 2013) to create the statements and explanatory document. We will shortly circulate the statements for comment before publication. Researchers, clinicians, policymakers, or other stakeholders may participate in the review process by e-mailing record@record-statement.org or obtain further information at record-statement.org. The increased recognition of algorithm validation as an important method in health administrative data research is an example of the evolving methodology of such research. We believe that editorials such as yours and collaborative efforts such as the RECORD statement will highlight the strengths and weaknesses of studies using routinely collected health data. As these data become more available for research, it is of great importance that we refine the methods used to conduct such research to ensure the most accurate and useful results.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.