Mapping the occurrence of acute myeloid leukemia: Methodological limitations and future direction
Bibliographic record
Abstract
A recent article by Ghazawi et al1 examines crude incidence rates of acute myeloid leukemia (AML) across Canada by provincial, municipal, and postal code forward sortation area (FSA) geographic units. This article is among the first to report on the burden of AML across Canada and hypothesizes that industrial pollution, namely benzene, in the ambient environment has led to “clusters” of high AML incidence rates in Ontario. Important principles of spatial disease surveillance are well documented, as noted in an editorial by Samet and Cockburn.2 We wish to raise additional methodological limitations to guide spatial surveillance work. The importance of complete age standardization for cancer risk hypotheses cannot be understated; rudimentary age stratification at 65 years adjusted Sarnia's AML incidence rate from 47.61 to 34.27 per 1,000,000 person-years (Canadian incidence rate, 30.61).1 In addition, correlative analyses require standardization. The authors cite CAREX Canada, which shows that Ontario (~40% of the Canadian population3) has the largest number of workers exposed to benzene.4 However, per capita occupational benzene exposure, as reported by CAREX, ranks Quebec and Alberta above Ontario; also, Alberta, Nova Scotia, and Manitoba have higher proportions of workers exposed to benzene in comparison with the national average.4 Regarding outdoor air exposures, in comparison with Ontarians, population exposures by benzene concentration levels estimated by CAREX report that more Quebecers and British Columbians (by numbers and proportions) are exposed to >1.5 and >2.5 times the annual national average, respectively.5 Thus, simple, expected dose-response relationships appear to be lacking to support the hypothesis. Once age-standardized incidence rates are available, correlation tests provide objective tests of the consistency with which high measures coincide. Supporting Figure 3 in Ghazawi et al1 highlights several cities with high AML crude incidences and higher average annual benzene estimates, and they note “a very consistent trend.” Nevertheless, the greater Toronto area, Sudbury, Timmins, Ottawa, Windsor, and other “nonhigh” AML incidence cities have higher benzene estimates, too (ie, inconsistency). Spatial clusters are described as the authors present findings by more granular FSAs. Well-established statistical tests of local spatial clusters are available (eg, the Space and Time Scan statistic6 and the Getis-Ord Gi* statistic7), and they provide objective evidence of spatial clustering. Using these tests strengthens findings to reduce concerns about selective reporting. Ghazawi et al1 continually contrast each geographic unit's 95% confidence intervals with the national rate1 (Supporting Appendix 1) and thus inflate type I errors. With 5418 census subdivisions (“cities”) as of the 2006 census,8 this is a significant issue. Furthermore, because geographic units were examined at multiple geographic levels, chance detection of nonelevated rates is expected. Methods to adjust P values for multiple testing are available. Though conservative, Bonferroni correction is straightforward to use.9 Altered boundaries for smaller area geographic units can be a critical issue. Major municipal amalgamation occurred between the 1990s and 2000s in Ontario.10, 11 For example, in Supporting Table 3, Ghazawi et al1 report East York as a low-incidence city, yet this city amalgamated with Toronto in 1998.12 We therefore flag concerns about the methods used to align 19 years of cancer incidence data with population data. Such concerns exist regarding FSA boundaries, too. For example, examining the historical boundaries of the “highest” crude incidence FSA in Canada, we find that N7V changed boundaries twice between 1996 and 2006.8, 13 Smaller area estimates are typically sensitive to misclassification bias, which can produce highly variable estimates. We encourage studies to describe methodological approaches to these challenges or discuss them as limitations. Care should be taken when one is describing geographic places and terminology. We note that FSA N7V, with the highest crude incidence rate in Canada, is the town of Point Edward (not “downtown Sarnia”1). Sarnia FSAs N7T and N7S contain and are immediately east-and-north of, respectively, “chemical valley", eg, https://www.thestar.com/news/world/2017/10/14/in-sarnias-chemical-valley-is-toxic-soup-making-people-sick.html but they were not reported as high-incidence areas (ie, variability and inconsistency). The authors describe “contiguous” FSAs with elevated rates that do not share a common border (Fig. 5 in Ghazawi et al1). In addition, 91 FSAs were considered to have a high incidence rate, with 48 described as industrial. We are left wondering about the 47% of the high-incidence FSAs (n = 43) that are nonindustrial. Careful description and presentation of findings reduce concerns about selective reporting. Sophisticated methods for ecological studies are available (eg, Bayesian models14), and additional issues abound (eg, population mobility15); however, the methodological issues that we raise are fundamental. We have presented several factors requiring consideration to support the hypothesized association between benzene and AML: adjustment for obvious confounders (eg, age), dose response and consistency (ie, many higher benzene exposure regions did not have higher AML incidence rates), biases (eg, a potential misclassification bias) and statistical tests that consider multiple testing. No specific funding was disclosed. The authors made no disclosures.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.340 | 0.526 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.004 | 0.003 |
| Bibliometrics | 0.008 | 0.014 |
| Science and technology studies | 0.004 | 0.005 |
| Scholarly communication | 0.007 | 0.007 |
| Open science | 0.009 | 0.006 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".