Comparison of methods to identify outliers observed in health services small area variation studies
Bibliographic record
Abstract
Small area variation analysis (SAV) is an established methodology in health services and epidemiological research. The goal is to demonstrate that rates differ across areas, and to explain these differences by differences in physician practice styles or patient characteristics. While the SAV statistics provide an overall variation estimate, they do not provide a statistical means to identify significant outliers. We compared the chi-square (chi2) test with three approaches in determining significant outliers in SAV. We used data from the Canadian Institute for Health Information (CIHI) for Ontario residents discharged between 1989 and 1991. Coronary artery bypass surgery, hysterectomy and hip replacement data were used to compare four statistics in determining outliers: the chi2 test, Swift's approximate bootstrap confidence interval (ABC), Carriere's T2 (T2) with simultaneous confidence intervals (SCI), and Gentleman's normalized scores (GNS). Both the ABC and SCI correct the skewness of the distribution of the adjusted rates. With large data, confidence intervals calculated by the normal or the ABC methods are indistinguishable. The T2 can be applied to also nonbinary events. For binary events, it is asymptotically the same as the chi2. The GNS ranks the rates, but the distribution of these ranks does not differ significantly from that of the adjusted rates. We concluded that when using large data with binary events, there is little advantage in using the ABC, SCI or GNS over the commonly known chi2. The chi2 remains a useful tool in small area variation analysis to 'screen' or flag potential differences beyond chance alone.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.074 | 0.057 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".