Statistical Power Analyses for Quantifying the Similarity of Categories of Surgical Procedures Among Pairs of Hospitals and Ambulatory Facilities
Bibliographic record
Abstract
INTRODUCTION: Mixed methods are often used to understand organizational associations and differences. For example, one might compare hospitals and ambulatory surgery centers, each described by its relative distribution of cases' categories of surgical procedures, quantified using anesthesia Current Procedural Terminology (CPT) codes. The similarity of these distributions between facilities can be assessed using a metric akin to a correlation coefficient. Conceptually, identifying similar organizational pairs is feasible, as most U.S. states and Canadian provinces maintain databases containing such administrative data. However, research proposals based on mixed methods may be hindered by the lack of statistical power analysis to determine whether the quantitative phase will yield a sufficient number of similar facilities to support the qualitative phase (i.e., interviews). MATERIALS AND METHODS: Data were obtained from the American Society of Anesthesiologists' National Anesthesia Clinical Outcomes Registry. The dataset included 12,902,159 cases across 272 procedure categories, performed at 2442 facilities in the United States. The similarity index between facilities ranged from 0 (no overlap in surgical procedures) to 1 (identical distribution of procedures). Values ≥0.80 were considered indicative of high similarity. We estimated the proportion of highly similar facility pairs (similarity index ≥0.80) with low standard errors (<2.0). For each pair, we computed the inverse of the standard normal distribution based on the ratio of the difference from 0.80 to the standard error. The average of these values yielded the mean prevalence of high similarity. This estimated prevalence was then used in power analyses based on the binomial distribution. RESULTS: Only 1.00% (standard error: 0.01%) of facility pairs had a similarity index ≥0.80. Based on this prevalence, a database would need to include just 38 organizations to have an ≥80% probability of identifying at least five highly similar pairs for interviews. With data from 67 organizations, there would be a ≥95% probability of identifying at least 15 pairs. In contrast, consider an individual organization deciding whether to (a) join a consortium to identify similar organizations for shared strategies, or (b) invest in analysts to explore mandatory state or provincial databases for such purposes. Unless more than 1,000, and ideally more than 2,100, organizations contribute data, the probability of finding multiple highly similar peers may be low. CONCLUSIONS: Investigators can expect a high probability of obtaining sufficient organizational sample sizes for qualitative interviews when using large-scale databases. Although only a small fraction (approximately 1%) of organization pairs exhibit high similarity, the sheer number of potential pairs in state, provincial, and national databases compensates for this. However, for an individual organization seeking to identify peers for qualitative comparison, the chance of finding highly similar matches based on similar surgical procedures is extremely low, unless joining a very large data collective.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.234 | 0.574 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.008 |
| Bibliometrics | 0.010 | 0.011 |
| Science and technology studies | 0.002 | 0.005 |
| Scholarly communication | 0.003 | 0.005 |
| Open science | 0.004 | 0.004 |
| Research integrity | 0.003 | 0.005 |
| Insufficient payload (model declined to judge) | 0.013 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".