A Simulation Study of Sampling in Difficult Settings: Statistical Superiority of Little-Used method
Bibliographic record
Abstract
Abstract Background Taking a representative sample to determine prevalence of variables such as disease is difficult when little is known about the target population. Several methods have been proposed that apply cluster sampling techniques to Primary Sampling Units (PSUs). The PSUs are typically towns or census tracts. Some methods are based on random walks within towns, e.g., the original World Health Organization’s Extended Program on Immunization (‘EPI’) surveys and variants, including sampling from four quadrants of each town (‘Quad’). Several major international surveys take random samples from small areas (‘SA’) such as census tracts. Another method uses satellite images and Global Positioning Systems to randomly sample within PSUs from squares in a superimposed grid (‘Square’). We used computer simulations to compare these sampling methods and simple random sampling within towns (‘SRS’) in virtual populations. SRS was our standard, even though it is impractical in low-information settings. Methods We constructed 50 virtual populations with varying characteristics, each comprising about a million people spread over 300 towns. The risk of disease for each person varied within and between towns. We created a binary exposure variable and allocated disease statuses to individuals assuming four relative risks (RRs) from exposure. We added three populations with equal risk of disease for every person in the population. For each population, each of the sampling methods – EPI, Quad, SA, Square, and SRS - and each of three sample sizes per PSU (7, 15, and 30), we simulated 1000 samples. For each simulation we estimated the prevalence and RRs. We used the bias and variance of the estimates to calculate the Root Mean Squared Error (RMSE) of these estimates. We ranked the RMSEs of each method and computed the ratio of each method’s RMSE to that of SRS for each population. We computed the mean ranks and ratios across the 50 populations. Results Apart from SRS, Square had the lowest mean rank of RMSEs for all samples sizes when estimating prevalence. When estimating RR, Square had the lowest mean ranks for samples sizes of 15 and 30 per PSU; for n=7 per PSU, the Quad mean rank was the lowest. The results for the mean ratios of RMSEs showed the same pattern; Square had the lowest values for all sample sizes when estimating prevalence and the lowest values for the two larger sample sizes when estimating RRs. Notably, when estimating prevalence, the ratios increased with sample size per PSU for SA, Quad, and EPI, suggesting those methods did not benefit as much from the larger sample sizes as would be expected from statistical theory. Conclusions Of several methods that are practical in an imperfectly known population, the Square method was mostly the best, especially for the larger sample sizes. The methods that sample within small areas (Quad, SA, and EPI) do not gain as much statistical benefit as expected from larger sample sizes per PSU, because of some clustering within the areas.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.089 | 0.315 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.003 |
| Scholarly communication | 0.002 | 0.003 |
| Open science | 0.004 | 0.002 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.005 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".