Predicting Rare Events from Large Spatiotemporal Data: Application to Wildland Fires and Species Occupancy
Bibliographic record
Abstract
Subsampling of large data is commonly employed in statistical modelling with the goal of efficiency. When the event being modelled is rare, the data is imbalanced and thus sampling methods focus on preferentially subsampling the observations which represent those rare event occurrences. This thesis extends methodology for the subsampling of large data when modelling rare events, motivated by applications in environmetrics and ecology. The first two projects present extensions to response-based sampling. The response-based sampling approach takes independent samples of event occurrence and non-occurrence, often sampling all occurrences and a small proportion of the non-occurrences. I propose a stratified sampling approach, which defines strata based on a key variable. Independent samples of occurrences and non-occurrences are then sampled from each stratum. The bias induced by this sampling must be accounted for in the logistic regression model. The first project employs sampling weights in the logistic to account for the bias induced by this sampling design. The second project instead uses stratum-specific offsets to the same end, which now allows for the model to include multiple predictors. These approaches are validated using simulation, where they are compared to existing approaches for sampling imbalanced data. I apply these methods to fine-scale human-caused fire occurrence prediction in a region of Ontario, Canada where stratifying on a measure of fire weather and sampling more extreme observations leads to more locally precise estimates of fire occurrence. The third project presents a novel method for subsampling species detection data to fit occupancy models. When a species is rarely detected, the number of detections will be far outnumbered by the non-detections. I propose a response-based sampling method for species detection data, which allows preferential sampling of the rarer detection observations. I present a method for estimating occupancy and detection probabilities of the subsampled data, as the assumptions of traditional occupancy models no longer hold. I apply this method to detection data of Canada Warbler (Cardellina canadensis) from the Breeding Bird Survey, where we can accurately estimate the occupancy and detection parameters using just 10% of the original dataset, including estimating the effects of a habitat-related covariate.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.026 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".