Predicting Rare Events from Large Spatiotemporal Data: Application to Wildland Fires and Species Occupancy
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Subsampling of large data is commonly employed in statistical modelling with the goal of efficiency. When the event being modelled is rare, the data is imbalanced and thus sampling methods focus on preferentially subsampling the observations which represent those rare event occurrences. This thesis extends methodology for the subsampling of large data when modelling rare events, motivated by applications in environmetrics and ecology. The first two projects present extensions to response-based sampling. The response-based sampling approach takes independent samples of event occurrence and non-occurrence, often sampling all occurrences and a small proportion of the non-occurrences. I propose a stratified sampling approach, which defines strata based on a key variable. Independent samples of occurrences and non-occurrences are then sampled from each stratum. The bias induced by this sampling must be accounted for in the logistic regression model. The first project employs sampling weights in the logistic to account for the bias induced by this sampling design. The second project instead uses stratum-specific offsets to the same end, which now allows for the model to include multiple predictors. These approaches are validated using simulation, where they are compared to existing approaches for sampling imbalanced data. I apply these methods to fine-scale human-caused fire occurrence prediction in a region of Ontario, Canada where stratifying on a measure of fire weather and sampling more extreme observations leads to more locally precise estimates of fire occurrence. The third project presents a novel method for subsampling species detection data to fit occupancy models. When a species is rarely detected, the number of detections will be far outnumbered by the non-detections. I propose a response-based sampling method for species detection data, which allows preferential sampling of the rarer detection observations. I present a method for estimating occupancy and detection probabilities of the subsampled data, as the assumptions of traditional occupancy models no longer hold. I apply this method to detection data of Canada Warbler (Cardellina canadensis) from the Breeding Bird Survey, where we can accurately estimate the occupancy and detection parameters using just 10% of the original dataset, including estimating the effects of a habitat-related covariate.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it