Ensemble Experiments on Numerical Weather Prediction Error and Uncertainty for a North Pacific Forecast Failure
Bibliographic record
Abstract
An intense maritime cyclone off the northwest coast of North America during 9–14 February 1999 was remarkable in the repeated inability of numerical weather prediction (NWP) models to forecast its evolution. Each day during that period, the operational and research models of the United States and Canada were initialized with new analysis fields. Each day, the new short-term NWP forecasts predicted the storm to strike the densely populated Lower Mainland (Vancouver) area of southwest British Columbia, Canada, during the next 24–48 h. NWP guidance prompted the local forecast office to issue storm warnings including one or more of the following for Vancouver: heavy snow, heavy rain, and strong winds. Satellite imagery clearly showed the storm off the coast, but the storm did not strike Vancouver until much later and in a decayed state. This synoptic case is studied with an aim to understand the source of the NWP error, and an ensemble of research model runs is made to address three possibilities for failure: 1) initial condition (IC) error, 2) model error for a particularly nonlinear or sensitive event, and 3) sympathetic data denial. To estimate the effect of IC uncertainty, a short-range ensemble system is developed and tested on a limited-area model for a sequence of successive 3-day reforecasts covering the 10-day period surrounding the storm. This IC ensemble shows some correlation between spread and skill and provides one estimate of IC uncertainty. To estimate the effect of model uncertainty, a physics-based ensemble is run for the same period. The effect of data denial is investigated by comparing forecasts made with the same model but from analyses created at different operational centers. Results suggest that if the runs initialized at 0000 UTC 10 February 1999 were used for guidance, model uncertainty was likely responsible for the forecast failure. It had larger-than-average model error but lower-than-average IC error. Subsequent forecast errors were likely dominated by IC uncertainty. An attempt at assessing sympathetic data denial is inconclusive.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.011 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".