Formalizing Trust in Historical Weather Data
Bibliographic record
Abstract
Abstract Historical instrumental weather observations are vital to understanding past, present, and future climate variability and change. However, the quantity of historical weather observations to be rescued globally far exceeds the resources available to do the rescuing. Which observations should be prioritized? Here we formalize guidelines help make decisions on rescuing historical data. Rather than wait until resource-intensive digitization is done to assess the data’s value, insights can be gleaned from the context in which the observations were made and the history of the observers. Further insights can be gained from the transcription platforms used and the transcribers involved in the data rescue process, without which even the best historical observations can be mishandled. We use the concept of trust to help integrate and formalize the guidelines across the life cycle of data rescue, from the original observation source to the transcribed data element. Five cases of citizen science-based historical data rescue, two from Canada and three from Australia, guide us in constructing a trust checklist. The checklist assembles information from the original observers and their observations to the current transcribers and transcription approaches they use. Nineteen elements are generated to help future data rescue projects answer the question of whether resources should be devoted to rescuing historical meteorological material under consideration. Significance Statement Historical weather observations, such as ships’ logs and weather diaries, help us to understand our past, present, and future climate. More observations are waiting to be rescued than there are resources. Only after they have been rescued—transcribed—can the records be indexed, searched, and analyzed. Given the vast task, citizen scientists are often recruited to transcribe past weather records. Various tools, including software platforms, help volunteers transcribe these handwritten records. We provide guidance on choosing observations to rescue. This guidance is novel because it emphasizes trust throughout the data rescue process: trust in who the observers were and how the observations were made, trust in who the current transcribers are, and trust in the software tools that are used for transcription.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.218 | 0.463 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.007 | 0.005 |
| Science and technology studies | 0.009 | 0.030 |
| Scholarly communication | 0.015 | 0.033 |
| Open science | 0.005 | 0.020 |
| Research integrity | 0.006 | 0.007 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".