Testing automatic observation quality control for assimilation in a non-hydrostatic simulation in the framework of the MAGDA HE Project
Bibliographic record
Abstract
Climate change poses significant challenges for agriculture, impacting crop production with rising temperatures, altered precipitation patterns, and more frequent extreme weather events. In this context, data assimilation techniques in Numerical Weather Prediction (NWP) models could become essential tools. By integrating real-time observations into forecasts, data assimilation enhances accuracy, providing farmers with valuable insights to adapt to climate change effectively. Thus, observations from high-resolution networks are of interest for non-hydrostatic model simulations both for verification purposes and for data assimilation. Such observations can be influenced by sub-grid scale processes that cannot be represented in the model dynamics; moreover they can be subject to gross errors. Furthermore, initial errors can grow, during a non-hydrostatic simulation, along a large number of independent modes, influencing a variety of dynamic scales. Suitable automatic data quality control techniques are then necessary and their application can improve assimilation results by enabling the representation of weather features that the model can effectively simulate.The MAGDA (Meteorological Assimilation from Galileo and Drones for Agriculture) Horizon Europe project (https://www.magdaproject.eu), started in 2022, has been developing a toolchain aimed at providing valuable weather and irrigation information to agricultural operators. At the scientific core of MAGDA activity lies the assimilation in non-hydrostatic NWP models of various sources of high-resolution observations, including in situ observations, GNSS (Global Navigation Satellite System), weather radar and meteodrones .In this work we study the effect of assimilating high-resolution in situ observations in a short-range simulation with the WRF model of a convective event of interest for MAGDA agricultural applications. We aim to assess the impact of preliminary quality control checks on observations being assimilated. In particular, the Spatial Consistency Test based on Optimal Interpolation and Cross Validation can be effective in rejecting data affected by gross errors or large representativeness errors that could otherwise introduce detrimental noise in the model simulation.This type of application paves the way for the inclusion of low-cost IoT (Internet of Things) sensors in the assimilation procedure, the "metIS hub" used in the Magda project for instance, but it can also be utilized for various other applications using low-cost sensors as observational tools.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.014 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".