Using the network Common Data Form (netCDF) for storage of Atmospheric Data
Bibliographic record
Abstract
Many tools and data formats exist for atmospherical data. To disseminate this wealth of information to the geospatial communities is very cumbersome: in general the geospatial communities use other data formats and they use GIS for their analyses. Therefore, time-consuming and inefficient conversions are needed to use atmospherical data. Within the ADAGUC project (Atmospheric Data Access for the Geospatial User Community) we provide selected space borne atmospheric and land datasets using web services that can be used for data comparison, resampling, selection, manipulation and visualization in GIS. Within the ADAGUC project data is stored in a standardized way. In this paper we focus on the data format used within the ADAGUC project to store the data. The ADAGUC data format uses the network Common Data Form (netCDF) as the data format to store the data. The format follows the Climate and Forecast conventions (CF conventions) and uses the directives of the infrastructure for Spatial Information in Europe (INSPIRE) and the DUTCH NL metadata standard which are both based on ISO-19115. Currently the ADAGUC data format is limited to two data types: vector data and raster data. To compose the files a programming interface has been created and support has been added to the GDAL/OGR library. The GDAL/OGR library is a translator tool to convert various geographical data formats to other geographical formats. Support for the ADAGUC data format has been added, which makes it possible to convert ADAGUC files to any other format supported by GDAL/OGR. To provide access to the atmospheric datasets, a spatial data infrastructure based on OGC compliant web services is developed: Web Mapping Services (WMS) for online visualization, Web Feature Services (WFS) for downloading vector data and Web Coverage Services (WCS) for downloading raster data. The development of this infrastructure is a dynamical process. During this process we encountered several problems that have been solved during the project. Atmospheric datasets are special in the way that they are temporal and that the file size may be huge. Most server solutions are optimized for static datasets by using caching, which does not work well for temporal datasets. Also the OGC standards are not fully adapted yet to temporal data. For example the TIME property that optimizes the retrieval of temporal data is available in the OGC-WMS specification and OGC-WCS specification but is not yet available in the OGC-WFS specification. The data format and solutions to these problems will be presented on the conference.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.024 |
| Meta-epidemiology (narrow) | 0.003 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.004 | 0.006 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.006 |
| Open science | 0.005 | 0.004 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.191 | 0.090 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".