Assessing the relevance of environmental exposure data sets
Bibliographic record
Abstract
Abstract Environmental exposure data are used by decision-makers to assess environmental risks and implement actions to mitigate risks from contaminants. The first article in this series summarized the available evaluation schemes for environmental exposure data, of which there are few compared to those available for environmental hazard data. The second article covered the assessment of the reliability of environmental exposure data sets under the Criteria for the Reporting and Evaluation of Exposure Data (CREED). The aim of this article is to provide an overview and practical guidance on the relevance assessment in the context of the CREED approach for evaluating exposure monitoring data sets. Systematically considering relevance is critical for both evaluating existing data sets and for optimizing the design of new monitoring studies. Relevance is defined here as the degree of suitability or appropriateness of a data set to address a specific purpose or to answer the questions that have been defined by the assessor or for those generating exposure data. The purpose definition will be the foundation for the relevance assessment, to clarify how the assessor should rate the assessment criteria (fully met, partly met, not met/inappropriate, not reported, not applicable). This will provide transparency for anyone reviewing the outcomes. An explicit gap analysis (i.e., an articulation of the data set limitations for the stated purpose) is an important outcome of the relevance assessment. The relevance evaluation approach is demonstrated with three case studies, all relating to the freshwater aquatic environment, where the data sets are scored as relevant with or without restrictions, not relevant, or not assignable. The case studies represent both organic and inorganic constituents, and have different data characteristics (e.g., percentage of censored data, sampling frequencies, relation to supporting parameters). Integr Environ Assess Manag 2024;20:1004–1018. © 2023 SETAC Key Points This article provides an overview and guidance on the relevance assessment in the context of the Criteria for the Reporting and Evaluation of Exposure Data approach for evaluating exposure monitoring data sets. Relevance is defined here as the degree of suitability or appropriateness of a data set to address a specific purpose that has been defined by the assessor, or for those generating exposure data. A clear purpose definition is the foundation for the relevance assessment, to clarify how the assessor should rate the assessment criteria (fully met, partly met, not met/inappropriate, not reported, not applicable). This approach provides transparency for anyone reviewing the outcomes, and an explicit gap analysis (i.e., an articulation of the data set limitations for the stated purpose) is an important outcome of the relevance assessment.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.242 | 0.524 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.004 |
| Bibliometrics | 0.018 | 0.011 |
| Science and technology studies | 0.002 | 0.005 |
| Scholarly communication | 0.010 | 0.006 |
| Open science | 0.004 | 0.011 |
| Research integrity | 0.004 | 0.004 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".