A systematic review of transparency in the methods of expert knowledge use
Bibliographic record
Abstract
Abstract The use of expert knowledge (EK) as an alternative to empirical data is increasing in the ecological disciplines in response to the growing need for rapid decisions that are embraced by local communities and implemented under strong resource constraints. Despite this increasing use, the validity of EK as a data source is still questioned by some who label it as biased or unreliable. Transparency in the methods applied in the use of EK allows confirmation of methodological rigour and study repeatability; key steps towards promoting the acceptance of EK as a valid data source. However, the practiced levels of methodological transparency in EK use are currently unknown in the ecological disciplines. To fill this knowledge gap, we performed a systematic review of transparency in methods applied in the use of EK, focusing on research in forest ecology and management as a case study. Covering the period from 1990 to 2016, we identified 296 publications that employed EK, which were assessed for their reporting on 15 methodological criteria of EK use. Our results indicate that transparency in the use of EK is generally very low, with only three out of 15 methodological criteria addressed in more than 50% of publications. Nevertheless, overall reporting on EK methods, and the reporting of four individual methodological criteria, were positively affected by publication year, suggesting that researchers increasingly recognize the importance of transparency in the use of EK methods. Synthesis and applications . Given the growing use of expert knowledge (EK) in the ecological disciplines, there is an urgent need to further its acceptance as a valid data source. To achieve this goal, researchers need to be more diligent in following best practices and reporting on EK methods. We therefore present a proposed set of guidelines for the reporting of EK methods. However, journal editors and reviewers also can play an important role in reaching this goal by requiring authors to transparently report on EK methods, just as they would expect for the use of empirical data.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.339 | 0.685 |
| Meta-epidemiology (narrow) | 0.002 | 0.003 |
| Meta-epidemiology (broad) | 0.010 | 0.010 |
| Bibliometrics | 0.036 | 0.027 |
| Science and technology studies | 0.003 | 0.006 |
| Scholarly communication | 0.010 | 0.012 |
| Open science | 0.005 | 0.007 |
| Research integrity | 0.004 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".