A systematic review of transparency in the methods of expert knowledge use
Bibliographic record
Abstract
Abstract The use of expert knowledge (EK) as an alternative to empirical data is increasing in the ecological disciplines in response to the growing need for rapid decisions that are embraced by local communities and implemented under strong resource constraints. Despite this increasing use, the validity of EK as a data source is still questioned by some who label it as biased or unreliable. Transparency in the methods applied in the use of EK allows confirmation of methodological rigour and study repeatability; key steps towards promoting the acceptance of EK as a valid data source. However, the practiced levels of methodological transparency in EK use are currently unknown in the ecological disciplines. To fill this knowledge gap, we performed a systematic review of transparency in methods applied in the use of EK, focusing on research in forest ecology and management as a case study. Covering the period from 1990 to 2016, we identified 296 publications that employed EK, which were assessed for their reporting on 15 methodological criteria of EK use. Our results indicate that transparency in the use of EK is generally very low, with only three out of 15 methodological criteria addressed in more than 50% of publications. Nevertheless, overall reporting on EK methods, and the reporting of four individual methodological criteria, were positively affected by publication year, suggesting that researchers increasingly recognize the importance of transparency in the use of EK methods. Synthesis and applications . Given the growing use of expert knowledge (EK) in the ecological disciplines, there is an urgent need to further its acceptance as a valid data source. To achieve this goal, researchers need to be more diligent in following best practices and reporting on EK methods. We therefore present a proposed set of guidelines for the reporting of EK methods. However, journal editors and reviewers also can play an important role in reaching this goal by requiring authors to transparently report on EK methods, just as they would expect for the use of empirical data.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.003 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".