Taxonomic error rates affect interpretations of a national‐scale ground beetle monitoring program at National Ecological Observatory Network
Bibliographic record
Abstract
Abstract Parataxonomists are responsible for taxonomic identifications in large‐scale biodiversity monitoring programs. However, they may lack formal taxonomic training, and thus, quantifying error rate in identification is paramount for evaluating data quality of larger biomonitoring efforts. In large‐scale biomonitoring in particular, parataxonomist error rate could vary among regions with different species richness and composition. Here, we tested whether error rates in identification of ground beetles (Coleoptera: Carabidae) by parataxonomists increased in regions with greater species richness throughout the National Ecological Observatory Network (NEON), a national biomonitoring network spanning the United States. We compared identifications made by both parataxonomist and experts of 33,516 specimens collected between 2013 and 2017 from 18 ecoclimatic regions and analyzed error rates across ecoclimatic regions as a function of total richness identified by taxonomic experts. We then compared the additional level of taxonomic support that would be required to resolve identifications to species‐level identifications. We demonstrated the extent to which parataxonomist error rate can affect interpretation of common objectives of biomonitoring results, such as comparisons of species richness between ecoclimatic regions and capacity to identify target species of interest such as non‐indigenous species. Overall parataxonomist error rate was 11.1% and did not increase in regions with greater species richness. Expert taxonomists were required to resolve parataxonomist identifications to species in an additional 16% of specimens. With an average error rate of 11.1%, species richness estimates based on parataxonomists generally mirrored richness determined by experts. However, parataxonomist error rates as low as 5% were sufficient to misrepresent the gradient of species richness across ecoclimatic regions. Parataxonomist errors also led to false detection/missed detections of non‐indigenous species. As error rates were not influenced by increasing species richness, our study suggests that parataxonomists may be used consistently in large‐scale biomonitoring efforts to amplify the abilities of taxonomists by increasing the quantity and speed in which specimens are processed. However, our study also highlights that due to parataxonomist limitations, a subset of their materials must be regularly verified by professionals to ensure the quality of data collected.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.020 | 0.114 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".