On the prevalence of uninformative parameters in statistical models applying model selection in applied ecology
Bibliographic record
Abstract
Research in applied ecology provides scientific evidence to guide conservation policy and management. Applied ecology is becoming increasingly quantitative and model selection via information criteria has become a common statistical modeling approach. Unfortunately, parameters that contain little to no useful information are commonly presented and interpreted as important in applied ecology. I review the concept of an uninformative parameter in model selection using information criteria and perform a literature review to measure the prevalence of uninformative parameters in model selection studies applying Akaike's Information Criterion (AIC) in 2014 in four of the top journals in applied ecology (Biological Conservation, Conservation Biology, Ecological Applications, Journal of Applied Ecology). Twenty-one percent of studies I reviewed applied AIC metrics. Many (31.5%) of the studies applying AIC metrics in the four applied ecology journals I reviewed had or were very likely to have uninformative parameters in a model set. In addition, more than 40% of studies reviewed had insufficient information to assess the presence or absence of uninformative parameters in a model set. Given the prevalence of studies likely to have uninformative parameters or with insufficient information to assess parameter status (71.5%), I surmise that much of the policy recommendations based on applied ecology research may not be supported by the data analysis. I provide four warning signals and a decision tree to assist authors, reviewers, and editors to screen for uninformative parameters in studies applying model selection with information criteria. In the end, careful thinking at every step of the scientific process and greater reporting standards are required to detect uninformative parameters in studies adopting an information criteria approach.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.421 | 0.788 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.004 | 0.007 |
| Bibliometrics | 0.023 | 0.024 |
| Science and technology studies | 0.003 | 0.013 |
| Scholarly communication | 0.013 | 0.015 |
| Open science | 0.005 | 0.008 |
| Research integrity | 0.008 | 0.009 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".