Evaluating the Reproducibility and Verifiability of Nutrition Research: A Case Study of Studies Assessing the Relationship Between Potatoes and Colorectal Cancer
Bibliographic record
Abstract
Background: The credibility of nutritional research is dependent on the rigor with which studies are conducted and the ability for independent assessment to be performed. Despite the importance of these, more work is needed in the field of nutrition to buttress the trustworthiness of nutrition research. Objective: To develop and apply a process for evaluating the rigor, reproducibility, and verifiability of nutritional research, using the relationship between potato consumption and Colorectal cancer (CRC) as a case study. Methods: We updated existing systematic reviews to include studies on potatoes and CRC, assessing their design, execution, and reporting quality. We attempted to reproduce and verify the results of included studies by requesting raw data from authors and following statistical methods as described in the publications. Rigor was evaluated using four different tools: ROBINS-E, STROBE-Nut, Newcastle-Ottawa scale, and additional criteria related to transparency. Results: Eighteen studies were included, none of which publicly share data. We managed to access data for only two studies, successfully reproducing and verifying the results for one. The majority of studies exhibited a high risk of bias, with significant limitations in reporting quality and methodological rigor. Conclusions: Research on the relationship between potato consumption and CRC risk is insufficiently reproducible and verifiable, undermining the trustworthiness of its findings. This study highlights the need for improving transparency, data sharing, and methodological rigor in nutritional research. Our approach provides a model for assessing the credibility of research in other areas of nutrition.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.763 | 0.861 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.006 | 0.007 |
| Bibliometrics | 0.015 | 0.016 |
| Science and technology studies | 0.009 | 0.017 |
| Scholarly communication | 0.014 | 0.016 |
| Open science | 0.007 | 0.015 |
| Research integrity | 0.010 | 0.006 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".