Evaluating the Reproducibility and Verifiability of Nutrition Research: A Case Study of Studies Assessing the Relationship Between Potatoes and Colorectal Cancer
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Background: The credibility of nutritional research is dependent on the rigor with which studies are conducted and the ability for independent assessment to be performed. Despite the importance of these, more work is needed in the field of nutrition to buttress the trustworthiness of nutrition research. Objective: To develop and apply a process for evaluating the rigor, reproducibility, and verifiability of nutritional research, using the relationship between potato consumption and Colorectal cancer (CRC) as a case study. Methods: We updated existing systematic reviews to include studies on potatoes and CRC, assessing their design, execution, and reporting quality. We attempted to reproduce and verify the results of included studies by requesting raw data from authors and following statistical methods as described in the publications. Rigor was evaluated using four different tools: ROBINS-E, STROBE-Nut, Newcastle-Ottawa scale, and additional criteria related to transparency. Results: Eighteen studies were included, none of which publicly share data. We managed to access data for only two studies, successfully reproducing and verifying the results for one. The majority of studies exhibited a high risk of bias, with significant limitations in reporting quality and methodological rigor. Conclusions: Research on the relationship between potato consumption and CRC risk is insufficiently reproducible and verifiable, undermining the trustworthiness of its findings. This study highlights the need for improving transparency, data sharing, and methodological rigor in nutritional research. Our approach provides a model for assessing the credibility of research in other areas of nutrition.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.226 | 0.140 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.002 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it