Evaluation Of Dietary Assessment Tools: Does ‘Validated’ Mean What We Think It Means?
Bibliographic record
Abstract
Evaluation of the extent to which self‐report dietary assessment tools accurately capture true intake is critical to informing the collection of dietary intake data that can advance our understanding of diet and health. Such evaluation is challenging due to the lack of unbiased reference measures for dietary intake, forcing reliance on self‐report (and thus error‐prone) measures as comparators. Depending on the type of instrument and its intended use, evaluation is also complicated by the need to test tools in different settings and populations. The objective of this work is to examine challenges in evaluating dietary assessment tools and the ways in which the findings of validation studies are interpreted. We draw upon a scoping review of Canadian studies of free‐living adult populations that included an assessment of dietary intake to illustrate the extent and ways in which validity of tools are addressed. Of 58 studies that used a frequency questionnaire or screener to assess diet, tools were reported to have been validated for the given study sample in a small subgroup. In close to two‐thirds, authors reported that the tool has been validated, in some cases with few details on how it was evaluated and in what population(s). In close to a third, there was little attention to whether tools used have been tested. Given that self‐report instruments cannot be expected to perfectly capture true intake, more nuanced discussions of what it means for a tool to be ‘valid’ as well as the implications of limitations in tools for study results appear warranted. Such discussions are key to improving the quality of self‐reported dietary intake data. Support or Funding Information This scoping review was supported by the Canadian Institutes of Health Research (137238) and Canadian Cancer Society Research Institute (702855).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.651 | 0.870 |
| Meta-epidemiology (narrow) | 0.002 | 0.003 |
| Meta-epidemiology (broad) | 0.009 | 0.007 |
| Bibliometrics | 0.012 | 0.018 |
| Science and technology studies | 0.006 | 0.021 |
| Scholarly communication | 0.030 | 0.031 |
| Open science | 0.008 | 0.010 |
| Research integrity | 0.008 | 0.013 |
| Insufficient payload (model declined to judge) | 0.004 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".