When and how to establish a new reference standard for medical tests: a scoping review identifying methodological priorities
Bibliographic record
Abstract
BACKGROUND AND OBJECTIVE: The reference standard, defined as the best available test or test strategy for diagnosing a target disease/condition in a specific clinical population, is rarely perfectly accurate. This imperfection raises the question of when (and how) to establish a new test as the reference standard test. The objective of this scoping review was to assess methodological priorities for establishing a new reference standard for medical tests. METHODS: This scoping review was performed using JBI methodology and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping reviews reporting guideline. We included sources describing methodological approaches to establishing a new reference standard across all healthcare populations and settings. Data were analyzed using a combination of descriptive and qualitative content and thematic analyses. RESULTS: We screened 5396 studies and included 13 in our final analysis (7 methodological studies and 6 applied studies). Three main methodological themes emerged: (1) demonstrating the imperfectness of the current reference standard, (2) comparing candidate reference standard tests, and (3) decision principles for accepting a new reference standard. Among the six applied studies that established reference standards for specific conditions, five used composite reference standards. CONCLUSION: Based on the findings of our review, we define a potential workflow for structuring decision-making relating to whether to accept a new test as the reference standard. Development of formal methodological guidance for when and how to establish or replace a reference standard would likely benefit guideline developers, clinicians, technology assessors, and, ultimately, patients. PLAIN LANGUAGE SUMMARY: "Reference standard" tests are considered the best available method to determine whether someone has a particular health condition. However, these tests are often imperfect, and there is little guidance on when a new test should replace the current reference standard. We reviewed published research to understand how new reference standards have been created or evaluated. Thirteen studies met our search criteria. Together, they showed four key steps: first, defining the purpose of the reference standard (for clinical care or research); second recognizing when the current reference standard is imperfect; third, comparing potential new tests; and fourth, deciding whether adopting a new test will improve patient care. Our findings highlight the need for formal guidance to support groups creating new reference standards in the future.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.581 | 0.800 |
| Meta-epidemiology (narrow) | 0.003 | 0.005 |
| Meta-epidemiology (broad) | 0.012 | 0.015 |
| Bibliometrics | 0.039 | 0.035 |
| Science and technology studies | 0.007 | 0.011 |
| Scholarly communication | 0.026 | 0.035 |
| Open science | 0.011 | 0.014 |
| Research integrity | 0.011 | 0.012 |
| Insufficient payload (model declined to judge) | 0.004 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".