Novel Approach to Proficiency Testing Reveals Significant Variations in Biomarker Practice Leading to Critical Differences in Lung Cancer Management
Bibliographic record
Abstract
Introduction: Timely access to quality biomarker testing in NSCLC is critical to patient outcomes. The Canadian Pathology Quality Assurance provides external quality assurance (EQA) to laboratories in Canada. The Canadian Pathology Quality Assurance has recently developed a novel approach to molecular biomarker EQA testing, assessing accuracy, turnaround time, and interpretation of reports. This study reports the results of the first end-to-end biomarker EQA challenge in NSCLC. Methods: Three challenge specimens were made using NSCLC tissue and paired with clinical vignettes mimicking referred-in cases. Participants were to provide all required molecular testing (immunohistochemistry and gene sequencing) and submit final reports for each case, while being timed. Reports were assessed by molecular pathologists and medical oncologists who recommended a systemic treatment based on vignettes and reports. Results: A total of 13 Canadian laboratories participated. The turnaround time of molecular reporting ranged from five to 57 (median 22.5) calendar days. Two laboratories (15%) reported their results within 2 weeks. Four laboratories (31%) reported the results of their biomarkers after more than 30 days.Only three laboratories received optimal status (23%). One laboratory (8%) failed due to a critical genotyping error, three (23%) received a suboptimal status due to inappropriately long turnaround times, and the remaining six (69%) received an adequate status. Conclusions: This report demonstrates the utility of this proficiency testing style compared with standard laboratory self-reporting. The approach has elucidated substantial differences in the quality of NSCLC biomarker results produced by Canadian laboratories. Ongoing efforts to improve turnaround times and clarity of reporting, including regular external measurement, are tools that can improve patient outcomes in NSCLC.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.027 | 0.063 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.003 | 0.001 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".