Immunohistochemistry Critical Assay Performance Controls (ICAPC) Reduce Interobserver Variability in the Interpretation of BRAFV600E Immunohistochemistry
Bibliographic record
Abstract
The utility of prognostic and predictive immunohistochemistry biomarkers in the context of cancer is plagued by inconsistent interpretation of results which can lead to poor rates of adoption or inappropriate use of novel therapeutic strategies. To monitor immunohistochemistry assay performance, a new on-slide control motif, Immunohistochemistry Critical Assay Performance Controls (ICAPC) was developed. We hypothesized that the use of these controls by the diagnosing pathologist to interpret BRAFV600E would result in reduced interobserver and intraobserver interpretation errors. A cross-sectional, sequentially obtained sample of surgical pathology cases stained for BRAFV600E was assembled from a single hospital in Vancouver, British Columbia. Half of the cases had normal on-slide controls and the remainder with ICAPC. Results from 6 independent and blinded readers were compared with each other and to the gold-standard pathologic diagnosis with the goal of demonstrating superior interrater agreement with ICAPC relative to standard on-slide controls. Cohen's κ was used to compute pair-wise reader agreements, whereas Fleiss' κ was used to compare to the gold standard. The implementation of ICAPC resulted in statistically significant improvements in the interobserver agreement of BRAF mutation status ascertained by BRAFV600E immunohistochemistry. Half of the readers demonstrated significant improvements in agreement with the gold-standard diagnosis with the addition of ICAPC. Across all readers, the mean increase in κ was 0.14 with a 95% confidence interval of 0.01-0.28 (P=0.04). This study demonstrates that the addition of ICAPC serves to significantly reduce interobserver variability in the assessment of BRAFV600E immunohistochemistry. As such, we recommend that this approach should be used as part of a comprehensive quality management strategy in the setting of histopathology.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.001 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".