A Tautology Worth Repeating: Well-Characterized, Analytically Robust Assays Underpin Reliable Clinical Performance
Bibliographic record
Abstract
It is timely with interest in prostate cancer screening rekindled by the recently revised and updated recommendations issued by the United States Preventative Services Task Force (USPSTF)3 (1) that this cancer-themed issue of the Journal of Applied Laboratory Medicine includes the study by Higgins and coauthors (2), who evaluated the analytical performance of one such screening test designed to avoid the overdiagnosis of indolent prostate cancer, the 4Kscore. Such information is also timely, if not overdue, given that this test has been in use since 2014 with only a single published Abstract (3), presented at the 2015 AACC Annual Meeting, describing the test's analytical performance and stability of its constituent biomarkers in clinical samples. The 4Kscore is marketed as a Laboratory Developed Test by Opko Health and uses levels of 4 kallikrein protein biomarkers in the circulation combined with patient age and digital rectal examination results to determine the likelihood of a patient having aggressive prostate cancer (Gleason score ≥7; Grade group >1). The 4 biomarkers comprise total prostate specific antigen (PSA), free PSA, intact PSA, and human kallikrein-related peptidase 2 (hK2). Intact PSA denotes a subset of free PSA with the main chain intact at the Lys145–Lys146 linkage and accounts for about 10% of all PSA molecules. hK2 circulates at a concentration one-fiftieth of that of total PSA. The relatively low concentration of intact PSA and hK2 compared with total PSA concentration, and the high-sequence homology among these biomarkers, places a premium on using both highly sensitive and specific assays if these fractions are to be reliably differentiated and accurately quantified. Reporting the test result as a personalized risk of aggressive prostate cancer aligns well with the shared decision-making paradigm of current disease management, whereby the patient and the healthcare professional together weigh the pros and cons of intervening, in this case to biopsy the prostate against the likelihood of positive findings. Studies of the 4Kscore, involving more than 25000 patients collectively, have been presented in 18 peer-reviewed scientific publications (360Dx Daily News. Breaking News issued May 18, 2018), including retrospective investigations (4, 5, 6) and prospective testing of large cohorts of men scheduled for prostate biopsy because of an increased total PSA concentration (7, 8). The 4-marker test was demonstrated to distinguish aggressive from low-grade prostate cancer with an area under the receiver operating characteristic (ROC) curve of 0.82, an improvement from the AUC (area under the ROC curve) of 0.75 achieved when only total and free PSA were included in the test (7). The 4Kscore test has been included in the Early Detection Prostate Cancer Guidelines of the National Comprehensive Cancer Network since 2015 and the European Association of Urology Prostate Cancer Guidelines since 2016. Both guidelines recommend that the 4Kscore test can be used for decision-making before a first or repeat prostate biopsy in men with increased PSA concentrations or other clinical symptoms. The test identifies aggressive prostate cancer in a patient population recommended by the National Comprehensive Cancer Network with a sensitivity of 95% and a negative predictive value of 93%. The 4Kscore has the potential to substantially avoid biopsies driven solely by an increased total PSA concentration. For example, in 1 prospective study of 1012 men, 43% of biopsies could have been avoided by the 4Kscore test using a 9% probability of Gleason ≥7 prostate cancer as the decision cutoff (7). Higgins and coworkers have carried out an in-depth evaluation of the analytical performance of the 4Kscore, focusing in particular on intact PSA and hK2, the 2 biomarkers that lack US Food and Drug Administration approval and hence require in-house validation to ensure a reliable and robust Laboratory Developed Test. Using CLSI guidelines, they assess the imprecision, linearity, analytical specificity, and limits of blank, detection, and quantification of the assays of these 2 biomarkers, the matrix equivalence of serum and EDTA plasma, the reference interval of biopsy-negative patients, and the stability of the 4Kscore risk calculated from blood and plasma specimens stored for variable lengths of time at room temperature and 4 °C after the blood was drawn. The latter realistically simulates the conditions specimens would undergo in transit to the centralized laboratory carrying out the testing. The total precision is determined across 4 laboratories offering the test, and as such gives a robust measure of what imprecision could be expected over extended time and presumably over different lots of reagent, although the latter can only be inferred as the authors never explicitly detail the number of reagent lots or preparations in play across the 4 laboratories during the evaluation of precision. This paper serves to highlight that analytical and clinical performance of tests are inextricably intertwined. Reliable clinical application is underpinned by analytically robust, well-characterized assays. It is not an exaggeration to state that the latter is a sine qua non for the former. As Higgins and coworkers state in the final sentence of their paper, “the combination of robustly performing analytical assays with thoroughly characterized analyte stability and matrix-specific profile ensures that the 4Kscore is a highly reliable laboratory-based assay for predicting the risk of aggressive prostate cancer.” United States Preventative Services Task Force prostate specific antigen human kallikrein-related peptidase 2.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.169 | 0.361 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.005 | 0.002 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.003 | 0.037 |
| Scholarly communication | 0.017 | 0.033 |
| Open science | 0.004 | 0.008 |
| Research integrity | 0.017 | 0.033 |
| Insufficient payload (model declined to judge) | 0.007 | 0.011 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".