Detection of Systematic Error Using the Average of Deltas
Bibliographic record
Abstract
Traditional quality control (QC) procedures only provide a snapshot of assay performance, and as such laboratories employ alternate QC strategies to detect systematic error (SE). One strategy, the delta check, compares a patient’s most recent chemistry results to historical values. However, delta checks are best suited to detect large SE. Another strategy, moving averages (MA) monitors the mean patient analyte value to detect SE. MA, however, cannot equitably detect SE in skewed patient populations. The objective of our study was to combine the delta check and MA to develop an average of deltas (AoD) strategy that monitors the mean difference between pairs of consecutive, intrapatient results. Methods: From a database of 4.2 million results spanning 638 days, we generated arrays for each assay in our study containing pairs of patient results collected within 18-26 hours of each other. To develop sensitive AoD protocols that detect SE equal to the reference change value for each assay, we employed a simulated annealing algorithm in MatLab (Mathworks, Natick, MA) to select the number of patient pairs to average (Np) and truncation limits to eliminate large deltas. Using MatLab, we simulated SE by adding positive or negative bias at fixed intervals to the arrays of paired results for serum assays of albumin, alanine aminotransferase, alkaline phosphatase, amylase, aspartate aminotransferase, bicarbonate, bilirubin (total and direct), calcium, chloride, creatinine, lipase, sodium, phosphorus, potassium, total protein, and magnesium. For each assay the average number of deltas to detection (ANDD) was calculated in response to induced SE. Results: ANDD varied between AoD protocols assays, and protocol with the lowest ANDD was amylase with a + 20 U/L shift detected with an ANDD of 6.2 intrapatient deltas and a –20 U/L shift detected with an ANDD of 6.6. A shift of 1 mg/dL in calcium was detected with an ANDD of 6.5 and 11.9 patient pairs, respectively. Creatinine had the highest ANDD in our validation set with an ANDD of 44.6 for a + 0.3 mg/dL shift and an ANDD of 43 for a –0.3 mg/dL shift. Conclusions: We have demonstrated that the AoD rapidly detects SE. AoD is complimentary to other QC strategies such as MA in that AoD detects SE for skewed distributions such as amylase which are challenging for MA. AoD’s limitation is that repeated daily patient testing is required; however, for larger inpatient facilities, attaining sufficient numbers of repeated daily labs should not be an issue. AoD analysis as part of a middleware package would enhance detection of SE and would complement other QC strategies already employed in the clinical laboratory.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.021 | 0.075 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.004 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".