121.7: Validation study of the new Canadian deceased organ donation dataset and metrics
Bibliographic record
Abstract
Introduction: An environmental scan of Canadian donor audit (DA) practices in 2019 revealed significant variability in donor definitions, data collection, and performance reporting. To address this, in 2021, a Canadian forum established a national deceased donation minimum data set (MDS), donor definitions, and reporting metrics (Figure). Forum participants recommended piloting and assessing feasibility of data collection before implementation. We aimed to validate the feasibility of ODO collecting the MDS variables. Secondary objectives were to establish agreement on data variables to collect for each donor definition, validate the reporting metrics, and develop knowledge translation tools to support provincial implementation. Figure: Deceased donation branching logicMethod: All Canadian organ donation organizations (ODO) were invited to participate. ODO completed an evaluation tool while reviewing patient charts to establish the feasibility (available, potentially accessible, not available) of collecting data variables in the MDS. The evaluation tool consisted of 40 variables, distributed between seven donor definitions. We set a benchmark feasibility threshold of 80%, meaning each data variable could be collected 80% of the time (either currently or as possible to collect). To establish agreement on which of the 40 variables should be collected for each of the seven donor definitions, we employed a modified Delphi methodology using an iterative survey and feedback. Finally, we sought ODO input on the development of knowledge translation tools to support implementation. Results: 8/11 (73%) ODOs participated. ODO piloted collected the MDS in three iterative rounds with a total of 533 patient charts reviewed. 34/40 (85%) variables met an 80% threshold of being currently collected or possible to collect. Four of six variables not meeting threshold focused on diversity, equity, and inclusion. Our threshold of 70% consensus on variable grouping by donor definition was obtained after two modified Delphi survey rounds. Knowledge translation tools were drafted to support implementation with ODO input. Conclusion: The recommended Canadian deceased donation MDS can be feasibly collected by the majority of Canadian ODO. Data groupings and calculated metrics now align with current ODO practices. The data collected through the MDS and data metrics can support pan-Canadian and jurisdictional estimations of donor potential, quantify actual numbers of deceased donors, assess system performance, and identify missed donation opportunities for quality improvement. This study has provided ODO with an opportunity to familiarize staff with the proposed donor definitions, data elements, and reporting metrics to smooth future implementation. Additionally, expert and user led development of knowledge translation tools will strengthen adoption of the proposed MDS. Organ Donation and Transplantation Collaborative. Canadian Blood Services.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.045 | 0.097 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.007 | 0.010 |
| Science and technology studies | 0.005 | 0.002 |
| Scholarly communication | 0.004 | 0.002 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".