P2–156: Training for manual hippocampal segmentation based on the EADC‐ADNI harmonized protocol
Bibliographic record
Abstract
A Harmonized Protocol (HP) for manual hippocampal segmentation was defined by a Delphi panel (Boccardi et al., Neurology 2012:78(S1):S04003) and described in a written document. Benchmark images were segmented by 5 HP-expert tracers as the reference gold standard and uploaded on a web-based platform providing a standard training system. Seventeen “Naïve” tracers with ICC>0.80 in hippocampal segmentation based on their local protocols and no previous experience with the HP were recruited from EADC/ADNI (European Alzheimer Disease Consortium/Alzheimer's Disease Neuroimaging Initiative) centres. They were provided with the same HP criteria and segmentation instructions through the web-platform, and with the same version of MultiTracer for manual segmentation. Training images from 10 ADNI subjects for whom benchmark labels were available were divided into three rounds (n=2, n=4, n=4), balanced by magnetic strength field and degree of hippocampal atrophy. Tracers were asked to segment both hippocampi based on the HP and upload segmented images on the platform. Visual feedback was provided showing point by point discrepancies of segmentations versus the reference in color code. Written feedback was provided slice by slice in addition to the visual feedback for the first two rounds. In subsequent rounds tracers were asked to upload the images of the previous round that had been corrected based on the feedback, and to segment 4 additional images. Dice and Jaccard overlapping indices were computed versus the mean of the HP-experts' segmentations. A slice by slice visual quality check (QC) was carried out to match overlapping values with levels of compliance with the HP criteria. The 10 tracers who completed the training so far had very high Dice values (1.5T: median=0.90 range=0.82–0.91; 3T: median=0.91 range=0.89–0.92). Jaccard values (1.5T: median=0.81 range=0.74–0.84; 3T: median=0.83, range=0.80–0.85) better discriminated segmentations based on compliance versus the HP, with values below 0.71 denoting major segmentation mistakes, and values below 0.80 denoting incomplete compliance with the HP criteria. Sample images of tracing and quantitative visual feedback vs benchmark segmentations (blue dotted lines). Red =far from benchmark; green=similar to benchmark; pink=to be evaluated qualitatively.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.034 | 0.047 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.100 | 0.052 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".