Improving generalization performance of electrocardiogram classification models
Bibliographic record
Abstract
Abstract Objective. Recently, many electrocardiogram (ECG) classification algorithms using deep learning have been proposed. Because the ECG characteristics vary across datasets owing to variations in factors such as recorded hospitals and the race of participants, the model needs to have a consistently high generalization performance across datasets. In this study, as part of the PhysioNet/Computing in Cardiology Challenge (PhysioNet Challenge) 2021, we present a model to classify cardiac abnormalities from the 12- and the reduced-lead ECGs. Approach. To improve the generalization performance of our earlier proposed model, we adopted a practical suite of techniques, i.e. constant-weighted cross-entropy loss, additional features, mixup augmentation, squeeze/excitation block, and OneCycle learning rate scheduler. We evaluated its generalization performance using the leave-one-dataset-out cross-validation setting. Furthermore, we demonstrate that the knowledge distillation from the 12-lead and large-teacher models improved the performance of the reduced-lead and small-student models. Main results. With the proposed model, our DSAIL SNU team has received Challenge scores of 0.55, 0.58, 0.58, 0.57, and 0.57 (ranked 2nd, 1st, 1st, 2nd, and 2nd of 39 teams) for the 12-, 6-, 4-, 3-, and 2-lead versions of the hidden test set, respectively. Significance. The proposed model achieved a higher generalization performance over six different hidden test datasets than the one we submitted to the PhysioNet Challenge 2020.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.007 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".