Referee report. For: Cell-type classification of cancer single-cell RNA-seq data using the Subsemble ensemble-based machine learning classifier [version 1; peer review: 2 approved with reservations]
Bibliographic record
Abstract
Background The advent of single-cell RNA sequencing (scRNA-seq) has provided a high-resolution overview of the cellular heterogeneity of different tissue types. Manual cell type annotation of gene expression datasets remains a useful but time-intensive task. Ensemble machine learning methods leverage the predictive power of multiple classifiers and can be applied to classify high-dimensional gene expression data. Here, we present a novel application of the Subsemble supervised ensemble machine learning classifier used to classify novel cells with known cell type labels using gene expression data. Methods First, we tested the classification performance of different pre-processing steps used to normalize and upsample the training dataset for the Subsemble using a colorectal cancer dataset. Second, we conducted a cross-validated performance benchmark of the Subsemble classifier compared to nine other cell type classification methods across five metrics tested, using an acute myeloid leukemia dataset. Third, we conducted a comparative performance benchmark of the Subsemble classifier using a patient-based leave-one-out cross-validation scheme. Rank normalized scores were calculated for each classifier to aggregate performance across multiple metrics. Results The Subsemble classifier performed best when trained on a dataset that was log-transformed then upsampled to generate balanced class distributions. The Subsemble classifier was consistently the top-ranked classifier across five classification performance metrics compared to the nine other baseline classifiers and showed an improvement in performance as the training dataset increased. When tested using the patient-based leave-one-out cross-validation scheme, the Subsemble was the top-ranked classifier based on rank normalized scores. Conclusions Our proof-of-concept study showed that the Subsemble classifier can be used to accurately predict known cell type labels from single-cell gene expression data. The top-ranked classification performance of the Subsemble across two validation datasets, two cross-validation schemes, and five performance metrics motivates future development of accurate ensemble classifiers of scRNA-seq datasets.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.069 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.005 | 0.004 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.003 | 0.003 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.004 | 0.003 |
| Insufficient payload (model declined to judge) | 0.649 | 0.402 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".