Snke OS 3D Lung CT Segmentation Challenge
Bibliographic record
Abstract
This is the structured challenge design document for the "Snke OS 3D Lung CT Segmentation Challenge". More details can be found on the challenge's website. The structured design was introduced by the <strong>B</strong>iomedical <strong>I</strong>mage <strong>A</strong>nalysis Challenge<strong>S</strong> (BIAS) initiative. <strong>Background:</strong> Since the outbreak of the global Covid19 pandemic, the number of confirmed COVID-19 cases has reached over 16 million globally [1, 2], affecting virtually every territory, and with a fatality rate ~2-3% among the cohort of PCR-positive cases. Given the high demand for effective diagnosis and treatment of cases, the WHO recently released a rapid advice guide in July 2020 [3], in which chest imaging is conditionally recommended for several purposes, e.g. to aid diagnosis in the absence/delay of PCR testing, to assess the need for ICU admission and to inform the therapeutic management of patients. <br> <strong>Purpose:</strong> In this challenge, we aim to aid radiologists and physicians through objective and quantitative computational assessment of chest imaging in the context of COVID-19. We provide access to a large dataset of 3D chest CT imaging of the lung, collected from several European and international radiological centers. We call the international research community to develop and test artificial intelligence algorithms on this dataset. <strong>Dataset:</strong> We provide access to low-dose chest CT imaging volumes from a mixed cohort of COVID-19 and non- COVID-19 cases. The dataset contains 113 labeled/segmented cases (79 COVID-19, 34 non-COVID-19), and >100 unlabeled volumes. A particular scientific challenge will lie in the effective use of unlabeled data through semi- and self-supervised training techniques. Labels represent five lung lobes and two lesions types, consolidation and ground-glass opacities. Labels are provided in a multi-hot encoding to allow region overlaps (e.g. lesions within lung lobes). For local development, we provide a realistic toy dataset of 96 synthetic volumes with 4D labelmaps. <strong>Infrastructure:</strong> To maintain privacy, the anonymized imaging data remains non-disclosed within a biobank. Participating teams can design their algorithms locally using the representative synthetic dataset. Once ready, teams can submit training and validation jobs on the real dataset through Eisen, a deep learning framework based on pyTorch. Models are trained in the cloud by sponsorship of AWS. We actively promote open science, and require all participating teams to provide their solutions open-source to the technical and medical research community. <strong>Participation:</strong> You can participate in two ways. <em>Hunters:</em> Participate as a team with a maximum of 3 members as a competing team in the challenge. The incentive to the hunters: AWS cloud credits worth 7,500 EUR. <em>Rangers:</em> Participate individually or in a team to help solve the Covid-19 challenge. You can submit tutorials, code or any educational material that is useful for the challenge. The incentive to the rangers: TBA. <strong>Requirements:</strong> After the registration, there will be a “micro challenge” with the task of segmentation based on our synthetic toy dataset, for all teams in order to qualify for the main task.<br> <strong>References</strong> [1] Bell, D.J. COVID-19. https://radiopaedia.org/articles/covid-19-4 [2] ACR. ACR Recommendations for the use of Chest Radiography and Computed Tomography (CT) for Suspected COVID-19 Infection. https://www.acr.org/Advocacy-and-Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-for-Suspected-COVID19-Infection [3] WHO - Radiation and health. Use of chest imaging in COVID-19. https://www.who.int/publications/i/item/useof-chest-imaging-in-covid-19 <strong>UPDATES</strong> 1st september 2020: Updated the schedule
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.374 | 0.057 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".