Implementation plans for the data reduction pipeline for METIS at the ELT
Bibliographic record
Abstract
METIS will be the first-light mid-infrared instrument at the ELT. Given the expected performance of the ELT’s adaptive optics systems, METIS will be able to probe regions of the sky previously inaccessible to astronomers. In support of both the METIS integration and verification efforts as well as the astronomical community at large, the METIS pipeline team has begun work on the METIS data reduction pipeline. The METIS pipeline will be written mostly in Python to take advantage of the new data reduction tools released by ESO. The development schedule has been set in such a way that the pipeline team will be able to directly support the testing and verification efforts during the upcoming system integration phase for METIS. In order to ensure that the required pipeline functionality is available when it is needed, the recipes and workflows functionality has been broken down into four levels of readiness: skeleton, functional, performance, and science-grade. This breakdown aims to ensure a more agile approach to the pipeline implementation as well as enabling productive contributions from all members of the highly geographically distributed team.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.016 | 0.018 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.001 | 0.004 |
| Insufficient payload (model declined to judge) | 0.041 | 0.039 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".