Galaxy merger challenge: A comparison study between machine learning-based detection methods
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Aims . Various galaxy merger detection methods have been applied to diverse datasets. However, it is difficult to understand how they compare. Our aim is to benchmark the relative performance of merger detection methods based on machine learning (ML). Methods . We explore six leading ML methods using three main datasets. The first dataset consists of mock observations from the IllustrisTNG simulations, which acts as the training data and allows us to quantify the performance metrics of the detection methods. The second dataset consists of mock observations from the Horizon-AGN simulations, introduced to evaluate the performance of classifiers trained on different, but comparable data to those employed for training. The third dataset is composed of real observations from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) survey. We also compare mergers and non-mergers detected by the different methods with a subset of HSC-SSP visually identified galaxies. Results . For the simplest binary classification task (i.e. mergers vs. non-mergers), all six methods perform reasonably well in the domain of the training data. At the lowest redshift explored 0.1 < ɀ < 0.3, precision and recall generally range between ~70% and 80%, both of which decrease with increasing ɀ as expected (by ~5% for precision and ~10% for recall at the highest ɀ explored 0.76 < ɀ < 1.0). When transferred to a different domain, the precision of all classifiers is only slightly reduced, but the recall is significantly worse (by ~20–40% depending on the method). Zoobot offers the best overall performance in terms of precision and F1 score. When applied to real HSC observations, different methods agree well with visual labels of clear mergers, but can differ by more than an order of magnitude in predicting the overall fraction of major mergers. For the more challenging multi-class classification task to distinguish between pre-mergers, ongoing-mergers, and post-mergers, none of the methods in their current set-ups offer good performance, which could be partly due to the limitations in resolution and the depth of the data. In particular, ongoing-mergers and post-mergers are much more difficult to classify than pre-mergers. With the advent of better quality data (e.g. from JWST and Euclid ), it is of great importance to improve our ability to detect mergers and distinguish between merger stages.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.000 | 0.002 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it