PEAMATL: A Strategy for Developing Near-Infrared Spectral Prediction Models Under Domain Shift Using Self-Supervised Transfer Learning
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Near-infrared (NIR) spectroscopy combined with spectra prediction models has been widely employed as quick and cost-effective analytical techniques in the pharmaceutical, chemical, and food industries. However, calibration has to be conducted for a prediction model being constructed on data from the source domain if we want to apply the model to a new target domain. Most deep transfer learning (DTL) methods, which extract domain-invariant features from the source domain samples and transfer these features to enhance the representation ability for target domain data, are available to calibrate prediction models. However, due to the difficulty of measuring samples’ reference values (label), the reliance on labeled samples for supervised techniques to extract domain-invariant features remains a major bottleneck. In this study, we propose a novel self-supervised TL (SSTL) approach named pyramid external attention model and masked autoencoder (MAE)-based TL (PEAMATL) for learning and transferring generalized domain-invariant features from samples’ spectra, aiming to accurately predict unseen samples’ reference values. PEAMATL first trains a pyramid encoder consisting of three external attention modules (EAMs) to extract multiscale features from unlabeled source domain samples using a self-supervised learning (SSL) framework; second, it transfers the pretrained spectra encoder followed by an initialized prediction head network to build a prediction model; finally, PEAMATL refines the model parameters using a portion of the labeled target domain samples to adapt to unseen target domain samples. The calibration analysis is tested on tablet, melamine, and apple datasets for predicting active pharmaceutical ingredient (API), turbidity point, soluble solid content (SSC), and firmness. Compared with three existing supervised and two self-supervised TL methods, the proposed PEAMATL method achieves at least 3.32%–30.88% prediction error reduction on 19 out of 20 scenarios involving three types of domain shift. Therefore, PEAMATL has the potential to be a generic framework for tackling the common problem of domain shift-induced performance degradation of prediction models in the domain of NIR-based quantitative analysis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it