PEAMATL: A Strategy for Developing Near-Infrared Spectral Prediction Models Under Domain Shift Using Self-Supervised Transfer Learning
Bibliographic record
Abstract
Near-infrared (NIR) spectroscopy combined with spectra prediction models has been widely employed as quick and cost-effective analytical techniques in the pharmaceutical, chemical, and food industries. However, calibration has to be conducted for a prediction model being constructed on data from the source domain if we want to apply the model to a new target domain. Most deep transfer learning (DTL) methods, which extract domain-invariant features from the source domain samples and transfer these features to enhance the representation ability for target domain data, are available to calibrate prediction models. However, due to the difficulty of measuring samples’ reference values (label), the reliance on labeled samples for supervised techniques to extract domain-invariant features remains a major bottleneck. In this study, we propose a novel self-supervised TL (SSTL) approach named pyramid external attention model and masked autoencoder (MAE)-based TL (PEAMATL) for learning and transferring generalized domain-invariant features from samples’ spectra, aiming to accurately predict unseen samples’ reference values. PEAMATL first trains a pyramid encoder consisting of three external attention modules (EAMs) to extract multiscale features from unlabeled source domain samples using a self-supervised learning (SSL) framework; second, it transfers the pretrained spectra encoder followed by an initialized prediction head network to build a prediction model; finally, PEAMATL refines the model parameters using a portion of the labeled target domain samples to adapt to unseen target domain samples. The calibration analysis is tested on tablet, melamine, and apple datasets for predicting active pharmaceutical ingredient (API), turbidity point, soluble solid content (SSC), and firmness. Compared with three existing supervised and two self-supervised TL methods, the proposed PEAMATL method achieves at least 3.32%–30.88% prediction error reduction on 19 out of 20 scenarios involving three types of domain shift. Therefore, PEAMATL has the potential to be a generic framework for tackling the common problem of domain shift-induced performance degradation of prediction models in the domain of NIR-based quantitative analysis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".