Improved estimates of organic carbon using proximally sensed vis– <scp>NIR</scp> spectra corrected by piecewise direct standardization
Bibliographic record
Abstract
Summary We investigated the use of piecewise direct standardization ( PDS ) to remove the effects of water and other environmental factors from proximally sensed (field) visible–near infrared (vis– NIR ) spectra. Our hypothesis was that the PDS ‐standardized field spectra can be used to predict soil carbon effectively with calibrations derived from existing spectroscopic databases of spectra recorded in the laboratory on dried, ground and sieved samples. In our experiments we used field spectra recorded in situ with a portable spectrometer at 124 sites in 11 paddy fields in Z hejiang P rovince, C hina. We sampled the soil at these same sites, recorded their spectra in the laboratory and measured their soil organic carbon ( SOC ) contents with a conventional laboratory technique. Two‐thirds of the samples were used to relate the laboratory spectra to SOC by partial least squares regression ( PLSR ), and the remaining one‐third was used as an independent validation dataset. We selected a representative set of samples from corresponding field and laboratory spectra that we could use as the PDS transfer set. Piecewise direct standardization was used to relate each wavelength in the laboratory spectra to the corresponding wavelength and its neighbours in the field spectra. The field spectra of the validation samples were then corrected with PDS so that they acquired the characteristics of the spectra measured under laboratory conditions. The approach was evaluated by (i) quantifying the similarity between the PDS ‐standardized spectra and their corresponding laboratory spectra, (ii) measuring the accuracy of their SOC predictions on the independent validation dataset and (iii) comparing these results with those of direct standardization ( DS ). Both PDS and DS led to considerable improvements in the predictions of SOC ( R 2 = 0.71, R 2 = 0.60, respectively), compared with those with original field spectra ( R 2 = 0.03). However, fewer transfer samples were needed with PDS to obtain similar results.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".