SPARC: New Cloud, Snow, and Cloud Shadow Detection Scheme for Historical 1-km AVHHR Data over Canada
Bibliographic record
Abstract
Abstract The identification of clear-sky and cloudy pixels is a key step in the processing of satellite observations. This is equally important for surface and cloud–atmosphere applications. In this paper, the Separation of Pixels Using Aggregated Rating over Canada (SPARC) algorithm is presented, a new method of pixel identification for image data from the Advanced Very High Resolution Radiometer (AVHRR) on board the NOAA satellites. The SPARC algorithm separates image pixels into clear-sky and cloudy categories based on a specially designed rating scheme. A mask depicting snow/ice and cloud shadows is also generated. The SPARC algorithm has been designed to work year-round (day and night) over the temperate and polar regions of North America, for current and historical AVHRR/NOAA High-Resolution Picture Transmission (HRPT) and Local Area Coverage (LAC) data with original 1-km spatial resolution. The algorithm was tested and applied to data from the AVHRR sensors flown on board NOAA-6 to NOAA-18. The method was employed in generating historical clear-sky composites for the 1982–2005 period at daily, 10-day, and monthly time scales at 1-km resolution for an area of 5700 km × 4800 km centered over Canada. This region also covers the northern part of the United States, including Alaska, as well as Greenland and the surrounding oceans. The SPARC algorithm is designed to produce an aggregated rating that accumulates the results of several tests. The magnitude of the rating serves as an indicator of the probability for a pixel to belong to the clear-sky, partly cloudy, or overcast categories. The individual tests employ the spectral properties of five AVHRR channels, as well as surface skin temperature maps from the North American Regional Reanalysis (NARR) dataset. These temperature fields are available at 32 km × 32 km spatial resolution and at 3-h time intervals. Combining all test results into one final rating for each pixel is beneficial for the generation of multiscene clear-sky composites. The selection of the best pixel to be used in the final clear-sky product is based on the magnitude of the rating. This provides much-improved results relative to other approaches or “yes/no” decision methods. The SPARC method has been compared to the results of supervised classification for a number of AVHRR scenes representing various seasons (snow-free summer, winter with snow/ice coverage, and transition seasons). The results show an overall agreement between the automated (SPARC) and the supervised classification at the level of 80% to 91%.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".