MétaCan
Menu
Back to cohort
Record W4393712493 · doi:10.5281/zenodo.4272063

Labels for Emergency Response Imagery from Hurricane Barry, Delta, Dorian, Florence, Ida, Isaias, Laura, Michael, Sally, Zeta, and Tropical Storm Gordon

2022· dataset· en· W4393712493 on OpenAlexaff
Evan B. Goldstein, Daniel Buscombe, Eli D. Lazarus, Katherine Anarde, Andrew D. Ashton, Tomas Beuzen, Katherine A. Castagno, Nicholas Cohn, Matthew P. Conlin, Isamar Cortés, Andrew S. Davey, Ashley Ellenson, Megan N. Gillen, Paige A. Hovenga, Ryan Mieras, Somya D. Mohanty, Michael Mungaray, Jin‐Si R. Over, Rose V. Palermo, Katherine Ratliff, Ian Reeves, Lily H. Sanborn, Rajesh Sigdel, William Speiser, Jacob Stasiewicz, Jessamin A. Straub, Steven Tagner, Luke Taylor, Elizabeth Wallace, Phillipe A. Wernette, Hannah Williams

Bibliographic record

VenueZenodo (CERN European Organization for Nuclear Research) · 2022
Typedataset
Languageen
FieldEarth and Planetary Sciences
TopicTropical and Extratropical Cyclones Research
Canadian institutionsUniversity of British Columbia
Fundersnot available
KeywordsTropical cycloneStormHistoryMeteorologyGeography

Abstract

fetched live from OpenAlex

The csv files contain human-generated labels for Emergency Response Imagery collected by US National Oceanic and Atmospheric Administration (NOAA) after Hurricane Barry, Delta, Dorian, Florence, Ida, Isaias, Laura, Michael, Sally, Zeta, and Tropical Storm Gordon. All authors contributed to labeling the imagery. All labeling was done with an open-source labeling tool (Rafique et al., 2020). All csv files provide the userID (the ID of the anonymous labeler), the NOAA flight, the NOAA image, and 6 labels — allWater (if the image was all water), devType (if the image had buildings/development), washoverType (if the image had washover deposits), dmgType (if the image showed damage to built environment), impactType (if the labeler could identify the coastal impact, using the Storm Impact Scale from Sallenger, 2000), and terrainType (the type of physical environment). Images labeled here correspond to multiple NOAA flights — all listed in the csv file for each jpeg image. These jpeg images can be downloaded directly from NOAA (https://storms.ngs.noaa.gov/) or using Moretz et al. (2020a, 2020b). There are three csv files: ReleaseData_10172022.csv has 10,237 labels for 4250 images. These labels were generated by coastal scientists. The csv also contains the Latitude and Longitude of the image center (from NOAA). ReleaseDataQuads.csv has 400 labels for 100 images. These labels were generated by coastal scientists. The images labeled in this set correspond to original NOAA images that have been split into quadrants. Splitting images was done with ImageMagick. The command used to split the images was: `magick mogrify -crop 2x2@ +repage -path ../quadrants *.jpg` The naming convention corresponds to the image quarter — the *-0.jpg is upper left, *-1.jpg is upper right, *-2.jpg is lower left, and *-3.jpg is the lower right. ReleaseDataNCE.csv has 400 labels for 100 images. These images were labeled by non-coastal scientists. Note that the 100 images were also labeled by coastal scientists — those labels can be found in ReleaseData_v3.csv. There is another companion dataset to this, with slightly different labels (Goldstein et al., 2020). A zip file of images is also provided for demonstration purposes (images.zip). These are resized copies made with imagemagick, with the longest dimension set at 2000 pixels ( `mogrify -resize 2000x2000`). For full size images, please download the jpegs directly from NOAA.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.001
metaresearch head score (Gemma)0.001
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMeta-epidemiology (narrow), Science and technology studies, Insufficient payload (model declined to judge)
Consensus categoriesInsufficient payload (model declined to judge)
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Dataset · Consensus signal: Dataset
Teacher disagreement score0.398
Threshold uncertainty score1.000

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0010.001
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0010.000
Bibliometrics0.0000.001
Science and technology studies0.0030.000
Scholarly communication0.0010.000
Open science0.0020.001
Research integrity0.0000.001
Insufficient payload (model declined to judge)0.4010.003

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.025
GPT teacher head0.240
Teacher spread0.216 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; both teacher heads agree on what is shown here.

Study designNot applicable
Domainnot available
GenreDataset

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations0
Published2022
Admission routes1
Has abstractyes

Explore more

Same venueZenodo (CERN European Organization for Nuclear Research)Same topicTropical and Extratropical Cyclones ResearchFrench-language works237,207