Large-scale and fine-grained phenological stage annotation of herbarium specimens datasets
Bibliographic record
Abstract
This upload is constituted of four datasets of specimens from American herbaria covering different levels of information precision and different floras - from temperate to equatorial. Three of these datasets consist of selected specimens from herbaria located in different geographic and environmental regions. Each specimen of these three datasets was annotated with the following fields: family, genus, species name, fertile / non-fertile, presence / absence of flower(s), presence / absence of fruit(s). The resulting dataset was composed of 163,233 herbarium specimens belonging to 7,782 species, 1,906 genera, and 236 families. Specimens were annotated as “fertile” if any reproductive structures were present, such as sporangia (ferns), cones (gymnosperms), flowers, or fruits (angiosperms). Non-fertile specimens were those that lacked any reproductive structures. The fourth dataset consists of 20,371 herbarium specimens from 11 genera in the sunflower family (<em>Asteraceae</em>). The main difference in this dataset is that it is annotated with fine-grained phenophase scores rather than presence/absence attributes (see description below). Each of these datasets is described below: NEVP: this dataset of New England vascular plant (NEVP) specimens was produced by members of the Consortium of Northeastern Herbaria. The dataset comprises 42,658 digitized specimens that belong to 1,375 species and come from several North American institutions. Most of the specimens in this dataset are from the north-temperate region of the northeastern United States. FSU: this dataset was produced by the Florida State University's Robert K. Godfrey Herbarium (FSU), a collection that focuses on northern Florida and the U.S. Southeast Coastal Plain, one of North America's biodiversity hotspots. This dataset contains 54,263 digitized herbarium specimen records that belong to 3,870 species, making it the taxonomically richest dataset in this study. Most species in this dataset grow under subtropical or warm temperate conditions in the southeastern region of the United States. CAY: this dataset comes from the IRD’s Herbarium of French Guiana (CAY). CAY is dedicated to the Guayana Shield flora, with a strong focus on tropical tree species. This dataset is composed of 66,312 herbarium specimens that belong to 3,024 species. All digitized specimens of this herbarium are accessible online. Most specimens were collected in the tropical rainforests of French Guiana, with the remaining specimens coming mostly from Suriname and Guyana. PHENO: this dataset includes 20,371 herbarium specimens of 139 species in the <em>Asteraceae</em> produced in a study of phenological trends in the U.S. Southeast Coastal Plain. The dataset is composed of specimen records from 57 herbaria. Each recorded specimen was annotated for quartile percentages (0, 25, 50, 75, or 100%) of (i) closed buds, (ii) buds transformed into flowers, and (iii) fruits. According to the distribution of these three categories for each specimen, a phenophase code was computed. <strong>Datasets format</strong> These datasets are grouped in 3 tasks: fertility detection flowers and/or fruit detection phenophase classification The first 2 tasks are carried on the first 3 previous datasets and thus are based on the same set of images, unlike the third task which has its own disjoint set of images. This is why the dataset is presented into two separated files, one for each set of images. <em>Fertility detection & flower/fruit detection</em> These tasks are contained into the <em>herbarium_fertility_annotations.zip</em> archive. It consists of 3 files: <em>metadata.csv</em>: general information about all the herbarium specimens for these tasks <em>id</em>: specimen identifier <em>collection</em>: which of NEVP, FSU or CAY does the specimen come from <em>herbarium</em>: institution of origin of the specimen, especially for NEVP collection <em>clade</em>,<em> family</em>,<em> genus</em>,<em> species</em>: classification of the specimen <em>URL</em>: URL of the scan <em>fertility_task.csv</em>: specific information regarding the fertility detection task <em>id</em>: specimen identifier <em>is_fertile</em>: <em>True</em> if the specimen has an expression of fertility, <em>False</em> otherwise <em>train_test_set</em>: which subset does the specimen belong to; possible values are: <em>train</em>, <em>random_test</em>, <em>species_test</em> and <em>herbarium_test</em> <em>flower_fruit_task.csv</em>: specific information regarding the flower/fruit detection task <em>id</em>: specimen identifier, note that in this case not all the specimen described in <em>metadata.csv</em> are included in this task <em>has_flower</em>: <em>True</em> if the specimen has at least one flower, <em>False</em> otherwise <em>has_fruit</em>: <em>True</em> if the specimen has at least one fruit, <em>False</em> otherwise <em>train_test_set</em>: which subset does the specimen belong to; possible values are: <em>train</em>, <em>random_test</em>, <em>species_test</em> and <em>herbarium_test</em> <em>Phenophase classification</em> These tasks are contained into the <em>herbarium_asteraceae_phenophase_annotations.zip</em> archive. It consists of a single file: <em>annotations.csv</em>: <em>id</em>: specimen identifier <em>URL</em>: URL of the scan <em>genus</em>: genus of the specimen <em>phenophase</em>: integer from 1 to 9 describing the phenophase of the specimen <em>train_test_set</em>: which subset does the specimen belong to; possible values are: <em>train</em> and <em>test</em> <strong>Additional ressources</strong> More information can be found in the related paper:<br> <em>Lorieul, T., K. D. Pearson, E. R. Ellwood, H. Goëau, J.-F. Molino, P. W. Sweeney, J. M. Yost, J. Sachs, E. Mata-Montero, G. Nelson, P. S. Soltis, P. Bonnet, and A. Joly. 2019. Toward a large-scale and deep phenological stage annotation of herbarium specimens: Case studies from temperate, tropical, and equatorial floras. Applications in Plant Sciences 7(3): e1233.</em> For an example of usage of these datasets as well as a baseline, see: http://doi.org/10.5281/zenodo.2549996
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.934 | 0.015 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".