Analyzing and Visualizing Uncertain Knowledge: The Use of TEI Annotations in the PROVIDEDH Open Science Platform
Bibliographic record
Abstract
The underlying uncertainty in digital humanities research data affects decision-making and persists during a project’s lifecycle. This uncertainty is inevitable since most empirical claims cannot be assessed against an absolute truth (Drucker 2011; Binder et al. 2014). This situation has been previously recognized together with the need to report the degrees of uncertainty that accompany such claims (Blau 2011). Although TEI makes it possible to annotate text with notions of certainty or precision, examples of actual projects taking advantage of this are scarce. There are many possible explanations for uncertainty’s lack of visibility in computationally supported humanities research; among them, the need for tools specifically designed to address the goal of defining and managing uncertainty stands out. Thus, efforts to provide technical support for humanities research should focus on managing and making uncertainty more transparent, rather than removing it. Another challenge is the fact that there is no agreement on a generic taxonomy for the different types of uncertainty that researchers may face. Various researchers across disciplines, working on varying projects and data sets, can use different categories to classify the uncertainties present in a particular case. In this paper, we introduce a collaborative platform for collective annotation of TEI data sets. We briefly present the flexible taxonomy of uncertainty used in the platform and describe two data sets used for its testing. Then we describe use cases of annotations available on the platform, and how they translate into TEI annotations. Creating and interpreting annotations with and without uncertainty should now be easier, especially for researchers who do not know TEI markup.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.017 | 0.055 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.008 | 0.006 |
| Science and technology studies | 0.003 | 0.003 |
| Scholarly communication | 0.008 | 0.011 |
| Open science | 0.002 | 0.010 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".