Designing an API-Based Protocol for the Interoperability of Textual Resources
Bibliographic record
Abstract
Designing a protocol for the interoperability of digital textual resources—or, more simply, a “IIIF for texts”—remains a challenge, as such a protocol must cater to their vastly heterogenous formats, structures, languages, text encodings and metadata. There have been many attempts to propose a standard for textual resource interoperability, from the ubiquitous Text Encoding Initiative (TEI) format to more recent proposals like the Distributed Text Services (DTS) protocol. In this paper, we introduce our proposal called SHINE, which prioritizes instead the ease for software developers to represent and exchange textual resources and their associated metadata. We do so by combining a hierarchical model of textual structure with a flexible metadata scheme in SHINE, and we continue to define and develop it based on user-centered and iterative design principles. Therefore, we argue that SHINE is a protocol for textual interoperability that successfully balances flexibility of resource representation, consistency across resource representation, and overall simplicity of implementation.RésuméConcevoir un protocole pour l’interopérabilité des ressources textuelles numériques – c’est-à-dire, un IIIF pour des textes – demeure un défi, puisqu’un tel protocole doit correspondre à leurs formats considérablement hétérogènes, ainsi qu’à leurs structures, langues, encodages textuels et métadonnées. Il existe déjà plusieurs tentatives de proposer des standards pour l’interopérabilité des ressources textuelles, tel que l’ubiquiste Text Encoding Initiative (TEI – Initiative d’encodage textuel) ou des propositions plus récentes comme le protocole de Distributed Text Services (DTS – Services de texte distribuées). Dans cet article, nous présenterons une proposition que nous appelons SHINE, qui priorise la facilité de la représentation et de l’échange des ressources textuelles et des métadonnées associées pour les développeurs de logiciel. Nous le ferons en combinant un modèle de structure textuelle hiérarchique avec un schéma de métadonnées flexible dans SHINE et nous le définirons et le développerons selon des principes axés sur l’utilisateur et selon des principes de conceptions itératifs. Par conséquent, nous avançons que SHINE est un protocole pour l’interopérabilité textuelle qui équilibre systématiquement la flexibilité de la représentation de ressources, ainsi que la simplicité globale de l’implémentation, pour toute représentation de ressources.Mots-clés: format d’échange; modélisation de documents; métadonnées; infrastructure numériques; interopérabilité
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.034 | 0.037 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.004 | 0.003 |
| Science and technology studies | 0.004 | 0.006 |
| Scholarly communication | 0.011 | 0.019 |
| Open science | 0.005 | 0.012 |
| Research integrity | 0.005 | 0.010 |
| Insufficient payload (model declined to judge) | 0.010 | 0.009 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".