Identifying metadata commonalities across restricted health data sources: A mixed methods study exploring how to improve the discovery of and access to restricted datasets
Bibliographic record
Abstract
Background: While open datasets are adopting FAIR principles to improve their discovery and use, restricted data—those only accessible via request or application—have fallen behind. Metadata is not an inherent characteristic of restricted data, which limits its ability to be found and used. To better understand discoverability and accessibility of restricted data, this study reviewed restricted health data sources to determine how they describe their datasets and access procedures, what descriptive commonalities exist across data sources, and to what extent the commonalities we found can be accommodated within existing metadata schemas. Methods: This study extracted dataset and access information provided by a sample of 48 restricted data sources, identified commonalities across these data sources to develop possible metadata elements for restricted data, and mapped these metadata elements to existing metadata schemas (e.g., DataCite) to evaluate how well they accommodate information supplied by restricted data sources. Results: Restricted data sources describe their datasets (35 commonalities) and access procedures (27 commonalities) in similar ways. Dataset descriptions aligned with existing metadata schemas, with the DDI-Lifecycle and -Codebook schemas receiving 91.4% and 85.7% exact matches respectively with the dataset elements we identified. Access procedures did not align with metadata available in existing schemas. Discussion: While descriptive dataset metadata for restricted data sources will make their data more findable, the accessibility of these datasets could be significantly improved by structured metadata capturing data access information. Presently, metadata schemas do not accommodate the level of detail restricted data sources provide about access procedures and requirements.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.213 | 0.347 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.003 |
| Bibliometrics | 0.010 | 0.008 |
| Science and technology studies | 0.004 | 0.003 |
| Scholarly communication | 0.007 | 0.009 |
| Open science | 0.003 | 0.008 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".