Considerations and Consequences when using First Nations Identifiers in Administrative Data Research.
Bibliographic record
Abstract
ObjectivesLack of consistent and relevant Indigenous identifiers in Canadian data sources leads to misclassification and under-recognition of the health and social issues impacting Indigenous Peoples, further perpetuating the harms of colonization. We are evaluating and optimizing our approach for identifying First Nations (FN) individuals in administrative data in Manitoba. MethodsIn partnership between the First Nations Health and Social Secretariat of Manitoba and the Manitoba Centre for Health Policy, we sought to evaluate to what extent four distinct Manitoba datasets derived from surveillance and social programs were able to identify FN individuals among a cohort of children in low-income Winnipeg neighbourhoods between 2005-2016. In choosing data sources to identify FNs, the population and research question were considered. We then compared the number of FNs identified in each dataset to the First Nations population research file, considered gold standard, but cognizant of its roots in colonial systems of registration. ResultsThe total cohort comprised N=78,864; among these, n=27,347 children were identified as FN in either the FN registry or at least one of the datasets. The FN registry was the most sensitive dataset and identified 85.7% of these individuals. The two program datasets identified 58.2% and 46.2%, and the surveillance-based datasets each identified less than 10%. The First Nations Registry is critical to accurately identify FN individuals. Our analyses demonstrated that without it we would miss 23.5% of those identified as FN in at least one other dataset. However, using it alone could potentially cause us to miss 15% of FN individuals identified in other datasets. The proportions of FN individuals that would be missed by excluding any of the other datasets were smaller (<6%). ConclusionWe found inconsistent FN identification across the datasets evaluated. Among the issues is that some datasets rely on self-identification. In Manitoba, no single administrative dataset can reliably and comprehensively identify FN individuals, and linking multiple datasets together is currently our best approach.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.686 | 0.823 |
| Meta-epidemiology (narrow) | 0.002 | 0.003 |
| Meta-epidemiology (broad) | 0.003 | 0.003 |
| Bibliometrics | 0.014 | 0.024 |
| Science and technology studies | 0.012 | 0.029 |
| Scholarly communication | 0.018 | 0.017 |
| Open science | 0.009 | 0.013 |
| Research integrity | 0.007 | 0.011 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".