Providing data analytic services for national knowledge users in a federated system: learnings from two Canadian use cases
Bibliographic record
Abstract
Objective and ApproachCanada’s federated health data system along with access pathways geared toward academic research, pose challenges for knowledge Users (KUs) requiring timely, pan-Canadian evidence to inform decisions. To understand challenges and explore solutions, we conducted two use cases for a pan-Canadian health technology assessment organization and a federal agency, using health administrative data at one federal and six provincial data centres. The population-based cohort studies described socio-demographics, comorbidities, treatment patterns, service utilization and costs. One focused on spinal muscular atrophy, a rare disease and the other on dementia, a complex chronic disease. ResultsAdministrative challenges included aligning varied ethical review and data access policies/procedures including requiring local and/or academic principal investigators and differing definitions of “research” vs. planning, evaluation, and monitoring. Data-related challenges included differences in structure, timeliness, and completeness across regions resulting in difficulty aligning constructs such as incident cases and episodes of care. Privacy requirements prohibited pooling jurisdictional estimates resulting in “small cells” that couldn’t be shared with KUs. ConclusionsWe provided analytic outputs from multiple regions, albeit with some differences and gaps, increasing knowledge around both diseases while developing capacity for combined analyses and gaining insight into national possibilities for data access. ImplicationsAs decision makers must rely on best available information, some data is better than none. However, to improve data-analytic services for Pan-Canadian KUs, next steps will include improving data harmonization, expanding data assets and filling data gaps, implementing common data models, and exploring options for federated and/or pooled analyses.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.138 | 0.199 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.010 |
| Science and technology studies | 0.026 | 0.011 |
| Scholarly communication | 0.016 | 0.009 |
| Open science | 0.007 | 0.013 |
| Research integrity | 0.004 | 0.005 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".