Increasing the accountability of automated decision-making systems: An assessment of the automated decision-making system introduced in Canada's temporary resident visa immigration stream
Bibliographic record
Abstract
States are increasingly turning to automated decision-making systems to increase efficiency in program and service delivery. While automation offers several desirable benefits, great care must be given to establishing and increasing the accountability of automated decision-making systems in the public sector. This paper focuses on accountability in automated decision-making systems in migration management. A key issue is what the impact of automated decision-making is on accountability in migration management? This paper seeks to explore this question by evaluating the accountability mechanisms established by the Canadian government in the use of automated decision-making systems to triage Temporary Resident Visa immigration applications. This paper begins with an explanation of the interaction between public administration and digital governance, with a particular focus on the human decision-making component of public administration and a review of accountability in the public sector. What follows is an explanation of how decision-making in Canada's Temporary Resident Visa Application stream traditionally occurs. A brief review of the Canadian Algorithmic Impact Assessment Tool introduces a thorough explanation of the Canadian government's Temporary Resident Visa (TRV) eApps Advanced Analytics Pilot to showcase changes between the traditional human decision-making process and the more recent experiment engaging automated decision-making in this particular immigration stream. The paper concludes by posing a question on what accountability amounts to for the Canadian government and whether the accountability measures introduced in Canada's TRV Pilot are sufficient.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.013 | 0.007 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".