Addressing Challenges for Reliable Machine Learning Model Updates
Bibliographic record
Abstract
The dynamic nature of data and user expectations requires machine learning models to be updated throughout deployment for maximal performance. Updating models presents a set of challenges that must be handled carefully to ensure that performance improves.The non-convex nature of neural network training makes it challenging to update a model without introducing new errors relative to the original model, regardless if accuracy increases overall. Flipping previously correct predictions to incorrect is known as predictive churn and decreases user trust. I expand the understanding of how churn happens by explaining it through the lens of incompatible parameter updates. I use this analysis to motivate the need for a churn reduction method which does not suffer from a stability-plasticity tradeoff. Finally, I propose a solution for reducing predictive churn called accumulated model combination (AMC) that achieves state-of-the-art churn reduction performance. In some cases, the quality of newly gathered data may be lower than the initial training data. This leads to model updates decreasing performance; i.e., deterioration. To understand how deterioration occurs, I use model-dependent noise (MDN) where samples of a particular difficulty are corrupted. I show that depending on the sample difficulty targeted (easy, intermediate, hard), deterioration occurs mainly through two mechanisms: forgetting or learning prevention. Given that hard samples are more likely to be corrupted than easy samples, I demonstrate the potential societal implications as difficulty-based subgroups can coincide with demographic subgroups. I also provide various insights which explain why continual learning is particularly susceptible to deterioration. Lastly, I introduce the feedback loop problem which occurs in settings where data is confounded by model predictions. I investigate the role that clinician trust plays in limiting feedback loops, and also show the feasibility of detecting this phenomenon. I then present a variety of design choices which need to be adjusted to account for feedback loops in order to limit their effect. As part of a first-of-its-kind study on machine learning deployment in healthcare, I investigate to what extent clinician impressions are influenced by an early warning system, and how these predictions subsequently change behaviour and outcomes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.010 | 0.069 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.003 | 0.006 |
| Open science | 0.004 | 0.005 |
| Research integrity | 0.003 | 0.008 |
| Insufficient payload (model declined to judge) | 0.002 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".