Towards Trustworthy Machine Learning in High-Stakes Decision-Making Systems
Bibliographic record
Abstract
Machine learning systems are increasingly applied to tasks of high-stakes decision-making, in areas like healthcare, personal finance, and criminal justice. In these areas, the trustworthiness of the system is essential; beyond just having a high-accuracy tool, a number of other desiderata are required. Stressing that creating trustworthy machine learning is a problem without a well-defined solution, we focus in particular on two of these desiderata: fairness and robustness. To these ends, we discuss three potential approaches to improving the trustworthiness of machine learning systems. In the first, we consider the task of learning representations which guarantee that downstream classifiers yield predictions that align with fairness metrics when trained on transfer tasks, demonstrating how designing an adversarial loss function can correspond to various commonly-used fairness metrics. In the second, we provide an intuitive heuristic for detecting underspecification, show how it can be computed in a post-hoc fashion using only second-order statistics of the trained model, and discuss how we navigate computational hurdles for calculating this score in deep neural networks using techniques for eigenspectrum estimation. In the third, we reframe rejection learning as a task which should be inherently adaptive, depending on the properties of external decision-makers, and show how to formulate this as a mixture-of-experts-type learning objective, studying its impact on both fairness and accuracy. Through theoretical analysis and empirical evidence, we examine the strengths and weaknesses of each approach, and critically discuss how each fits into the larger project of building trustworthy machine learning systems.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.038 | 0.134 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.003 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.002 | 0.012 |
| Scholarly communication | 0.009 | 0.012 |
| Open science | 0.004 | 0.010 |
| Research integrity | 0.006 | 0.011 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".