Bibliographic record
Abstract
To design good policy, we need accurate models of how the decision makers that operate within a given system will respond to policy changes. For example, an economist reasoning about the design of an auction needs a model of human behavior in order to predict how changes to the auction design will be reflected in outcomes; or a doctor deciding on treatments needs a model of people's health responses under different treatments to select the best treatment policy. We would like to leverage the accuracy of modern deep learning approaches to estimate these models, but this setting brings two non-standard challenges. First, decision problems often involve reasoning over sets of items, so we need deep networks that reflect this structure. The first part of this thesis develops a deep network layer that reflects this structural assumption, and shows that the resulting layer is maximally expressive among parameter tying schemes. We then evaluate deep network architectures composed of these layers on a variety of decision problems from human decision making in a game theory setting, to algorithmic decision making on propositional satisfiability problems. The second challenge is that predicting the effect of policy changes involves reasoning about shifts in distribution: any policy change will, by definition, change the conditions under which decision makers operate. This violates the standard machine learning assumption that models will be evaluated under the same conditions as those under which they were trained (the ``independent and identically distributed'' data assumption). The second part of this thesis shows how we can train deep networks that make valid predictions of the results of such policy interventions, by adapting the classical causal inference method of instrumental variables. Finally, we develop methods that are robust to some violations of the instrumental variable assumptions in settings with multiple instrumental variables.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.018 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.002 | 0.004 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.002 | 0.005 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".