The Limits of Policy Analytics: Early Examples and the Emerging Boundary of Possibilities
Bibliographic record
Abstract
Policy analytics has emerged as a modification of traditional policy analysis, where the discrete stages of the policy cycle are reformulated into a continuous, real-time system of big data collection, data analytics, and ubiquitous, connected technologies that provides the basis for more precise problem definition, policy experimentation for revealing detailed insights into system dynamics, and ongoing assessment of the impact of micro-scale policy interventions to nudge behaviour towards desired policy objectives. Theoretical and applied work in policy analytics research and practice is emerging that offers a persuasive case for the future possibilities of a real-time approach to policymaking and governance. However, policy problems often operate on long time cycles where the effect of policy interventions on behaviour and decisions can be observed only over long periods, and often only indirectly. This article surveys examples in the policy analytics literature, infers from those examples some characteristics of the policy problems and settings that lend themselves to a policy analytics approach, and suggests the boundaries of feasible policy analytics. Rather than imagine policy analytics as a universal replacement for the decades-old policy analysis approach, a sense of this boundary will allow us to more effectively consider the appropriate application of real-time policy analytics.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.041 | 0.063 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.008 | 0.010 |
| Science and technology studies | 0.009 | 0.078 |
| Scholarly communication | 0.022 | 0.049 |
| Open science | 0.004 | 0.014 |
| Research integrity | 0.011 | 0.020 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".