Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent
Bibliographic record
Abstract
PURPOSE: Individual-level state-transition microsimulations (iSTMs) have proliferated for economic evaluations in place of cohort state transition models (cSTMs). Probabilistic economic evaluations quantify decision uncertainty and value of information (VOI). Previous studies show that iSTMs provide unbiased estimates of expected incremental net monetary benefits (EINMB), but statistical properties of iSTM-produced estimates of decision uncertainty and VOI remain uncharacterized. METHODS: We compare iSTM-produced estimates of decision uncertainty and VOI to corresponding cSTMs. For a 2-alternative decision and normally distributed incremental costs and benefits, we derive analytical expressions for the probability of being cost-effective and the expected value of perfect information (EVPI) for cSTMs and iSTMs, accounting for correlations in incremental outcomes at the population and individual levels. We use numerical simulations to illustrate our findings and explore the impact of relaxing normality assumptions or having >2 decision alternatives. RESULTS: iSTM estimates of decision uncertainty and VOI are biased but asymptotically consistent (i.e., bias approaches 0 as number of microsimulated individuals approaches infinity). Decision uncertainty depends on 1 tail of the INMB distribution (e.g., P[INMB <0]), which depends on estimated variance (larger with iSTMs given first-order noise). While iSTMs overestimate EVPI, their direction of bias for the probability of being cost-effective is ambiguous. Bias is larger when uncertainties in incremental costs and effects are negatively correlated since this increases INMB variance. CONCLUSIONS: iSTMs are useful for probabilistic economic evaluations. While more samples at the population uncertainty level are interchangeable with more microsimulations for estimating EINMB, minimizing iSTM bias in estimating decision uncertainty and VOI depends on sufficient microsimulations. Analysts should account for this when allocating their computational budgets and, at minimum, characterize such bias in their reported results. HIGHLIGHTS: Individual-level state-transition microsimulation models (iSTMs) produce biased but consistent estimates of the probability that interventions are cost-effective.iSTMs also produce biased but consistent estimates of the expected value of perfect information.The biases in these decision uncertainty and value-of-information measures are not reduced by more parameter sets being sampled from their population-level uncertainty distribution but rather by more individuals being microsimulated for each parameter set sampled.Analysts using iSTMs to quantify decision uncertainty and value of information should account for these biases when allocating their computational budgets and, at minimum, characterize such bias in their reported results.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.020 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".