Reducing the computational cost of automatic calibration through model preemption
Bibliographic record
Abstract
Computational budget is frequently a limiting factor in both uncertainty‐based (e.g., through generalized likelihood uncertainty estimation (GLUE)) and optimization‐based (e.g., through least squares minimization) calibration of computationally intensive environmental simulation models. This study introduces and formalizes the concept of simulation model preemption during automatic calibration. The proposed “model preemption” method terminates a simulation model early to save computational budget if it is recognized through intermediate simulation model results that a given solution (model parameter set) is so poor that it will not benefit the search strategy. The methodology proposed here is referred to as deterministic model preemption because it leads to exactly the same calibration result as when deterministic preemption is not applied. As such, deterministic preemption–enabled calibration algorithms which make no approximations to the mathematical simulation model are a simple alternative to the increasingly common and more complex approach of metamodeling for computationally constrained model calibration. Despite its simplicity, the deterministic model preemption concept is a promising concept that has yet to be formalized in the environmental simulation model automatic calibration literature. The model preemption concept can be applied to a subset of uncertainty‐based and optimization‐based automatic calibration strategies using a variety of different objective functions. Results across multiple calibration case studies demonstrate actual preemption computational savings ranging from 14% to 49%, 34% to 59%, and 52% to 96% for the dynamically dimensioned search, particle swarm optimization, and GLUE automatic calibration methods, respectively.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".