Measuring Perceived Risk of Pitfalls Associated with Systems Engineering Tradeoff Analyses
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
The U.S. Department of Defense (DoD) has recently revised the defense acquisition system to address suspected root causes of unwanted acquisition outcomes. One of the major changes in the revised acquisition system is an increased emphasis on systems engineering trade-offs made between capability requirements and lifecycle costs early in the acquisition process (Cilli, Parnell, Cloutier, & Zigh, 2015). Given that systems engineering trade-off analyses will play a pivotal role in future defense acquisition efforts, this paper takes an in-depth look at the state of systems engineering trade-off analysis capability through a review of relevant literature and a survey of systems engineering professionals and military operations research professionals involved in defense acquisition. The survey was developed to measure the perceived level of difficulty associated with compliance to the revised defense acquisition system mandate for early systems engineering trade-off analyses and to measure perceived likelihood and impact of potential pitfalls within systems engineering trade-off studies. The survey instrument was designed using Survey Monkey and was deployed through a link posted on several groups within LinkedIn, a professional social media site, and was also sent directly via email to those with known experience in this research area. Although increased systems engineering activity early in the life cycle is a compelling change for DoD, the findings of the literature review and the survey of practitioners both indicate that there is much to be done in order to position the systems engineering community for success so that the improved defense acquisition outcomes as envisioned by the architects of 2015 DoDI 5000.02 can be realized.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it