Human-Machine Teaming Considerations Necessary to Develop Trustworthy, Mission Ready Large Language Models
Bibliographic record
Abstract
Large Language Models (LLMs) are changing how people do their work, and the Intelligence Community (IC) is no exception. LLMs can be used for a variety of different mission applications and can potentially save users time and effort. However, LLMs have a tendency to “hallucinate”, or fabricate information. These hallucinations can negate many of the benefits that come from using LLMs and can often call the trustworthiness of these systems into question. Therefore, this research seeks to understand how to develop, refine, and prototype methods and tools to make LLMs mission ready for intelligence analysts. More specifically, this work strives to discern the human-machine teaming considerations that are necessary to understand what intelligence analysts need in order to use LLMs effectively to generate intelligence reports. To ground this work, we start by learning more about analyst’s processes, mental models, and challenges. This enables us to understand analysts’ current processes, so that when developing LLM-enabled tools, we support them in ways that align with known methods. Through published resources from the Five Eyes countries (i.e., Australia, Canada, New Zealand, the United Kingdom, and the United States), as well as first-hand accounts from current and former analysts, we have developed an understanding of where analysts may benefit from the support of LLMs, how they might be used effectively in their processes, and how we might enhance analysts’ trust when using such tools. Through this presentation, we will first discuss our findings from the research and then share our thinking about the tools we are prototyping based off these insights, that not only support analysts in their work, but that also enable a calibrated level of trust between analysts and the LLM-enabled tools they are using.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.025 | 0.120 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.003 | 0.005 |
| Scholarly communication | 0.011 | 0.019 |
| Open science | 0.003 | 0.008 |
| Research integrity | 0.002 | 0.006 |
| Insufficient payload (model declined to judge) | 0.005 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".