Implementation, Experiences, Impact, and Costs of Artificial Intelligence in Chest Diagnostics: Protocol for a Mixed Methods Evaluation
Bibliographic record
Abstract
BACKGROUND: The ability to perform complex tasks has seen artificial intelligence (AI) used to support radiology in clinical settings, including lung cancer detection and diagnosis. Evidence suggests that AI can contribute to accurate diagnosis, reduce errors, and improve efficiency. The National Health Service England (NHSE)-funded Artificial Intelligence Diagnostic Fund (AIDF) is currently supporting 12 National Health Service (NHS) networks to implement AI for chest diagnostic imaging. There is, however, limited evidence on real-world AI implementation and use, including staff, patient, and caregiver experience, and costs and cost-effectiveness. A National Institute for Health and Care Research Rapid Service Evaluation Team Phase 1 evaluation provided insights into the early implementation of these tools and developed a framework for monitoring and evaluation of AI tools for chest diagnostic imaging in practice. OBJECTIVE: This mixed methods evaluation of AI tools for chest diagnostic imaging aims to address previous research gaps by exploring the implementation of AI tools for chest diagnostic imaging, the impact and costs of implementing these service models, and the experiences of patients, caregivers, and staff. METHODS: This study will be a mixed method evaluation of implementation, experiences, impact, and costs of AI for chest diagnostic imaging in NHS services in England, with the evaluation informed by the Major System Change Framework. Trust-level case studies (3 in-depth and up to 9 light-touch) will be performed, including staff member, patient, and caregiver; NHSE AIDF team interviews; meeting observations; and analysis of key relevant documentation. Qualitative data will be analyzed using Rapid Assessment Procedures and inductive thematic analysis, supplemented by in-depth deductive thematic analysis. Data from case study sites and other relevant sources will be used to assess outcomes at the other sites and for comparators. A pragmatic economic model of the chest diagnostic imaging pathway will be developed to estimate key costs and resource use associated with AI tool deployment. Together with input from national stakeholders and staff workshops, the study findings will then be finalized for reporting. RESULTS: As of September 2025, trust-level research and development approvals with participating sites are complete, and data collection has commenced. Results are expected to be reported by the end of February 2026. CONCLUSIONS: The study will provide new insights into the facilitators and barriers to the adoption of AI technology in health care and the perceptions of both the general public and health care staff on its use. It will also inform best practices in approaches for service performance evaluation, for the implementation of AI into existing care pathways, and for the development of models to best support evidence-based decision-making. It will thus establish a framework upon which the greatest benefits of the use of AI in health care can be realized. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/81421.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.008 | 0.007 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".