OP07 Technical Aspects Of Artificial-Intelligence-Based Tools Applied In Health Technology Assessment Processes: A Scoping Review
Bibliographic record
Abstract
Introduction In the last decade, artificial intelligence (AI) has been increasingly applied in health technology assessment (HTA) to accelerate evidence synthesis, optimize resources allocation, and guarantee timely delivery of trustworthy technologies in health. The aim of the present scoping review is to map AI models applied in HTA, and technical characteristics of AI-based automation and semi-automation applied in HTA. Methods A search strategy containing core expressions “AI” and “HTA” and correlated terms was conducted in nine specialized databases (health and informatics) in February 2022. Inclusion criteria were publications testing AI models applied in HTA. Study selection was performed by independent pairs, with consensus meetings. No filters were applied. Data on year and country of publication, HTA phase, subsets of AI (e.g., machine learning [ML], neural networks), type of algorithm (e.g., support vector machine [SVM], K-nearest neighbors), and performance scale were extracted. Data were analyzed as descriptive frequency statistics. Used metrics will be presented narratively. Results Sixty-one publications were included. The first study identified was published in 2006, and since then the number of publications has been consistently growing, with 11 publications in the year 2021. Canada, USA, and the UK concentrate 72 percent of publications (44 in 61) equally distributed. The most common HTA phase was the evidence synthesis, with 59 studies (96%). The main task performed was study screening/selection (66.6%). The majority of ML models (80.9%) contained two learning nodes or fewer, and applied SVM and decision-tree-based algorithms. Inter-rater agreement, accuracy, and 95 percent recall were the most common scales observed. Conclusions Although recent developments in AI applied to HTA show increasing potentiality, studies are concentrated in the study selection phase of evidence synthesis. Many areas need further development, such as horizon scanning and policymaking processes. Additionally, studies reporting time gain and economic gain outcomes are scarce and should be considered for the development of future studies in the field.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.040 | 0.152 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.005 |
| Bibliometrics | 0.038 | 0.038 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.009 | 0.007 |
| Open science | 0.002 | 0.005 |
| Research integrity | 0.004 | 0.002 |
| Insufficient payload (model declined to judge) | 0.009 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".