Bibliographic record
Abstract
Introduction: To make itself more relevant in a longer perspective health technology assessment (HTA) will have to make use of novel ways to improve its services; in particular in terms of rapid response, cost savings and reduction of risk of bias. The use of artificial intelligence (AI) offers significant assistance at essentially all stages of any HTA. It can search, retrieve, read and organize relevant literature, not only from traditional databases but from numerous data sources related to specific issues (e.g. clinical trials, health outcomes, payment of services), and from databases in other areas such as in social, justice, and educational services, and public health. Methods: This presentation will explain the use and feasibility of AI in HTAs based on the findings from a currently ongoing project in the province of Alberta Canada. It will (i) provide an overview of AI in healthcare, (ii) outline selected international efforts of using AI in systematic reviews, such as the Robotreviewer, (iii) describe the information needed, and the development of the algorithms for using AI in HTAs, and (iv) report on the findings from a comparative study of human versus AI resources in performing an HTA. Results: This project has just started, however preliminary findings from the comparative analysis of AI versus human performance on a specific topic for HTA will be presented. Conclusions: It is expected that the comparative study will demonstrate that artificial intelligence will become a useful tool in HTA in that it will significantly speed up systematic reviews, and decrease the risk of bias in syntheses of findings from research.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.052 | 0.133 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.009 | 0.013 |
| Science and technology studies | 0.001 | 0.005 |
| Scholarly communication | 0.011 | 0.006 |
| Open science | 0.002 | 0.005 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.036 | 0.007 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".