Response to redefining health technology assessment: a comment on “the new definition of health technology assessment: a milestone in international collaboration”
Bibliographic record
Abstract
The commentary by Culyer and Husereau (1) criticizes the development of the new definition of health technology assessment (HTA) with its accompanying notes, published in June of 2020 in the Journal (2) and in Value in Health (3), although they do not refer to the latter.The main arguments made by the commentators are that traditional conventions for developing definitions were not followed and the process to develop the definition was ill-described.As authors and participants in the process to develop the new definition, we do not support the arguments presented by the commentators.We provide the following information to respond to their critique, which centers on three main areas.First, we feel they are basing their commentary on a narrow and outdated understanding of HTA methods, processes, and practices.For example, they describe the core disciplines of HTA as biostatistics, economics, and epidemiology.This thinking excludes all the other disciplines and contributors (including users of HTA) from the current and future practice of HTA.For example, Staniszewska and Söderholm Werkö recently showed that clinical and economic evidence are not enough for HTA, and that "for HTA to be complete, we need to consider all relevant aspects of the phenomena, including patient-based evidence" (4).This also requires certain disciplines, such as psychology and ethics, and other skills or processes, such as coproduction.Furthermore, the commentators claim that "all HTA practitioners ought to possess a working knowledge of such entities as 'pandemic', 'specificity', 'median', and 'opportunity cost' is not substantiated."We would like to refer to Mueller et al. ( 5) who recently provided an overview of the core competencies necessary for HTA.They recommend HTA training covering all the required domains of HTA analysis as reflected in the new definition of HTA.Second, we note that the commentators criticize the process of developing the new definition as "an ill-described process."We disagree and reiterate some of the process issues that were articulated in our manuscripts announcing the new definition of HTA (2;3).The new definition with important clarifying information provided in four accompanying notes is not "a work in progress" as suggested by the commentators-it was deliberately developed as an aspirational definition using a structured, transparent, and inclusive process, and it has been well-received by colleagues and stakeholders across the HTA ecosystem.Much has been written about the history of HTA and there has been considerable effort to evaluate, improve, and harmonize the science, methods, and practice of HTA; however, there had never been a global consensus on the definition of HTA.With this as the background, leadership from HTAi and INAHTA asked relevant networks if there was a need to develop a new, internationally accepted definition.As their responses were affirmative, INAHTA and HTAi created a joint international Task Group, coled by INAHTA and HTAi.The goal of the Task Group was to develop an internationally accepted new definition of HTA that incorporated the central concepts of HTA in language that would be easily understood by anyone across different linguistic backgrounds and that was more memorable and aspirational than existing definitions.Membership on the Task Group included representatives appointed by the leadership of all relevant HTA networks and societies: INAHTA, HTAi, EUnetHTA, HTAsiaLink, RedETSA, the HTA Glossary Committee, ISPOR, while the WHO participated as an observer.The Task Group identified a set of guiding principles for developing the new definition and compiled a set of core concepts to include in the definition.They also designed an extensive, iterative consultation plan where draft versions of the definition were reviewed by the Boards of the organizations represented in the international joint Task Group, and an open consultation process to seek input from the broader HTA community.The Task Group relied heavily on input from the member representing the HTA Glossary Committee to ensure we followed generally accepted principles of lexicography, policies and procedures from the HTA Glossary Committee, and that we were congruent with ISO standards.The HTA Glossary Committee, while consisting of experts in HTA, also has representation from the Canadian Translation Bureau who ensures that definitions comply with the ISO standards
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.021 | 0.109 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.003 | 0.003 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.010 | 0.010 |
| Scholarly communication | 0.007 | 0.010 |
| Open science | 0.007 | 0.005 |
| Research integrity | 0.112 | 0.094 |
| Insufficient payload (model declined to judge) | 0.008 | 0.008 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".