Implementation of the ESMO-Magnitude of Clinical Benefit Scale: real world example from the 2022 Israeli National Reimbursement Process
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
In recent years there has been a surge in the number of new medications for the treatment of cancer.1Xin Yu J. Hubbard-Lucey V.M. Tang J. Immuno-oncology drug development goes global.Nat Rev Drug Discov. 2019; 18: 899-900Crossref PubMed Scopus (165) Google Scholar,2Kurzrock R. Kantarjian H.M. Kesselheim A.S. Sigal E.V. New drug approvals in oncology.Nat Rev Clin Oncol. 2020; 17: 140-146Crossref PubMed Scopus (25) Google Scholar Many of these agents are truly innovative and transformative and their incorporation into routine practice has improved outcomes for many patients. These medications often come with a high and ever-increasing price tag, however, making it unsustainable even for the wealthiest health systems to afford all new medications and new indications for established therapies.3Prasad V. Wang R. Afifi S.H. Mailankody S. The rising price of cancer drugs- a new old problem?.JAMA Oncol. 2017; 3: 277-278Crossref PubMed Scopus (20) Google Scholar, 4Petrou P. Assessing the pricing and benefits of oncology products: an update.Expert Rev Pharmacoecon Outcomes Res. 2021; 21: 335-342Crossref PubMed Scopus (4) Google Scholar, 5Vokinger K.N. Hwang T.J. Grischott T. et al.Prices and clinical benefit of cancer drugs in the USA and Europe: a cost–benefit analysis.Lancet Oncol. 2020; 21: 664-670Abstract Full Text Full Text PDF PubMed Scopus (93) Google Scholar The rising prices of effective drugs have led to the development of the 'Cost-Effective but Unaffordable' paradox.6Lomas J. Claxton K. Martin S. Soares M. Resolving the 'cost-effective but unaffordable' paradox: estimating the health opportunity costs of nonmarginal budget impacts.Value Heal. 2018; 21: 266-275Abstract Full Text Full Text PDF PubMed Scopus (45) Google Scholar In order to optimize the balance between improving patient outcomes and maintaining economic sustainability,7Padula W.V. Sculpher M.J. Ideas about resourcing health care in the United States: can economic evaluation achieve meaningful use?.Ann Intern Med. 2021; 174: 80-85Crossref PubMed Scopus (9) Google Scholar many health care systems implemented Health Technology Assessment (HTA) mechanisms to select which new therapies provide enough patient benefit to justify the cost of incorporating them into national insurance coverage schemes. Several tools aim to provide objective methods for the evaluation of the magnitude of clinical benefit generated by novel cancer drugs and indications such as the European Society for Medical Oncology-Magnitude of Clinical Benefit Scale (ESMO-MCBS8Cherny N.I. Sullivan R. Dafni U. et al.A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS).Ann Oncol. 2015; 26: 1547-1573Abstract Full Text Full Text PDF PubMed Scopus (574) Google Scholar), the American Society of Clinical Oncology-Value Framework (ASCO-VF9Schnipper L.E. Davidson N.E. Wollins D.S. et al.American Society of Clinical Oncology statement: a conceptual framework to assess the value of cancer treatment options.J Clin Oncol. 2015; 33: 2563-2577Crossref PubMed Scopus (736) Google Scholar) or the overall value of an intervention such as the National Comprehensive Cancer Network evidence blocks.10Carlson R.W. Jonasch E. NCCN evidence blocks.J Natl Compr Canc Netw. 2016; 14: 616-619Crossref PubMed Scopus (36) Google Scholar Irrespective of the scale that is used, conclusions from these scales may be confounded by biases deriving from study design, study implementation or data analysis.11Gyawali B. de Vries E.G.E. Dafni U. et al.Biases in study design, implementation, and data analysis that distort the appraisal of clinical benefit and ESMO-Magnitude of Clinical Benefit Scale (ESMO-MCBS) scoring.ESMO Open. 2021; 6: 100117Abstract Full Text Full Text PDF PubMed Scopus (31) Google Scholar Thus, in addition to the formal scales, the evaluation of the magnitude of clinical benefit requires a careful assessment of the evidence for potential biases that may have generated exaggerated or reduced benefit or underestimated potential harms. Given the acknowledged limitation of the generalizability of the knowledge gained from clinical trials,11Gyawali B. de Vries E.G.E. Dafni U. et al.Biases in study design, implementation, and data analysis that distort the appraisal of clinical benefit and ESMO-Magnitude of Clinical Benefit Scale (ESMO-MCBS) scoring.ESMO Open. 2021; 6: 100117Abstract Full Text Full Text PDF PubMed Scopus (31) Google Scholar the appraisal of clinical benefit may also be influenced by 'real world data' and the experience of physicians who actively work in the relevant field. The Israeli health system is universal and allows accessible health care to all Israeli citizens. To safeguard economic sustainability, however, not all licensed medications or specific indications are reimbursed. In order to be reimbursed and provided free of charge, licensed drugs, technologies or indications for the treatment of cancer must also be approved by the HTA process of the Ministry of Health (MOH).12Clarfield A.M. Manor O. Nun G.B. et al.Health and health care in Israel: an introduction.Lancet. 2017; 389: 2503-2513Abstract Full Text Full Text PDF PubMed Scopus (132) Google Scholar The HTA process is undertaken on an annual basis by the 'health basket' committee of the MOH,13Seidman G.I. Regulating life and death: the case of Israel's 'Health Basket' Committee.J Contemp Health Law Policy. 2006; 23: 9-63PubMed Google Scholar which includes physicians, administrators representing the payers and prominent public figures representing the public. Each year the government allocates a limited budget dedicated for financing new drugs and technologies for the following year. The health basket committee evaluates and ranks all candidate drugs and technologies to select those to be reimbursed within the allocated budget. These budget constraints mandate a thorough evaluation of each candidate, as well as a method to compare between them. As drugs and technologies may be part of any field of medicine, the comparison is extremely difficult, especially when comparing and choosing from diverse fields such as oncology and hematology on the one hand and preventive medicine, diabetes and hypertension on the other hand. In most recent years ∼30% of the budget expansion was allocated to oncology new drugs,14https://www.israelhayom.co.il/health/article/4850070Date accessed: January 10, 2022Google Scholar however this was smaller in 2021 (8%), due to COVID-19 considerations and also a significant allocation for the treatment of diabetes which had a very large budget impact. Three tiers lead to the final decision: (i) scientific evaluation of each new drug/indication/technology by experts in the specific field relevant for the application who submit a ranked listing of new therapies irrespective of cost (i.e. medical oncologists rank drugs and technologies candidates for solid cancer treatment, hematologists review candidates in hematological diseases, etc.); (ii) financial evaluation by the MOH including the total budget impact of incorporating the new therapy; (iii) final evaluation by the health basket committee. For solid tumor oncology submissions, ranked listing is developed by the Israeli Society of Clinical Oncology and Radiotherapy (ISCORT) together with the National Council for the Prevention, Detection and Treatment of Malignant Diseases (The National Council) which is a body appointed by the MOH, using a two-step structured method.(i)Evaluation and ranking of candidate medications/indications in each specialty field by the dedicated expert group of ISCORT (breast, thoracic, gastrointestinal, etc.). The ranking is informed by ESMO-MCBS scores and by independent review of the data.(ii)Submissions from the subspecialty groups are considered by a steering committee of ISCORT and the National Council to formulate a single ranking list for solid tumor oncology. The ranking is also informed by ESMO-MCBS scores, and the expert input of the subspecialty groups. For the year 2022, 77 new indications for 41 drugs were submitted by the MOH for evaluation (Table A1). The most common drugs were targeted therapies (26, 63%) followed by immune checkpoint inhibitors (ICIs) (7, 17%), chemotherapies (4, 10%) hormonal therapies (2, 5%) and antibody-drug conjugates (2, 5%). The most common indications were targeted therapies (35, 45%) and ICIs (30, 39%). A total of 42 of the 77 indications (55%) were recommended for reimbursement by the professional oncology societies; 30 (39%) indications were recommended for reimbursement from the allocated budget expansion and a further 12 indications (15.5%), which were alternatives to previously approved medications, were recommended for reimbursement but without added cost to the overall budget (Table A2). Treatments with curative intent were given the highest priority. Most submissions for potentially curative intent were recommended for funding (6 of 8, 75%), compared with only 36 of 69 (52%) indications for non-curative treatment. With one exception, the therapies with curative potential that were incorporated in the ranking achieved an ESMO-MCBS score of A. The thirty interventions recommended for reimbursement from the allocated budget expansion were ranked for prioritization. Five adjuvant treatments with curative intent were ranked above treatments with non-curative intent. With three exceptions, all non-curative indications in the top 30 list were scored by the ESMO-MCBS v1.1 as three or above. The exceptions were: (i) olaparib as maintenance therapy for BRCA-mutated, unresectable or metastatic pancreatic cancer (ESMO-MCBS v1.1 score 2) was ranked 20/30, because it delayed the need for further chemotherapy which the ISCORT GI-group members considered as a patient benefit not captured by the POLO study design endpoints; (ii) ipilimumab and nivolumab after progression on a single agent ICI in advanced melanoma (ESMO-MCBS v1.1 not scorable) was ranked lowly (24/30). It was strongly endorsed by the melanoma specialty group since it is a common off-label clinical practice, though it is not yet supported by any prospective randomized study and (iii) the study supporting the indication of pembrolizumab for patients with BCG-unresponsive high-risk, non-muscle invasive bladder cancer15Balar A.V. Kamat A.M. Kulkarni G.S. et al.Pembrolizumab monotherapy for the treatment of high-risk non-muscle-invasive bladder cancer unresponsive to BCG (KEYNOTE-057): an open-label, single-arm, multicentre, phase 2 study.Lancet Oncol. 2021; 22: 919-930Abstract Full Text Full Text PDF PubMed Scopus (191) Google Scholar (ranked 30/30) was a single-arm de-escalation study in the curative setting which is a design that is not scorable using ESMO-MCBS v1.1. This shortcoming is being addressed in the draft version 2.0 which is currently in validation testing. Indications 26-29 were of targeted therapies in rare clinical scenarios with a proven biomarker, tested in early, non-randomized clinical trials, but showing promising efficacy together with a strong biological rationale for their activity. These included sotorasib for non-small-cell lung cancer (NSCLC) harboring KRAS G12C mutation,16Skoulidis F. Li B.T. Dy G.K. et al.Sotorasib for lung cancers with KRAS p.G12C mutation.N Engl J Med. 2021; 384: 2371-2381Crossref PubMed Scopus (657) Google Scholar amivantamab and mobocertinib for NSCLC harboring epidermal growth factor receptor (EGFR) exon 20 insertion17Park K. Haura E.B. Leighl N.B. et al.Amivantamab in EGFR exon 20 insertion–mutated non-small-cell lung cancer progressing on platinum chemotherapy: initial results from the CHRYSALIS phase I study.J Clin Oncol. 2021; 39: 3391-3402Crossref PubMed Scopus (265) Google Scholar,18Riely G.J. Neal J.W. Camidge D.R. et al.Activity and safety of mobocertinib (Tak-788) in previously treated non-small cell lung cancer with EGFR exon 20 insertion mutations from a Phase I/II trial.Cancer Discov. 2021; 11: 1688-1699Crossref PubMed Scopus (139) Google Scholar and pemigatinib for cholangiocarcinoma harboring fibroblast growth factor receptor 2 (FGFR2) alterations.19Abou-Alfa G.K. Sahai V. Hollebecque A. et al.Pemigatinib for previously treated, locally advanced or metastatic cholangiocarcinoma: a multicentre, open-label, phase 2 study.Lancet Oncol. 2020; 21: 671-684Abstract Full Text Full Text PDF PubMed Scopus (799) Google Scholar Evidence for benefit for these indications was derived from single-arm studies with surrogate outcomes of overall response rate and duration of survival. The steering committee considered the rarity of certain driver mutations impacting the feasibility of conducting large registration trials, and accordingly ranked drugs with high response rate and durable response in this setting above what would be dictated by the ESMO-MCBS score. To enhance the value and contain the total budget impact and thus allow inclusion of more drugs, the committee members recommended narrowing some of the indications requested by the sponsors (Table A1). Some of these recommendations were inferred from the indirect comparisons of clinical trials, planned and post hoc subgroup analyses and biological understanding of mechanisms of actions. Thus, the final list recommended that coverage for the use of ICIs in urothelial, esophageal and gastric cancers should be restricted to programmed death-ligand 1-positive tumors only. The 12 indications for which reimbursement decisions did not require additional budget allocation (Table A2) were all for indications where there was little or no added marginal benefit compared with other agents already fully covered by the national insurance scheme, but where listing of an alternative may have a positive economic impact by the introduction of competitive pricing. Since equally effective therapies were already covered by the national insurance scheme, these 12 indications were not ranked for added clinical benefit. Unlike other countries (e.g. Australia, Germany, Austria, France, Canada, Hungary and UK), where submissions are considered on an ad hoc basis, the HTA process in Israel is an annual process whereby all new candidate technologies are considered simultaneously and competitively for funding from constrained budget expansion. Since the annual health basket evaluation mechanism allows only one opportunity for drugs to be reimbursed each year, the ranking process presented here is, to our knowledge, the only annual formal national ranking of all new cancer drugs. Ultimately, this process is not the final determinate. The list and the considerations underlying its development will be presented to the central HTA committee where all the data are reviewed and where value, cost and total budget impact of each indication are additional considerations in the determination of the new therapies to be included in the "free for user" basket of services. This very hands-on experience highlights many of the difficulties challenging HTA processes globally, illustrates the value of incorporating the ESMO-MCBS as part of the process, but also points to the limitations of data objectification where uncertainty regarding the veracity and generalizability of the available data is high. Among the non-curative therapies in addition to the ESMO-MCBS score, clinical experience from the perspective of the members of the relevant subspecialty faculty groups was also influential.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.003 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.010 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it