Imprecision in tuberculosis infection outcomes; implications for non-inferiority vaccine trials
Bibliographic record
Abstract
Abstract Introduction Randomised trials comparing new vaccines against tuberculosis for use in neonates and infants, for whom Bacille Calmette-Guérin (BCG) vaccination is established practice, are using tuberculosis infection as the primary endpoint in a non-inferiority design. Markers of tuberculosis infection have imperfect sensitivity and specificity. Flaws in the non-inferiority trial design typically bias towards the null, which may result in falsely declaring non-inferiority. Methods We conducted a statistical simulation study to assess the impact of imperfect markers of tuberculosis infection on the interpretation of tuberculosis vaccine trials testing a non-inferiority hypothesis of an infection primary outcome in a two-arm randomized comparison. Data were generated in three 2-year cumulative risk of tuberculosis infection scenarios (2%, 5%, and 8%). The specificity of tests of tuberculosis infection was assumed to range from 100% to 85%, while the sensitivity was assumed to range from 100% to 64%. Log-binomial regression was used to estimate the relative risk of tuberculosis infection. Results With 100% sensitivity and specificity, type-I and type-II error were both approximately equal to the expected values (2.5% and 80%, respectively) in all three cumulative tuberculosis risk scenarios. With modest deviations from perfect sensitivity and specificity (95% for both), the risk of falsely declaring non-inferiority was 96.8%, 53.2%, and 27.8% in the 2%, 5%, and 8% cumulative tuberculosis risk infection scenarios, respectively. Discussion Tuberculosis vaccine non-inferiority trials using an infection primary outcome must be designed and interpreted accounting for the specificity of the tools used to measure infection, otherwise they risk declaring non-inferiority by default. Key messages We conducted a statistical simulation study to assess the impact of imperfect sensitivity and specificity, in the primary outcome definition of tuberculosis infection, in vaccine trials testing a non-inferiority hypothesis. With only modest departures from perfect specificity in tuberculosis infection markers, the risk of falsely declaring non-inferiority is substantial. Vaccine trials testing a non-inferiority hypothesis with an infection primary outcome must account for the imprecision in the tools used to define the outcome, otherwise vaccines may be falsely declared non-inferior.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.632 | 0.884 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.006 | 0.010 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.001 | 0.014 |
| Scholarly communication | 0.010 | 0.008 |
| Open science | 0.006 | 0.004 |
| Research integrity | 0.010 | 0.012 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".