Ketone ester supplementation in endurance athletes: a miracle drink or ‘spin’?

As fellow scientists and endurance athletes, we read with interest the recent paper by Poffé and colleagues on ketone ester supplementation in endurance training (Poffé et al. 2019). The authors conclude that they demonstrated “that an oral ketone ester is a potent nutritional strategy that prevents the development of physiological overtraining symptoms and non-functional overreaching”. In the days following publication, the paper received a lot of media attention. Enthusiastic news articles reported that “endurance athletes have a big advantage when using ketones” (Dutch Broadcast Foundation, 03-05-2019), “ketones give athletes wings” (De Standaard, 03-05-2019), and “there is a new ‘super fuel’ for top athletes” (HBVL, 03-05-2019), many of them quoting that it gives athletes a “15%” advantage. We believe that these interpretations may be overly optimistic. Hypothesizing that ketone supplementation can blunt endurance training-induced overreaching, Poffé and colleagues randomized healthy, physically active male subjects to regularly receive a ketone ester or a control drink during a heavy three-week daily exercise regimen. A total of 18 participants (9 in both arms) were included, and these underwent a large number of tests and measurements: exercise tests (including a 30 min simulated time-trial, a 90 s isokinetic sprint and a 120 min endurance exercise performance test), body composition tests, questionnaires, nutritional intake monitoring, muscle biopsy, urine samples, blood samples and so on. Most of these tests were performed at multiple time points; exercise tests, for example, were performed six times per participant (pre-test, at days 7 and 14, post-test, and at days 3 and 7 in the recovery phase). This large number of measurements translates to an equally large number of statistical comparisons that can be performed between both study arms: we count no less than 85 P values in the Results section of the paper, 59 of which are statistically significant (P < 0.05); the figures and tables seem to contain many more. Because of this, the risk of spurious findings is high: the more statistical tests performed, the higher the chances of falsely rejecting a null-hypothesis (i.e. concluding that there is a difference between the groups, while in reality there is none). Especially with a relatively small number of participants, this is worrying: outliers (e.g. one athlete with very poor results) may have considerable impact on the results, depending on the statistics used. This could, to some extent, have been remedied by adjusting the P value threshold for multiple testing, but the authors seem not to have done so. Layered on top of this concern regarding multiple comparisons is the fact that the authors did not clearly define a primary outcome. Which one did they focus on in the original study protocol? In the Abstract they report that “sustainable training load in week 3, as well as power output in the final 30 min of a 2 h standardized endurance session were 15% higher in ketone ester drink than in control drink (both P < 0.05)”. We acknowledge that these are likely to be two relevant outcomes. What they do not mention in the Abstract is that other outcomes that may be equally relevant are not statistically significant: for example, power outputs in the 30 min time-trial at any time point, and 90 s isokinetic sprint at any time point. In addition, no confidence intervals are reported around this 15% difference, which would have shown the uncertainty of the finding. To tackle such deficiencies in the informativeness of abstracts, reporting guidelines such as CONSORT (Consolidated Standards of Reporting Trials) for Abstracts recommend to report the estimated effect size for the primary outcome along with its precision (Hopewell et al. 2008). We believe that researchers have a responsibility to present their results in a fair and balanced way. Only focusing on positive outcomes, especially in the case of a large number of statistical tests and in the absence of a pre-defined primary outcome, is a form of ‘spin’: the Abstract gives the impression that the advantage of ketone ester drink is impressive (or “15%”, as copied from the Abstract in multiple media reports), while in fact it is not or is at least doubtful. This is a common phenomenon in biomedical literature (Boutron et al. 2010; Yavchitz et al. 2012; Boutron et al. 2014; McGrath et al. 2017). Unfortunately, the impact of this ‘spin’ has been magnified in the form of media reports which may misinform the public about the findings of this study. How can such ‘spin’ be prevented? In interventional research, it is common practice that researchers define a clinically relevant primary outcome, based on which the study hypothesis can be tested, and perform a sample size calculation to ensure that a sufficient number of participants is included. No such practices were reported by Poffé and colleagues. Ideally, the primary outcome (and other important study protocol information) is then registered before study initiation in a publicly accessible trial registry, such as ClinicalTrials.gov (Zarin & Keselman, 2007). The Declaration of Helsinki states that “every research study involving human subjects must be registered in a publicly accessible database before recruitment of the first subject” (WorldMedicalAssociation, 2013). Poffé and colleagues state that their study “conforms to the Declaration of Helsinki”, but do not report whether their study has been registered in a trial registry. A major advantage of prospective registration is that it provides the opportunity to assess which primary and secondary outcomes were pre-defined. This way, selective reporting (e.g. reporting an outcome as primary while it was secondary in the original protocol) and cherry-picking (e.g. leaving out negative results and only reporting positive ones) can be detected and prevented in the peer review process. Currently, many biomedical journals only consider submitted papers of clinical trials for publication if these were registered before study initiation in a trial registry (De Angelis et al. 2005; Hooft et al. 2014). A similar policy at The Journal of Physiology may further improve the quality of published research. Ketone ester supplementation in endurance athletes may be an effective strategy. Whether this study provides convincing evidence to support this hypothesis remains open. None of the authors has any conflicts of interest. All authors have approved the final version of the manuscript and agree to be accountable for all aspects of the work. All persons designated as authors qualify for authorship, and all those who qualify for authorship are listed. No funding was received.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.