The defence of legitimate exercise physiology research from real and perceived bias: a rebuttal

Dear Editor, A recent publication entitled ‘Is this the new smoking? An expert panel review of the York University OHV health benefits study’ published in Health Promotion International (Bissix et al., 2012) claimed to systematically appraise the scientific methodology upon which a series of health-related investigations into off-highway vehicle (OHV) participation were based. The authors concluded that the science regarding the health impacts of OHV riding is ‘indefensible’ based on conceptual, methodological and interpretive limitations. Beginning with the title and prevalent throughout the article, there is a pervasive suggestion that our study had a research bias. To suggest that we have such a deficiency of integrity is the greatest possible insult to a serious scholar, and it is audacious of Bissix et al. to draw comparisons between the OHV health and chronic disease risk investigation and the travesties in some past smoking research. This is an accusation to which we (the authors of this original research) take great offense in no uncertain terms. Furthermore, we suggest that the article by Bissix et al. claiming to scientifically re-assess our work is an embarrassment to serious researchers, full of poor or irrelevant arguments, misrepresentations of fact, references taken out of context and ludicrous attribution of media reports to our research team. To be clear, we fully support the scientific process of collecting data with a transparent sharing of the methodology and results, allowing work to be re-tested, confirmed or refuted. It is through this scientific method that knowledge is gained and progress is made. However, Bissix et al. have little to contribute to this process owing to their lack of expertise in this area, which is painfully apparent from the uninformed arguments in their article and their absence of published research in our field. As such, this ‘scientific paper’ they have submitted is nothing more than a biased commentary. The egregious errors and omissions in their ‘re-interpretation of our data’ can only be attributed to either (i) a purposeful attempt to mislead or (2) outright ignorance of basic principles of exercise physiology. Extending the authors the benefit of doubt concerning their professional integrity, we will assume that the errors in their article stem primarily from the latter. In this paper, we critically examine and respond to the objections that Bissix et al. have raised in regard to our work. First, to infer that our research was in any way biased by our funding is patently incorrect and an unfounded defamatory accusation. We have formally acknowledged all funding arrangements in every presentation of our data, and made it very clear that our funding partners had no involvement at any stage of the research or manuscript preparation. Our funder's arms-length relationship to our research was no different from that which exists between any funding agency (be it cancer, heart disease etc.) and other front-line academic investigators performing peer-reviewed research which has met rigorous ethics approval standards. In fact, there was even less potential for funding agency bias in our study than may commonly exist, as this was a one-time project, thus eliminating suggestion of any prejudice towards favourable results bringing further funding. Despite the suggestion of Bissix et al., our research team fundamentally neither supports nor opposes the activity of OHV riding; we simply collected data to fulfil our purposes and interpreted these data in the framework of existing health-related exercise physiology and chronic disease knowledge. We have no vested interest in promoting or opposing OHV participation and our on-going official stance is represented only in our study conclusions on the topic investigated. In contrast, Bissix et al. have a clear and present bias as founders and chair of an organization (Kieran Pathways) which is concerned with land use and environmental issues in their province and has publicly declared its opposition to OHV use. Prior to the publication of our research, as a main argument against any benefits of OHV use, Bissix et al. have frequently cited the lack of peer-reviewed scientific research supporting any potential social or physiological health-related physical activity benefits from OHV riding (Bissix and Medicraft, 2008; Pitter, 2009). It is true that prior to the publication of our research there was no scientific evidence to support such benefits. It would appear that our research is of interest to Bissix et al., because it contradicts an argument they have long relied upon for leverage in conflicts unrelated to us or the science we perform. We were interested to learn that, unbeknownst to us, this group had been publicly criticising our funding agency and the undertaking of our research long before our results were even published (Bissix and Medicraft, 2008; Pitter, 2009). For the record, we are in no way opposed to any organization that promotes active transportation on trails, in fact our goals for promoting physical activity are much aligned in this regard. However, trying to bluntly discredit our research because it does not fit their chosen mandate for accomplishing these goals through their (politically rooted) organization is unprofessional and wrong. In their ‘review’ of our research, Bissix et al. claim to have enlisted a team of ‘exercise science and medicine experts’ to examine the conceptualization of our research problem, methodology, interpretations and conclusions. Given the contentious nature of the topic, prior to study onset our methodology was blind reviewed and enhanced by international experts in exercise science. Furthermore, each investigation was reviewed by both a dissertation committee of five experts and no less than two anonymous external international peer reviewers following study completion and prior to publication in well-respected journals. As such, we find it peculiar that great discrepancies in opinion could exist between the international exercise science experts who reviewed our work and Bissix’ peers. Assuming the group of ‘experts’ assembled by Bissix et al. was made up of legitimate exercise scientists, we question how thoroughly these ‘experts’ were consulted, and what role they played in the preparation of their review when the published criticisms of our work display a clear ignorance of primary exercise physiology and widely used physical activity guideline frameworks. As well, we would very much like to see the credentials of their anonymous ‘exercise science experts’ especially considering that this was given prime billing in the title of their review. Not providing this information in their article is not only highly suspect, but also a direct contravention of the highly respected ‘Appraisal of Guidelines for Research and Evaluation’ process (AGREE) that was initially developed for clinical practise guidelines and is now recommended for all best practise applications (Jamnik et al., 2011). Bissix et al. claim that a ‘fundamental error’ to our conclusion that OHV riding meets North American Guidelines in terms of exercise intensity is the fact that ‘the guidelines clearly indicate a minimum bout of 10 min is necessary’. Let us clarify to these authors that intensity refers to ‘how hard’, whereas duration refers to ‘how long’; this is taught in our first year introduction to Kinesiology class. They further criticise our research (both in Table 3 and in the text) given our extrapolation methods to typically encountered OHV riding exposures. In their ‘re-interpretation’ of our results, the authors claim that we only demonstrated minimal amounts of aerobic work (3.36 min for ATV and 9.12 min for motorcycles) and that this intensity (actually duration) is likely unevenly spread throughout a ride. First, as the authors should have noticed from the nationwide survey reported on in one of our papers that they referenced, off-road riders were found to ride for an average duration of 2–3 h per session (Burr et al., 2010a). Any exercise physiologist familiar with metabolic (VO2) testing knows that it is impractical to test continuously for this duration of time as it requires the participant's nose to be plugged and mouth to be attached to a collection device throughout this duration so that all expired air can be analysed. In fact, the average laboratory treadmill exercise test takes less than 12 min, thus our use of a 24-min ride over a scaled representative OHV riding course actually offers a relatively long window of observation and hence an accurate assessment of the aerobic demands during a representative portion of an OHV ride. Thus, extrapolating this representative portion of a test ride to a ‘typical’ ride would be recognized as an entirely appropriate and logical approach by any knowledgeable professional in our field. To take these data out of context and suggest that the small sample window represents the total exposure (and further imply that we arbitrarily extrapolated to a 2–3 h exposure to inflate the associated demands) demonstrates that Bissix et al. have a poor understanding of this type of research, or intentionally tried to misrepresent our data. We also question their report of a lack of clarity around intensity/duration dispersion measures as these data are clearly presented and discussed in the article text, within a table, and graphically (Burr et al., 2010c). Similarly misleading are the statements regarding an inability to justify findings as being related to OHV group membership. We agree that cause and effect cannot be conclusively determined with a cross-sectional study design, and have ourselves pointed this out in our reports (Burr et al., 2010a,b,c); however, this was precisely why this series of articles included a fourth study with a longitudinal design (Burr et al., 2011), which enabled the experimental testing of the results from the preceding investigations. It is outright deceptive of Bissix et al. to purposely omit any reference to this article in their review given that we can confirm their prior knowledge of the article's existence. In fact, we have already published a response to a similarly biased editorial by Bissix et al. specific to the article that they chose to omit (Burr et al., 2012). Referencing the online publication dates available from the PubMed index reveals that even our response to their editorial was published prior to the current article in which they ignore its existence. Bissix et al. also criticise the comparison of OHV users to normative population data, suggesting that a more appropriate group would be ‘healthy and active adults’, rather than a predominantly sedentary cohort as would be reflected in the normative Canadian population data that we utilized. This is completely nonsensical, as the purpose was to determine whether OHV users were different from the norm (not from a subset of healthy controls), and to determine whether participation in OHV riding could be associated with health risk reduction or changes in sedentary behaviour. To make matters worse, the authors claim that the population we used for comparison (circa 2000) differed from current Canadian norms, ‘as studies in the intervening years … have suggested higher levels of activity among Canadians’. Given that increasing levels of physical inactivity and hypokinetic disease represent some of the greatest challenges faced in the developed world today, we question how the authors arrived at this (almost comical) conclusion. Their suggestion that the opposite might be true is indefensible and completely incorrect. In fact, the only reference they submitted to support their claim (Colley et al., 2011) considers children (who we specifically did not address), not adults, and those authors reported quite the opposite. The appropriate reference for adults (Shields et al., 2010), which we did not refer to in our articles because that report was released after the publication of our results, actually shows quite the opposite to the claims of Bissix et al. and would reveal our findings of differences from the norm (who were on average more fit in circa 2000) to be conservative if anything. Throughout their article Bissix and colleagues repeatedly reference statements and interpretations from external organizations that picked up and broadcasted the findings of our research. We are incredulous that they use these reports to criticise our science as if these media statements were made by the research team or our institution; whereas in reality many of these statements were made unbeknownst to us and without our consult or consent. For example, they reprehended us for being ‘uncritically quoted in a number of professional newsletters, media reports and OHV user publications’. We have done our best to ensure that our results are being properly interpreted, particularly by our related government/industry partners; however, if our studies have been inappropriately represented in nonscientific circles and this issue is of concern to Bissix and colleagues, we suggest they raise the issue with the group who has published such inaccuracies—just as we are addressing this issue with their group. They are correct in reporting that our contact information appeared on some press-releases, as this factual information was taken directly from the published manuscripts. We have no issue with our contact information being published, so that readers who were unable to comprehend the research are able to contact us directly for clarity or further explanation. It is a shame that Bissix et al. did not do so, as it appears we could have offered a reasonable insight that they were obviously incapable of determining on their own. Although our group has no interest in engaging in this type of non-scientific banter, we were provoked into responding to the authors' unfounded criticisms. This also prompted us to delve further into this topic and we were concerned to find a large volume of unsubstantiated and ‘uncritically quoted’ material on this topic from the authors themselves. In particular, we note that the authors were quite quick to congratulate themselves for ‘taking apart’ and ‘debunking’ the findings of our research, while claiming to present ‘evidence [that] points quite the other way’ (Bissix, 2012). We would like to see this evidence. In fact, we recommend that this group undertake actual science to generate empirical data to substantiate their claims. A final issue, repeatedly raised by Bissix et al., concerns the fact that our investigations did not address issues around environmental impact or potential for traumatic injury associated with OHV riding. As we have acknowledged previously (Burr et al., 2010a, 2012), we recognize that these are important and highly contentious issues deserving of consideration in activity promotion. However, they were intentionally not addressed in our studies because they were not germane to our purpose, which was to systematically evaluate the effects of OHV riding on fitness, health and chronic disease risk. Furthermore, these issues are not within our scope of expertise as exercise physiologists and chronic disease specialists. However, it is our understanding that walking-related injuries do occur and that pedestrian traffic is known to cause environmental damage; nonetheless, we are unaware of any previous cases in which an exercise scientist has been criticised for not addressing these issues in a study of the benefits of walking. It would seem similarly foolish to suggest that the promising health-related results of a walking study be ignored by claiming ‘sure, but running is still better’, yet this argument seems to be relied upon by Bissix et al. to criticise our findings regarding potential for improvements in health from OHV riding. Although our research has focused on the higher range of fitness and health outcomes considering ACSM guidelines, we would be remiss not to mention that there is considerable evidence suggesting that shorter bouts of high intensity activity can be equally effective for stimulating certain health changes (Gibala and McGee, 2008) and alternatively, that levels of physical activity below the threshold required to bring about changes in aerobic fitness can cause meaningful positive changes in health status and chronic disease risk (Blair et al., 2004). In fact, there is a whole facet of research now dedicated to the health benefits of incidental activity of daily life, or non-exercise activity thermogenesis (NEAT) (Levine et al., 1999). Even more to this point, there is accumulating evidence that an avoidance of sedentary time, not just the participation in purposeful physical activity, is related to health (Owen et al., 2010). We do not suggest that off-road vehicle riding is the only, or even an optimal mode of physical activity for avoiding sedentary time or accumulating physical activity, but we have presented evidence that it is a viable option, particularly for persons who enjoy the activity and would not likely accumulate physical activity in other ways. Our conclusion that greater levels of OHV participation would be expected to result in greater effects on fitness and health, which the authors criticised, is true of virtually all physical activity. It is acknowledged by Bissix et al. (Bissix et al., 2012) that OHV riding is one of the fastest growing recreational activities in North America. Thus, as participation numbers continue to grow, OHV riding is clearly an activity we should seek to fully understand, considering both the positive and negative impacts on individuals and the community. The OHV riding data that we have provided is useful for comparing the benefits of this activity to the avoidance of all activity (sedentarism), to other similar physical activities, and the associated health-cost savings versus the health-cost expenditures of participation. We suggest that authors criticising our work should seek to increase their understanding of this area of research while maintaining respect for the scientific process and integrity in research.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.