Letter to Editor

We thank Dr McKenzie for his kind words and compliments on our paper. We understand and agree with his remarks, and wish to respond. He raises several issues of methodology in veterinary research that need to be considered if the scientific community wants to improve the quality of research and to promote evidence-based veterinary medicine (EBVM). A major obstacle to EBVM is the limited availability of internationally validated instruments for measuring outcomes. Their development would provide a standardized means for clinical assessment and comparison of the efficacy of veterinary treatments.1 In the absence of validated scoring systems for severity of osteoarthritis (OA) investigators use their own subjective (owner's evaluation) or semi-objective (clinical assessment) nonvalidated questionnaires. Doing so, they may not only miss relevant parameters that would reinforce their hypothesis but also include irrelevant variables that weaken it. This is one issue in the studies identified by Dr McKenzie where questionnaires included too many parameters to the point that the significance of those which improved with treatment was weakened by the large number which were not affected.2-5 Furthermore, in human medicine, psychometric instruments (questionnaires, scales) are commonly used to measure dimensions like satisfaction and quality of life which are now considered as important as purely clinical observations. It is commonly accepted that standard methods must be used for the stepwise development and testing of questionnaires designed for subjective assesment.6 A second obstacle relating to this dearth of standardized outcome measures is the lack of consensus about what is interpreted to be a useful clinical effect. Dr McKenzie considers as low a clinical improvement of 18% and a reduction of the carprofen dose of 0.32 mg/kg/day,2, 3 an opinion that we might agree with. However, would, for example, an improvement of 25% and of 0.60 mg/kg/day have been clinically significant? As we stated in our review, calculation and reporting of effect size in veterinary trials would be a convenient construct for comparing the magnitude of outcomes within and among trials.7 It is also well recognized, in human medicine, that it is currently difficult to demonstrate a significant difference between a new treatment and the current accepted treatment.8 There is also a risk that the sample size that would be required to show differences might not be reached because of difficulty in recruiting sufficient animals. Thirdly, the criteria for assessing quality of studies in systematic reviews should be ideally validated, and, here also, there are currently no internationally validated guidelines in veterinary medicine. In our study, we attempted to improve the content validity of our scale by conducting a Delphi process between several veterinarians involved in research design and epidemiology. In our scale, we attributed a score of 2 and 1% if we could answer positively respectively to the following items “results of any analyses are explained” and “the effect size and clinical significance are reported”. We admit that more weight could have been given to those items. Furthermore, more weight could be given to the statistical analysis. Another issue is also that the “yes” or “no” answer to those items restricts nuance: if of 5 measures compared between groups at 2 points in time and in terms of change over time, the test group improved more than the control group in terms of 2 measures, should we respond “yes” or “no” to the question referring to the existence of a significant effect? Other criteria in our study were also arbitrary such as the minimum number of 4 studies about 1 nutraceutical to define that the quantity of studies is adequate. If we had decided that the studies had to be conducted by different teams of researchers, our study would not have concluded that there was strong evidence of the efficacy of fatty acids. Systematic review is an imperfect process and we admit that our work did not escape this rule. However, our objective was to suggest a methodology to conduct systematic reviews, more than prove its validity. One quality of our work was the transparency of the scale and the generation of data that encouraged this interesting debate. Furthermore, we think systematic reviews should be a dynamic and interactive process. For example, after publication of this article, Moreau et al informed us about another interesting paper that fulfilled inclusion criteria but was not detected via our research equations in databases.9 The EBVM associations throughout the world should work toward an interactive process of systematic reviewing. It is the comparison of studies in a similar way that is important, rather than the classification of the studies as strong or weak. If the veterinary scientific community manages to produce guidelines for scoring studies in systematic reviews, it would not be useful any more to ordinate studies (as in our work, where we established arbitrarily cut off points between studies of high, intermediate, and low qualities) as every reader, in the end, would know what means a study with a score of 60% for example; like we know what means an outside temperature of 32°C. Finally, we want to thank the editors of JVIM for promoting evidence-based papers and discussions about EBVM.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.