By Matheus Oliveira de Almeida @Matheusoa, and Bruno T Saragiotto @bsaragiotto

Korakakis and colleagues’ recent systematic review of extracorporeal shockwave therapy (ESWT) concluded that the intervention may be effective in the treatment of some lower limb conditions¹. However, some aspects of the review raise doubts about the positive conclusion.

Although the authors reported that they conducted the review following recommendations in the Cochrane Handbook², there are some important departures, which have likely introduced biases. Firstly, the authors have included a non-randomised study (case-control study) in the meta-analysis. Effect estimates from non-randomised studies tend to be larger when compared to randomised controlled trials (RCT)^{3 4}. This is demonstrated in Figure 4 (Appendix 3; Comparison 1.2.3), with the effect size of the non-randomised study (Furia 2009; odds ratio 3.08, CI 1.59 to 5.95) being three times greater than one of the RCTs (Rompe 2009; odds ratio 9.90, CI 3.19 to 30.75) in the comparison between ESWT and control for self-perceived recovery at intermediate-term in greater trochanteric pain syndrome.

Another methodological concern is that the authors excluded a trial at low risk of bias due to the trial’s small sample size; introducing reporting bias. In a meta-analysis, studies are assigned weights based on mathematical criteria, such as study’s precision, which is primarily driven by sample size. Hence, studies with relatively poor precision are assigned less weight, while studies that yield more precise estimates are assigned more weight⁵. Further, the chance of detecting a real effect increases when several studies are combined. So, there is no reason to exclude a RCT from the analysis based on a small sample size.

Another concern is the authors’ application of the GRADE approach for the quality of evidence. The authors have chosen to increase the GRADE score by one level if the treatment effect was large (e.g. ESWT versus conservative treatment in proximal hamstring tendinopathy, Table 10). We believe that this decision came from the authors’ misunderstanding of the Cochrane Handbook chapter 12.2.1², which advocates that the quality of evidence for observational studies can be increased if they yield large effects and there is no obvious bias explaining those effects. This is a specific recommendation for observational studies and cannot be generalised.

These highlighted problems mean that these recommendations on the effectiveness of ESWT should be interpreted with some caution. ESWT is an expensive and sometimes painful treatment and decisions on its use need to be based upon unbiased summaries of the available evidence.

 Matheus Oliveira de Almeida (@Matheusoa) has PhD in Evidence Based in Medicine at Brazilian Cochrane Collaboration Centre. Currently I am a Post-Doctoral fellow, working on methodological projects in low back pain field”

References

1. Korakakis V, Whiteley R, Tzavara A, et al. The effectiveness of extracorporeal shockwave therapy in common lower limb conditions: a systematic review including quantification of patient-rated pain reduction. Br J Sports Med 2017 10.1136/bjsports-2016-097347.
2. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). 2011.
3. Savovic J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 2012;157(6):429-38.
4. Reeves BC. Principles of research: limitations of non-randomized studies. Surgery 2008;26(3):120-24.
5. Borenstein M, Hedges LV, Higgins JPT, et al. How a Meta-Analysis Works, in Introduction to Meta-Analysis. In: Sons JW, ed. Chichester, UK. 2009.