By Jacob J. Capin, Lynn Snyder-Mackler, May Arna Risberg & Hege Grindem

Imagine you are a sports medicine clinician responsible for an athlete with anterior cruciate ligament (ACL) reconstruction. Her surgery goes well, she completes her rehabilitation meticulously, passes your clinic’s return to sport (RTS) criteria, and starts to progress to full RTS. Now imagine another of your patients: his knee keeps flaring up after ACL surgery and his rehabilitation is arduous and prolonged. He’s not in a position to pass the RTS exercises (such as the hop test you can see me doing here), so you advise him he’s not ready to return. He returns to a lower-demand sport than originally planned–golf. Two years later, the woman has sustained a contralateral ACL injury during football. The man has completed two 5k runs, avoided pivoting sports and has no re-injury.

No clinician would interpret this scenario to mean that the woman could have avoided her contralateral ACL injury had she returned to football without passing RTS criteria. However, if the clinical picture is missing from a research paper’s conclusion, this interpretation is plausible. After all, she passed the criteria and he failed.

This scenario illustrates one of several problems in the literature on RTS testing–sports exposure is not considered in the analysis. Methods problems compound if authors of systematic reviews collate data in a meta-analysis without considering the limitations of the original studies. To safeguard against misleading conclusions, systematic review authors should perform a risk of bias assessment, and use this assessment as a guide to include or exclude research studies in the analyses and conclusions. [1–3]

We demonstrate the importance of considering risk of bias in our upcoming British Journal of Sports Medicine (BJSM) editorial, “Keep calm and carry on testing: a substantive reanalysis and critique of ‘what is the evidence for and validity of return-to-sport testing after anterior cruciate ligament reconstruction surgery? A systematic review and meta-analysis’” [4] That recent systematic review and meta-analysis, written by Webster and Hewett [5], reported some results that would likely discourage the use of RTS test batteries after ACL reconstruction. Webster and Hewett’s surprising suggestion was that passing RTS criteria might (paradoxically) be associated with a higher risk of contralateral ACL injury. Given that the article [5] is a systematic review and meta-analysis, along with the recent attention in The New York Times and on social media, [6,7] we suspect many clinicians have already viewed their findings.

Webster and Hewett did not perform a risk of bias assessment. Two articles with high risk of bias, including one from our research group [8] that was never designed to answer Webster and Hewett’s research question, were included in the meta-analysis.[5] Including these studies dramatically influenced the conclusions.

Our reanalysis of this systematic review and meta-analysis excluded two articles with high risk of bias and provides four key points:

(1) Passing RTS test batteries was associated with decreased risk of further knee injury by 72% (95% confidence interval [CI]: 6-96% lower risk, p=0.09).

(2) Passing RTS test batteries was associated with decreased odds of any ACL injury by 75% (95% confidence interval [CI]: 46-88% lower odds, p<0.01).

(3) Passing RTS test batteries was associated with a decreased odds of ACL graft rupture by 78% (95% confidence interval [CI]: 52-90% lower odds, p<0.01).

(4) Regarding contralateral ACL injuries, only 4 occurred among those who passed or failed RTS criteria, providing far too small a sample to make any meaningful conclusions regarding contralateral ACL injury risk.

We worry some clinicians may stop or never begin implementing RTS testing based on the conclusions of Webster and Hewett.[5] We hope this reanalysis and commentary will (i) reinforce the importance of implementing RTS test batteries after ACLR, (ii) restore vigor in developing and testing optimal RTS test batteries, and (iii) spur renewed dedication to increase the methodological rigor of systematic reviews. In the meantime, we urge clinicians to remain calm and continue testing—based on the best current evidence—to facilitate safe return to play and reduce knee and ACL re-injury risk.

 

***

Authors and their affiliations:

Jacob J. Capin^1,2, Lynn Snyder-Mackler^1,2,3,4, May Arna Risberg^5,6, Hege Grindem^5,7,8

¹Biomechanics and Movement Science, University of Delaware, Newark, Delaware, USA

²Physical Therapy, University of Delaware, Newark, Delaware, USA
³Biomedical Engineering, University of Delaware, Newark, Delaware, USA

⁴Delaware Rehabilitation Institute, University of Delaware, Newark, Delaware, USA

⁵Sports Medicine, Norwegian School of Sports Sciences, Oslo, Norway

⁶Orthopaedic Surgery, Oslo University Hospital, Oslo, Norway

⁷Oslo Sport Trauma Research Center, Department of Sports Medicine, Norwegian School of Sport Sciences, Oslo, Norway

⁸Stockholm Sports Trauma Research Center, Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden

Corresponding author contact: 

Lynn Snyder-Mackler, email: smack@udel.edu, phone: (302) 831-3613

Declarations:

• Competing interests: JJC has received grants from the Foundation for Physical Therapy Research—Promotion of Doctoral Studies (PODS) Level I and II Scholarships and the National Institutes of Child Health & Human Development (F30 HD096830). LSM has received grants from the National Institutes of Health (R37 HD037985, R01 AR048212, T32 HD007490, R44 HD068054, U54 GM104941 andP30 GM103333). MAR and HG have received grant funding from the National Institutes of Child Health & Human Development (R37 HD037985). HG is a BJSM senior associate editor, works as a sports physiotherapist, occasionally receives minor speaking fees for presentations, and holds grants from the International Olympic Committee, the Norwegian fund for post-graduate training in Physiotherapy, and the Swedish Research Council for Sport Science.
• Authors’ contributions: JJC and HG contributed to conception and drafting and editing the manuscript; MAR and LSM contributed to conception and critical review; all authors read and approved the final version of the manuscript.

References

1. Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 5. The Cochrane Collaboration 2011. www.handbook.cochrane.org
2. Springer Manuscript Guidelines Journals: Instructions for Authors. https://www.springer.com/authors/manuscript+guidelines?SGWID=0-40162-6-1397843-0 (accessed 28 Mar 2019).
3. Moher D, Liberati A, Tetzlaff J, et al.Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med2009;6:e1000097. doi:10.1371/journal.pmed.1000097
4. Capin JJ, Snyder-Mackler L, Risberg MA, et al

   “Keep calm and carry on testing: a substantive reanalysis and critique of ‘what is the evidence for and validity of return-to-sport testing after anterior cruciate ligament reconstruction surgery? A systematic review and meta-analysis’ ” https://bjsm.bmj.com/content/early/2019/07/08/bjsports-2019-100906

5. Webster KE, Hewett TE. What is the Evidence for and Validity of Return-to-Sport Testing after Anterior Cruciate Ligament Reconstruction Surgery? A Systematic Review and Meta-Analysis. Sport Med2019;:Epub ahead of print. doi:10.1007/s40279-019-01093-x
6. Reynolds G. After a Knee Injury, Be Wary When Returning to Sports. New York Times. 2019.https://www.nytimes.com/2019/04/24/well/move/after-a-knee-injury-be-wary-when-returning-to-sports.html (accessed 2 May 2019).
7. Reynolds G. The Knee Is Good to Go. Oh, Really? New York Times. 2019; April 30:D6.
8. Wellsandt E, Failla MJ, Snyder-mackler L. Limb Symmetry Indexes Can Overestimate Knee Function After Anterior Cruciate Ligament Injury. J Orthop Sport Phys Ther2017;47:334–8. doi:10.2519/jospt.2017.7285