Blog entry written on: Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis, (bmjebm-2021-111846)

Authors: Gerald Gartlehner, Robert Emprechtinger, Marlene Hackl, Franziska L Jutz, Jacob E Gartlehner, Julian N Nonninger, Irma Klerings and Andreea Iulia Dobrescu

Undeterred by allegations that homeopathy is a pseudoscience that lacks-approval by regulatory agencies for safety and effectiveness, an ever-increasing number of people worldwide rely on homeopathic remedies as complimentary or alternative treatments to evidence-based medical practices. The market for homeopathic remedies is estimated to reach a valuation of about US$18.6 billion globally by the year 2027. Adherents to homeopathy can be found in the scientific mainstream and fringe communities. Besides anecdotal testimonials, proponents of homeopathy often refer to existing clinical trials which scientifically prove the effectiveness of homeopathic remedies.

Ah yes, the clinical trials, the mainstay of evidence-based medicine. The best study design to minimize bias and confounding. It was precisely those clinical trials our research team was interested in. (bmjebm-2021-111846.R1 – Assessing the Magnitude of Reporting Bias in Trials of Homeopathy: A Cross-sectional Study and Meta-analysis).

Specifically, with a cross-sectional study and meta-analyses, our team investigated the magnitude of reporting bias, the lack of prospective registration, and post-hoc changes of primary outcomes in trials assessing homeopathic treatments. To do so, we looked at three questions. First, how many homeopathic trials which were registered, were subsequently published? Second, of those trials registered, how many changed their primary outcome from the time of registration until the time of publication? And third, what proportion of published homeopathy trials, were registered beforehand?

But why these questions? In general, all evidence-based research should include pre-trial registration and post-trial publication. Unpublished trials can be a sign of undesired results or trial complications. For homeopathic products, however, because regulatory agencies do not require any proof of effectiveness, homeopathy trials, particularly those which remain unpublished, often escape any kind of scrutiny from the greater research community. Did the proportion of homeopathy trials which eventually reached the publication stage reflect the proportion of all trials conducted or were the published trials only representative of a small proportion, namely those with mostly positive findings?

Since 2002, almost 38% of registered homeopathy trials have remained unpublished, and 53% of published randomized controlled trials have not been registered. Retrospective registration was more common than prospective registration. Furthermore, 25% of primary outcomes were altered or changed compared with those in the registry. Although we could detect a statistically significant trend toward an increase of registrations of homeopathy trials (p=0.001), almost 30% of trials published during the past 5 years had not been registered. Meta-analyses of registered and unregistered trials showed that unregistered homeopathy trials tended to have larger treatment effects than registered trials. This, in turn, suggests that unknown number of trials may exist that were neither registered nor published because they did not show the hoped-for results.

Which means?

Results from our study suggest a concerning lack of scientific and ethical standards in the field of homeopathy. More than a third of registered homeopathy trials remain unpublished, more than half of published RCTs had not been registered, and 25% of primary outcomes were altered or changed. This could mean that the available body of evidence of homoeopathic literature may substantially overestimate the true treatment effect of homeopathic remedies and that homeopathy researchers are moonlighting as cherry-pickers. Trials with hoped-for results are published to reinforce claims of homeopathy’s effectiveness while those which are less favorable are conveniently disappeared into a file drawer. Sound conclusions about the efficacy and the risks of homeopathic remedies thus remain impossible. Subsequently, proof of homeopathy’s effectiveness continues to be as diluted as a pinch of salt in a vast ocean of poor-quality research.

Blog Author

Jacob E. Gartlehner

Medical University Vienna
COIs: None to declare

DISCLAIMER

The views and opinions expressed on this site are solely those of the original authors. They do not necessarily represent the views of the BMJ and should not be used to replace medical advice. All information on this blog is for general information, is not peer-reviewed, requires checking with original sources and should not be used to make any decisions about healthcare. No responsibility for its accuracy and correctness is assumed by us, and we disclaim all liability and responsibility arising from any reliance placed on such commentary or content by any user or visitor to the Website, or by anyone who may be informed of any of its content. Any reliance you place on the material posted on this site is therefore strictly at your own risk.