You don't need to be signed in to read BMJ Blogs, but you can register here to receive updates about other BMJ products and services via our site.

The Media

Acupuncture for infantile colic – misdirection in the media or over-reaction from a sceptic blogger?

26 Jan, 17 | by Dr Mike Cummings

So there has been a big response to this paper press released by BMJ on behalf of the journal Acupuncture in Medicine. The response has been influenced by the usual characters – retired professors who are professional bloggers and vocal critics of anything in the realm of complementary medicine. They thrive on flexing their EBM muscles for a baying mob of fellow sceptics (see my ‘stereotypical mental image’ here). Their target in this instant is a relatively small trial on acupuncture for infantile colic.[1] Deserving of being press released by virtue of being the largest to date in the field, but by no means because it gave a definitive answer to the question of the efficacy of acupuncture in the condition. We need to wait for an SR where the data from the 4 trials to date can be combined.
On this occasion I had the pleasure of joining a short segment on the Today programme on BBC Radio 4 led by John Humphreys. My protagonist was David Colquhoun, who spent his short air-time complaining that the journal was even allowed to be published in the first place. Why would BBC Radio 4 invite a retired basic scientist and professional sceptic blogger to be interviewed alongside one of the journal editors – a clinician with expertise in acupuncture (WMA)? At no point was it made manifest that only one of the two had ever been in a position to try to help parents with a baby that they think cries excessively.
So what about the research itself? I have already said that the trial was not definitive, but it was not a bad trial. It suffered from under-recruiting, which meant that it was underpowered in terms of the statistical analysis. But it was prospectively registered, had ethical approval and the protocol was published. Primary and secondary outcomes were clearly defined, and the only change from the published protocol was to combine the two acupuncture groups in an attempt to improve the statistical power because of under recruitment. The fact that this decision was made after the trial had begun means that the results would have to be considered speculative. For this reason the editors of Acupuncture in Medicine insisted on alteration of the language in which the conclusions were framed to reflect this level of uncertainty.
David Colquhoun has focussed on multiple statistical testing and p values. These are important considerations, and we could have insisted on more clarity in the paper. P values are a guide and the 0.05 level commonly adopted must be interpreted appropriately in the circumstances. In this paper there are no definitive conclusions, so the p values recorded are there to guide future hypothesis generation and trial design. There were over 50 p values reported in this paper, so by chance alone you must expect some to be below 0.05. If one is to claim statistical significance of an outcome at the 0.05 level, ie a 1:20 likelihood of the event happening by chance alone, you can only perform the test once. If you perform the test twice you must reduce the p value to 0.025 if you want to claim statistical significance of one or other of the tests. So now we must come to the predefined outcomes. They were clearly stated, and the results of these are the only ones relevant to the conclusions of the paper. The primary outcome was the relative reduction in total crying time (TC) at 2 weeks. There were two significance tests at this point for relative TC. For a statistically significant result, the p values would need to be less than or equal to 0.025 – neither was this low, hence my comment on the Radio 4 Today programme that this was technically a negative trial (more correctly ‘not a positive trial’ – it failed to disprove the null hypothesis ie that the samples were drawn from the same population and the acupuncture intervention did not change the population treated). Finally to the secondary outcome – this was the number of infants in each group who continued to fulfil the criteria for colic at the end of each intervention week. There were four tests of significance so we need to divide 0.05 by 4 to maintain the 1:20 chance of a random event ie only draw conclusions regarding statistical significance if any of the tests resulted in a p value at or below 0.0125. Two of the 4 tests were below this figure, so we say that the result is unlikely to have been chance alone in this case. With hindsight it might have been good to include this explanation in the paper itself, but as editors we must constantly balance how much we push authors to adjust their papers, and in this case the editor focussed on reducing the conclusions to being speculative rather than definitive. A significant result in a secondary outcome leads to a speculative conclusion that acupuncture ‘may’ be an effective treatment option… but further research will be needed etc…
Now a final word on the 3000 plus acupuncture trials that David Colquhoun mentions. His point is that there is no consistent evidence for acupuncture after over 3000 RCTs, so it clearly doesn’t work. He first quoted this figure in an editorial after discussing the largest, most statistically reliable meta-analysis to date – the Vickers et al IPDM.[2] He admits that there is a small effect of acupuncture over sham, but follows the standard EBM mantra that it is too small to be clinically meaningful without ever considering the possibility that sham (gentle acupuncture plus context of acupuncture) can have clinically relevant effects when compared with conventional treatments. Perhaps now the best example of this is a network meta-analysis (NMA) using individual patient data (IPD), which clearly demonstrates benefits of sham acupuncture over usual care (a variety of best standard or usual care) in terms of health-related quality of life (HRQoL).[3]

Key to abbreviations
BMJ – British Medical Journal (company)
EBM – evidence-based medicine
HRQoL – health-related quality of life
IDP – individual patient data
IDPM – individual patient data meta-analysis
MCID – minimal clinically important difference
NMA – network meta-analysis
SR – systematic review
VAS – visual analogue scale (usually a 100mm line)

References
1.   Landgren K, Hallström I. Effect of minimal acupuncture for infantile colic: a multicentre, three-armed, single-blind, randomised controlled trial (ACU-COL). Acupunct Med 2017: acupmed-2016-011208. doi:10.1136/acupmed-2016-011208
2.   Vickers AJ, Cronin AM, Maschino AC, et al. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med 2012;172:1444–53. doi:10.1001/archinternmed.2012.3654
3.   Saramago P, Woods B, Weatherly H, et al. Methods for network meta-analysis of continuous outcomes using individual patient data: a case study in acupuncture for chronic pain. BMC Med Res Methodol 2016;16:131. doi:10.1186/s12874-016-0224-1


Declaration of interests

I am the salaried medical director of the British Medical Acupuncture Society (BMAS), a membership organisation and charity established to stimulate and promote the use and scientific understanding of acupuncture as part of the practice of medicine for the public benefit.

I am an associate editor for Acupuncture in Medicine.

I have a very modest private income from lecturing outside the UK, royalties from textbooks and a partnership teaching veterinary surgeons in Western veterinary acupuncture. I have no private income from clinical practice in acupuncture. My income is not directly affected by whether or not I recommend the intervention to patients or colleagues, or by whether or not it is recommended in national guidelines.

I have not chaired any NICE guideline development group with undeclared private income directly associated with the interventions under discussion. I have participated in a NICE GDG as an expert advisor discussing acupuncture.

I have used Western medical acupuncture in clinical practice following a chance observation as a medical officer in the Royal Air Force in 1989. My opinions are formed by data that spans the range of quality and reliability, much of which is in the public domain.

I have a logical mistrust of the motives of anyone who advertises an interest or hobby in being a ‘Skeptic’, as opposed to using appropriate scepticism within their primary profession, or indeed organisations that claim to promote generic ‘science’ as opposed to actually engaging in it.


Addendum
This blog was edited on 26 January to remove some parts of the text to be in line with usual editorial standards.

Trust Me, I’m an acupuncture expert – but I have never actually had it or used it…

4 Sep, 16 | by Dr Mike Cummings

 

MC trust me 2016

On Thursday 1st September the first episode of series five of Trust Me I’m A Doctor aired on BBC2. I was keen to see how acupuncture was treated after spending a day engaged in trying to demonstrate a change in pressure pain threshold in the lead presenter about a month previously. The experiment went relatively well, and Michael’s pressure pain thresholds doubled from before to after the experiment. Sham acupuncture involved the use of ‘non-penetrating’ retractable needles. It was the first time I had used these in earnest and they succeeded in masking the subject – Michael could not tell which of the interventions involved real acupuncture. I did note that the sham needles could inadvertently penetrate the skin, particularly if the retractable shaft had an overly stiff sliding action. Michael found the sham needling created quite a strong sensation, and I had to work quite hard to create as strong a sensation with the real acupuncture – I found this very interesting as someone who has used acupuncture therapeutically for over 20 years, but never actually tried to perform sham acupuncture in a ‘trial’. Experienced acupuncture practitioners who then have to perform sham interventions have often remarked to me that the sham techniques do far more than they expect in terms of sensation, and in the case of the ‘so called’ non-penetrating needles, how often they cause bleeding.

So that brings us to the ‘acupuncture expert’, who, according to ‘Trust Me’, “…has spent much of his career studying the effect of acupuncture.” If you go to PubMed and insert in the search box: Hróbjartsson A [au] AND acup*; you will only get 4 papers, and only one of them will have acupuncture in the title. That is the infamous BMJ systematic review (Madsen et al BMJ 2009)[1] that got away with pooling data from trials of acute and chronic pain, from surgical pain to headache to arthritis. Yes I did say pooling. The clinical heterogeneity in this review was simply breathtaking, but I guess that the relevant BMJ editors were eclipsed by the home address of the authors – the esteemed Nordic Cochrane Centre. But this was not a review performed within the remit of the Cochrane Collaboration. As a Cochrane author I know the rigors of the process very well, and I can assure readers that Madsen et al would never pass muster in such an arena. Yet the authors of the review used this address, perhaps to their advantage in securing a prominent publication.

So we have an expert with one highly controversial review paper on acupuncture to his name. An expert who has never received acupuncture treatment let alone used it. An expert who thinks we do not know how it works, despite over 60 years of laboratory data investigating mechanisms from endogenous opioids to adenosine release.[2] Dare I say, a medical expert who has never touched a patient therapeutically?

So yes, I have to admit I am disappointed with the superficial way the subject was covered, and the lack of acknowledgement of the challenges of performing blinded trials of acupuncture. Challenges that are eminently illustrated by Haake et al (2007)[3] – the biggest ever sham controlled trial of acupuncture in low back pain, with over 1000 patients. In this trial, sham acupuncture performed twice as well as rather intensive German guideline-based conventional care. Can our acupuncture expert really propose that this is simply a placebo response?

References
1.   Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ 2009;338:a3115. doi:10.1136/bmj.a3115
2.   Filshie J, White A, Cummings M. Medical Acupuncture – A Western Scientific Approach. 2nd ed. Elsevier 2016.
3.   Haake M, Müller H-H, Schade-Brittinger C, et al. German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups. Arch Intern Med 2007;167:1892–8. doi:10.1001/archinte.167.17.1892


Declaration of interests

I am the salaried medical director of the British Medical Acupuncture Society (BMAS), a membership organisation and charity established to stimulate and promote the use and scientific understanding of acupuncture as part of the practice of medicine for the public benefit.

I am an associate editor for Acupuncture in Medicine.

I have a very modest private income from lecturing outside the UK, royalties from textbooks and a partnership teaching veterinary surgeons in Western veterinary acupuncture. I have no private income from clinical practice in acupuncture. My income is not directly affected by whether or not I recommend the intervention to patients or colleagues, or by whether or not it is recommended in national guidelines.

I have not chaired any NICE guideline development group with undeclared private income directly associated with the interventions under discussion. I have participated in a NICE GDG as an expert advisor discussing acupuncture.

I have used Western medical acupuncture in clinical practice following a chance observation as a medical officer in the Royal Air Force in 1989. My opinions are formed by data that spans the range of quality and reliability, much of which is in the public domain.

I have a logical mistrust of the motives of anyone who advertises an interest or hobby in being a ‘Skeptic’, as opposed to using appropriate scepticism within their primary profession, or indeed organisations that claim to promote generic ‘science’ as opposed to actually engaging in it.


 

Acupuncture in Medicine Blog homepage

Acupuncture in Medicine

Acupuncture in Medicine is a bi-monthly scientific and clinical journal aimed at Western-trained physicians and other health professionals, and uses the prevailing understanding of neurophysiology and anatomy to interpret the effects of acupuncture. Visit site

Acupuncture in Medicine latest news

Acupuncture in Medicine latest news