By Iain Brassington

You might remember the couple of days a few years ago in which the overlyhonestmethods hashtag went viral on Twitter: for those of you who don’t, it was a little joke in which academics – mainly, I think, natural scientists – made not-entirely-serious “confessions” about how they do their work and the corners they might sometimes be tempted to cut.  (Everyone knows that those of us in the humanities don’t really have methods, natch.)  Then someone wrote a blog post, since taken down, on plos blogs that complained that the hashtag was dangerous because of the damage that it might do to science in the public mind.  Similar concerns were aired elsewhere; this is an example, though much less po-faced than the former.  And it was the former that sprang to mind when I read Jessica Hamzelou’s editorial piece in last week’s New Scientist.

Her target is the seasonal edition of the BMJ and its traditionally lighter tone.  Part of her complaint is that some of the jokes aren’t… well, aren’t all that funny.  She notes the paper about man-flu, which got a fair amount of media traction, as an example, asserting that “[i]f this is meant to be a joke, it’s not a very good one”.  Now: maybe papers like this are basically fluff; and maybe that even as jeux d’esprit, they sometimes don’t hit all the high comic notes.  But so it goes: I don’t think that there’s all that much to worry about here, and I’m not going to get into a discussion about humour, beyond pointing out that there’s a difference between papers that are meant to be taken lightly and those that are meant to be funny, and that I suspect the BMJ selections tend towards the former category.  But there’s another side to her complaint:

[N]ot everyone is in on the joke – and in an era of fake news, maybe it is time for a rethink. The BMJ tells journalists reporting its papers, including these daft ones, to “please remember to credit the BMJ – this assures your audience it is from a reputable source”. And indeed, this silly science often receives straight-faced coverage from influential media outlets. What’s more, once it is archived in scientific databases, these papers get cited like any other. They are even used as the basis for future studies. After all, why wouldn’t you take the BMJ seriously? […]

And how might it be read in the future? Months or years down the line, devoid of the context of Christmas, who is to say this paper won’t be cited seriously? Could it influence the study of flu?

Well: yeah, but no.  One of the points I keep making to my students is that they shouldn’t treat rhetorical questions as if they are, or are capable of doing the work of, arguments.  After all, there’s a danger someone might answer them, and not in the way they expected.  And with that in mind…

The first thing to point out is that the papers are published in a particular place at a particular time.  I don’t know what the circulation of the BMJ is, but it’s not a mass-circulation magazine.  The target audience is fairly small, with very few general readers – limited access to academic journals has that effect, and it’s not a newsstand publication; moreover, my guess is that most papers will be read only when someone does a search for publications on a particular topic (or, slightly differently, that papers will be read most in the light of such searches).  More importantly, those readers will have a fairly refined set of skills when it comes to reading and using published work, and if a given paper is going to carry any weight in future work, there is only a very small chance, verging on none, that it’ll carry that weight alone.  This means that the man-flu paper, if it’s ever cited at all, is likely to be cited as an example of work into rhinoviruses; and there’s a heck of a lot of that about.  I simplify slightly, but not too much.

Still: information does sometimes get widely disseminated, and not everyone is in on the joke.  But so what?  It’s a good thing that science be accessible to the public, and it’s part and parcel of throwing things into the public domain that they’re open to misunderstanding and misrepresentation.  To insist that nothing should be published that is open to that is to insist that nothing should be published.  (Much the same kind of concern, by the way, might well be aired in the context of something like the Ig Nobel prizes.  And it’d have just as little heft.)  It’s worth remembering, too, that the research that has caused most damage when got hold of by the lay community is arguably Andrew Wakefield’s MMR stuff, which was not meant as a joke.

Maybe the appeal to fake news is meant to soften the claim – to save it from the noting-should-be-published reductio.  But, that being the case, an important consideration is missing, which is that there’s a difference between things that can be taken the wrong way, and “fake news”.  Fake news is either mendacious, or bullshit, in the Frankfurtian sense of the word.  If it’s mendacious, it’s intended at best to obscure truth, and possibly to plant a false idea.  If it’s bullshit, that intention isn’t there; but the thing that characterises bullshit is there’s no regard for truth – to which the liar must pay some respect, if only to avoid it – at all.

These points can be combined, too, to invite us to consider where the problem lies if a paper is misinterpreted or misused.  Sometimes it’s with the writer.  Sometimes a document can be so badly written that it’s very hard to work out what’s supposed to be going on, and misinterpretation is therefore the fault of the author.  But that’s not always the case.  Sometimes the fault lies with the reader.  For example, were I to read a paper about, say, a particular protein, there’s a good chance I’d come away with at best the wrong idea, and likely as not no idea at all.  The fault in that case would be mine: it’d be my the level of my understanding that tripped me up, and there’d be no reason at all to think that the writer wasn’t the most lucid, efficient, and elegant in the field.  Or – brace yourselves – we might also think about The Paper Of Which We Do Not Speak, the problem with which was less to do with any fault in the author’s argument than with a misunderstanding among some about how ethical arguments sometimes run.  “Fault” here might have a moral overtone, but it doesn’t have to; this point shouldn’t need making explicit, but there might be a reader out there who’d infer the wrong thing, tee hee.

If we’re in the business of criticising scientific papers, we need to be sure where the criticism is properly focused.  To say that something might be misunderstood when out in the wild is true enough; but misunderstandings require someone to do the misunderstanding.  They aren’t inherent in papers.

This year’s traditionally less serious Christmas edition of New Scientist includes a Judge Dredd strip, by the way.  Don’t go mistaking it for genuine reportage, will you?