Edward Davies: The big bang meeting vs peer review

Edward DaviesIn the world of oncology, and indeed the world in general, meetings don’t come bigger than that of the American Society for Clinical Oncology. I hesitate to quote a cab driver as a source of fact, but the one I met at Chicago airport told me more than 30,000 delegates were coming through the city over the weekend, and I have no reason to doubt that. Indeed it seems an almost conservative estimate. Anybody who’s anybody is here. It’s the place to be seen. It’s where it’s at. Etc.

But just how much value does it add to patient care?

The program is a 336 page bound book, containing more sessions than I can possibly count, and each morning journalists are spoon fed half a dozen of the day’s highlights to write about, each prefaced with the adjective “remarkable,” and two with “breakthrough.” But very few of them have been published or peer reviewed and what the delegates take home or you end up reading about in your newspaper is often fresh from the author’s mouth and one or two acolytes. That’s by no means to say that peer review is the answer to all the world’s ills, or that these aren’t perfectly good papers anyway, but many of these subjects are on incredibly complex, super-specialised subjects, and we are effectively drinking unpasteurised milk. It may be fine, but it could end in tears.

Nor is this a slight on researchers presenting or indeed journalists reporting. I am a journalist. They are terrific people and experts in their field, every last one of them. Outstanding. But many at the conference are on hourly deadlines—indeed for some, every minute counts. That’s the nature of news. They are bright experienced people and will be able to get some comment and insight, but there’s no way anybody here can consult half a dozen independent experts, a methodologist, or a statistician about a story. They just can’t.

Much of what’s presented here is interesting, and it’s quite probable that some of what I’ve seen here is excellent. It will be published. It probably is “remarkable.” Who knows, maybe one was indeed “a breakthrough.” But you can be sure some of it won’t be—indeed, much of it. Of the 336 pages of talks, presentations, and posters, how many will end up changing patient care, and even if they do, what will be the benefit to patients that they were first presented here, unpasteurised, rather than after going through the eminently imperfect, but at least expert and thorough, crucible of peer review?

Add this to the cumulative time out of work and the many millions of air miles, and it all rather begs the question of how helpful a conference like this is?

Well, 30,000 registered attendees say it is useful, so I am well and truly put in my place. But if that is the case, what is the point in peer reviewed publications?

Edward Davies is US news and features editor, BMJ.