Jeffrey Aronson: When I use a word . . . Cherry picking and berry picking in systematic reviews

jeffrey_aronsonCherry picking originally meant “the action or practice of harvesting cherries” (Oxford English Dictionary). The term is recorded as having been first used in November 1849, in Godey’s Lady’s Book: “Recollections of a grown-up schoolboy. V.—Cherry-pickings, robbing orchards, and love-making”, one of a series by JF Otis, although it is hard to believe that there are no earlier instances.

However, it took until the 1960s for the term to acquire its colloquial metaphorical meaning, “the action or process of selecting only the best or the most profitable items, opportunities, etc”. In this sense it is first cited in an advertisement in the New York Times Book Review (11 July 1965): “The Great Discount Delusion . . . reveals the gimmicks some discounters use—among them: phony price tickets, short weight, . . . ‘creaming and cherry-picking’, ‘baiting-and-switching’.” A back formation soon appeared, to cherry-pick: “to choose selectively (the most beneficial or profitable items, opportunities, etc.) from what is available.” Incidentally, baiting and switching is a technique of advertising a product at an attractive price “in order to inveigle a consumer into buying a more expensive (or sometimes inferior) substitute” or “a hoax in which initially appealing aspects are designed to conceal a much less favourable prospect or outcome” (OED).

The similar term berry picking, which is not, or not yet, listed in the OED, originally meant no more than to pick berries. The earliest example I have found is from a 1933 poem by Duncan Campbell Scott (picture). The term acquired a different meaning in the 1970s, as far as I can discover, describing a surgical technique for repeatedly excising recurrent tumours, originally fibrous histiocytomas and then individual lymph nodes associated with thyroid tumours.

Since then the term has been applied in a different field altogether, starting with a 1989 article by Marcia J Bates, titled “The design of browsing and berrypicking techniques for the online search interface”. Bates described the classic model of information retrieval used in information science research as a linear one, in which a single query directly matched the information contained in a document to the information needed; the form of the query had to be tailored to the relation between the document and the requirement. However, she observed that increasing availability of whole libraries, accessible by computer, and searchable word by word, made the searcher’s task more complex and altered the search strategy. Now a query yields information, which causes revision of the query, and so on iteratively. Thus, the single query is replaced by multiple queries: She called this process “bit-at-a-time retrieval”, and likened it to picking berries in the forest. “The berries are scattered on the bushes; . . . One must pick them one at a time.”

Because the nature of the search process has changed, so too have the search techniques used. Bates described six: reference chasing (or backward chaining), citation searching (or forward chaining), searching whole runs of key journals, area scanning (i.e. browsing materials that are collocated with materials located earlier in the search, e.g. using MeSH terms), subject searches in bibliographies and abstracting and indexing services, and author searching. These are techniques with which anyone who has done a systematic review will be familiar. Before systematic reviews became de rigueur as part of evidence based medicine, authors could chose to cite only evidence that supported their assertions. Now such cherry picking is forbidden, but berry picking is essential.

This analysis shows, as is self-evident to modern researchers, that search strategies in systematic reviews are nonlinear. Indeed, it should be equally self-evident that that is true of most scientific endeavours, although in some areas that does not seem to have been so widely appreciated.

For example, when Vannevar Bush, in July 1945, outlined his vision for the future of US scientific research, in a document titled “Science, the endless frontier”, he stressed the importance of basic research. Since then, the extent to which basic science and applied science individually contribute to technological developments has been debated, but generally with the implicit assumptions that the components are dichotomous and that the transition from one to the other is linear. I shall discuss this next week in the context of translational research.

Meanwhile, I note in passing that actual berry picking may not be all that harmless a pursuit, at least in some parts of the world. In a Slovenian series of 2779 notified cases of tick borne encephalitis, 30% occurred from tick bites contracted during mushroom or berry picking.

Duncan Campbell Scott (1862–1947), one of Canada’s Confederation Poets

“Above the field on the rocky point
Was a cluster of canvas tents,
Nearly deserted, for the women had gone
Berry-picking at dawn
With most of the children.”

(From “A Scene at Lake Manitou”,1933)

Jeffrey Aronson is a clinical pharmacologist, working in the Centre for Evidence Based Medicine in Oxford’s Nuffield Department of Primary Care Health Sciences. He is also president emeritus of the British Pharmacological Society.

Competing interests: None declared.