Similar to many other journals, here at BMJ Quality & Safety, each year we choose a list of notable articles from the previous year to share with our readers. Our goals with this are to highlight specific examples of important work shared in the journal. But what does it mean to be selected as a ‘Top 10’ article? Here, we describe our editorial process for selecting these ‘Top 10 Articles’ and the attention these selected articles have received relative to articles that are not chosen.
There are many ways we might operationalize what constitutes ‘important work’, but as a first step we want to select those papers that have received the most attention, based on a combination of performance metrics. We therefore obtain a full list of articles published in the preceding year with their corresponding performance metrics, including Altmetrics and its subcomponents (including Twitter and other social media attention, press coverage and Mendeley readers), citations, and downloads. Only articles that include an abstract are eligible for inclusion in our ‘Top Articles’ list: Original Research, Systematic Reviews, Research and Reporting Methodology, Short Reports, and Quality Improvement Reports. Viewpoints and Editorials are therefore not included, although we do recognize that many of these also receive significant attention. As we publish articles online until the end of the calendar year, this selection process does not take place until early the following year.
A shorter list of about 20-25 articles with the highest performance across these metrics is created, adjusted for period of time since publication, and then reviewed by a team of editors to ensure any more recent articles with notable methodologic innovation or unique subjects are not excluded based on recency of publication. The full editorial team then scores these articles based on the following criteria: Novelty, Generalizability/Addition to understanding, Rigor of methods, and Importance to clinical practice. In most years, there is high consistency across the team for the majority of the ‘Top Articles’. In cases where an obvious cut-point is not evident after the initial voting, the final selection is made by a small team of editors after discussion of all papers, aiming for the final Top 10 to reflect the full breadth of subject matter, geography and novelty encapsulated by our journal. Both the full list of 20-25 articles and the final ‘Top 10 Articles’ are highlighted on the BMJ Quality & Safety blog each year.
Given that part of the selection process for the ‘Top 10 articles’ is based on performance metrics, these articles will inherently have higher Altmetric and attention scores, and are much more likely to have higher downloads and citations. On the other hand, the difference may be small, or for some papers it may take some time before they get attention, so we should also consider whether these papers continue to have more attention after being selected. To evaluate and inform our process for creating the ‘Top 10 Articles’ list, we have analysed our past data and several trends have become apparent. Adjusted for the time since publication, papers included in the ‘Top 10’ selection have received citations at roughly twice the rate of non-selected articles and have more than twice the rate of unique page views. Altmetric attention scores exhibit an even higher rate for the selected articles, at a more than a six-fold increase. After being selected as a ‘Top 10 Article’, these articles continue to be associated with more than twice the citation rate compared with other articles. While we cannot include every paper that ends up having a large impact on the fields of quality improvement and patient safety in our ‘Top 10’ list, we hope our broad editorial review of article criteria showcases much of what our readers value in BMJ Quality & Safety.