One of China’s ambitions is to promote the health of the whole nation. One significant challenge is a dearth of locally applicable, trustworthy clinical research evidence. Producing such evidence has become an important mission of Chinese healthcare researchers.
In recent years, many initiatives, including those from governmental authorities, academics, and healthcare professionals, have used real world data to produce evidence about the effects of healthcare interventions, disease epidemiology, prognosis—including clinical prediction models—and diagnosis. [1] Such real world evidence has the potential to inform healthcare practice and policy decisions. For example, the National Centre for Adverse Drug Reactions Monitoring established the Chinese Hospital Pharmacovigilence System in 2015 to actively identify and ascertain drug safety signals by using electronic medical records from hospitals across the country. The National Insurance Claims for Epidemiological Research group used the national claims databases to investigate the association of air pollution with different diseases and to explore the prevalence of rare diseases. [2] Several clinician groups have developed patient registries in areas such as cancer, diabetes, and vascular diseases, to tackle questions about disease management. [3-5]
Nevertheless, the quality of real world evidence has now become a central concern. A large volume of data from a real world setting does not necessarily mean high quality research evidence—indeed, sometimes quite the contrary. [6 7] Generating high quality evidence not only better informs decision making, but also avoids waste and distortion, thus meeting sound research ethics.
Concerns about the quality of real world evidence are generally two fold. One is the limited quality of data. Although the value of real world data, particularly retrospective data (such as electronic health records) is often emphasised, such data are often limited by misclassification, incompleteness, and loss of patients to follow-up. A previous paper has discussed these important matters. [1] However, there has been a lack of data standards across China until now. The other concern is the choice of study designs in the real world setting. Currently, observational study designs (such as retrospective cohort studies or case control studies) are overwhelmingly used; however, these designs carry a major concern about confounding, inherent with their observational nature. [6, 7] There is a widespread misconception that randomisation is not necessary, even impossible, in the context of real world data setting. [1,8]
So, what could we do? In response to these challenges, as one of the ongoing initiatives across the country, the Chinese Evidence-based Medicine Center, in partnership with governmental authorities, opinion leaders, research scientists, database holders and other stakeholders, established the China Real World Data and Studies Alliance (ChinaREAL) in 2017. This network is committed to improving the use of real world data for better healthcare and policy decisions across the nation. To produce locally trustworthy evidence, we have proposed, based on our experiences, the following recommendations for clinical research in China. We believe that these recommendations should be prioritised to generate high quality evidence for evidence based medical practice.
Firstly, setting up data standards represents the primary and critical step for using real world data for clinical research. Optimal storage, standardisation, and coding data are fundamental strategies to maintain quality of raw data; and the use of data inspection and validation processes throughout the studies is key to ensuring usable research databases. We suggest that data standards, categorised by different types of databases (such as electronic medical records v claims databases), should be established across the country, so that the raw data may be enhanced through standardisation and structuring.
Secondly, cautions are warranted in the choice of study designs that use real world data. Although observational studies are currently extensively used for assessing treatment effects, they typically serve to generate hypotheses about effectiveness. As usual, there is a hierarchy of “real world evidence” in assessing therapeutic effects of healthcare interventions; pragmatic randomised controlled trials (RCTs) continue to provide the best available evidence about effectiveness or comparative effectiveness. For research questions other than treatments, observational studies may be appropriate. For instance, they may, if measurement of prognostic factors and outcomes are accurate, provide the best evidence for prognostic studies, including the development of prognostic models.
Thirdly, and closely related to the second, we should continue to investigate the use of innovative designs that both involve the strengths of randomisation and real world data (such as pragmatic RCTs). Such studies, conducted in the regular practice setting, typically involve broad patient populations, randomise interventions as administered in routine clinical practice, and borrow advanced health informatics for data collection, such as the use of mobile devices and other electronic data capture technologies. [9] In addition, the use of databases (registries or electronic health records, for example) can facilitate pragmatic RCTs, and increase their efficiency and reduce cost, by providing a framework for patient enrolment and the primary source of outcome data. Such “registry based” RCTs are becoming increasingly popular, and initial evidence suggests their results approximate those of traditional designs. [10]
We hope that colleagues from the research community and stakeholders in China will work together to use these strategies, and ultimately improve the generation of real world data and subsequent evidence for better healthcare practice and policy decisions in this country.
Xin Sun is professor and director of the Chinese Evidence-based Medicine Center and Chinese Cochrane Center, West China Hospital, Sichuan University, China.
Youping Li is professor of Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, China.
Gordon Guyatt is distinguished professor of Department of Health Research Methods, Evidence and Impact, McMaster University, Canada.
Competing interests: none declared.
References:
1 Sun X, Tan J, Tang L, et al. Real world evidence: experience and lessons from China. BMJ 2018;360:j5262.
2 Tian Y, Liu H, Zhao Z, et al. Association between ambient air pollution and daily hospital admissions for ischemic stroke: a nationwide time-series analysis. PLoS Medicine 2018;15(10):e1002668.
3 Weng J, Zhou Z, Guo L, et al. Incidence of type 1 diabetes in China, 2010-13: population based study. BMJ 2018;360:j5295.
4 Wang XJ, Chi P, Zhang YY, et al. Survival outcome of adjuvant radiotherapy after local excision for T2 early rectal cancer: An analysis based on the surveillance, epidemiology, and end result registry database. European Journal of Surgical Oncology 2018 doi:10.1016/j.ejso.2018.08.024.
5 Wang Y, Cui L, Ji X, et al. The China National Stroke Registry for patients with acute cerebrovascular events: design, rationale, and baseline patient characteristics. Int J Stroke 2011;6(4):355-61.
6 Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. New England Journal of Medicine 2000;342(25):1878-86.
7 Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JP. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ 2016;352:i493.
8 Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence: what is it and what can it tell us? New England Journal of Medicine 2016;375(23):2293-97.
9 Ford I, Norrie J. Pragmatic trials. New England Journal of Medicine 2016;375(5):454-63.
10 Mathes T, Klassen P, Pieper D. No difference were found between effect estimates from conventional and registry-based randomized controlled trials. Journal of Clinical Epidemiology 2018. doi:10.1016/j.jclinepi.2018.09.011.