12 Nov, 15 | by BMJ
The Graphic Appraisal Tool for Epidemiological studies (GATE) is a simple, easily remembered toolkit to help you critically appraise epidemiological studies that includes one picture, two equations and three acronyms. GATE uses a picture of a triangle, circle, square and two arrows to represent the generic structure of epidemiological studies. We call this picture ‘the GATE frame.’ All common epidemiological study designs, from randomised controlled trials to case-control studies, can be illustrated using a GATE frame. While the GATE approach to critical appraisal covers the same ground as other critical appraisal guides, its point of difference is its generic graphic framework that emphasises the similarities between all study designs. With GATE, your goal is to ‘hang’ a study on the GATE frame as follows:
- Draw a blank GATE frame or open the appropriate GATE CAT workbook.
- Label the GATE frame with the first acronym (PECOT). Almost every epidemiological study shares the same basic structure of five parts: Participants (P); Exposure (E) group(s); Comparison (C) group; Outcome (O) and Time (T). Then annotate the GATE frame to describe each of the PECOT parts; this can help you clarify the specific design features of a study. The Exposure & Comparison groups, for example, could be smokers & non-smokers’ or ‘aspirin & placebo.’ The arrows represent the timing of outcome measurements, with events measured over a period of time (e.g. new episodes of disease, like heart attacks) illustrated by a vertical arrow and events measured cross-sectionally (e.g. event states, like diabetes) by a horizontal arrow. You may want to draw several GATE frames or fill in several GATE CATs if there are multiple exposure groups or outcomes.
- Next, hang the study numbers on the labelled GATE frame, including the number of Participants, the numbers allocated to the Exposure and Comparison groups (EG & CG) and the numbers of Outcomes. In the simplest case, as shown in the picture above, there are 4 Outcome groups (a, b, c & d) – ‘a’ (and ‘b’) representing the number of participants from the Exposure (and Comparison) groups in whom the study outcome occurs; whereas ‘c’ & ‘d’ represent groups of participants in whom the study outcome has not occurred.
- If possible, then calculate the occurrence of outcomes in each of the Exposure/Comparison groups, using the first equation (occurrence = number of outcomes ÷ number in study group). This equation represents the underlying purpose of all epidemiological studies. In the study illustrated in the GATE frame picture above, the Exposure Group Occurrence (EGO) = a ÷ EG and the Comparison Group Occurrence (CGO) = b ÷ CG. Estimates of EGO and CGO can then used to calculate ‘relative risks’ (EGO ÷ CGO) and ‘risk differences’ (EGO – CGO). If the study outcome is numeric (e.g. blood glucose levels) rather than categorical (e.g. diabetes), then EGO and CGO can be calculated as means or medians (e.g. EGO = sum of the blood glucose levels of participants in EG/number of participants in EG).
- Assess the degree of bias (or non-random error) using the second acronym (RAMBOMAN) to help remind you of the main causes of bias in a study. Bias can occur when participants are being ‘Recruited’ (R) or when participants are ‘Allocated’ (A) to exposure and comparison groups. In studies that require follow-up, bias can also occur if participants are not ‘Maintained’ (M) in the groups they were initially allocated to. ‘Blind’ or ‘Objective’ ‘Measurement’ (BOM) of outcomes (and also of exposures and comparison exposures) is a way of reducing measurement bias. Finally, the ‘AN’ of RAMBOMAN refers to bias introduced at the ‘Analysis’ stage of a study, for example not undertaking ‘intention-to-treat analyses or not adjusting for confounders. A good way to help you judge the importance of biases in a study is to ask the question: ‘how is this bias likely to effect the calculated EGO or CGO?’
- Assess the degree of random error using the 95% confidence interval. The second formula: random error = 95% CI will remind you of this simple way to estimate the degree of random error (or precision) in measures of occurrence (EGO & CGO) or measures comparing occurrences (e.g. relative risks).
- If you are appraising a systematic review of studies rather than an individual study, the third acronym – FAITH – will remind you of the key issues to consider. In a good systematic review the investigators will systematically search for and ‘Find’ (F) the relevant studies. They will then ‘Appraise’ (A) the studies for their validity and precision and only ‘Include’ (I) the high quality studies (a common error is the inclusion of low quality studies). If the results of studies included in a systematic review are analysed as a meta-analysis, that ‘Total-up’ (T) the findings of individual studies, it is important to check that this was only done if the results of individual studies were sufficiently similar (i.e. not ‘Heterogeneous’ (H)).
Finally, the ‘X’ beneath the GATE frame is to remind you that the epidemiological evidence derived from valid studies is only one (albeit important) component of good clinical and other health-related decision-making. There are usually other important factors to consider, represented by the other three quadrants in the X-factor figure. An eXcellent decision-maker is one who integrates information from these four quadrants when making decisions; ideally a shared decision made with an informed patient or population.
If you would like to try out the GATE approach, we suggest you either start with a blank piece of paper and draw a GATE frame, or use one of our GATE CAT workbooks. We have developed a series of MS Excel GATE CAT (Critically Appraised Topic) workbooks, that are relevant to particular questions or study designs, that you can download from our website. You will also find a power point presentation, a YouTube video and a short paper describing the GATE approach to critical appraisal. This is a non-commercial initiative so feel free to copy and use and translate.
The GATE approach is equally applicable in clinical, health services or public health teaching and practice. We also use it for designing studies (Graphic Architectural Tool for Epidemiology) and for teaching introductory courses in epidemiology (Graphic Approach To Epidemiology). While the idea of GATE and much of the development has been undertaken by the author of this blog, many of colleagues and students have provided significant input and helped shape the current version.
Rod Jackson is a professor of epidemiology at the University of Auckland, New Zealand. He is medically trained, has a PhD in epidemiology and is a fellow of the New Zealand College of Public Health Medicine.
He has been teaching courses on clinical and public health epidemiology to undergraduate and postgraduate students for 25 years and runs workshops on critical appraisal of the clinical literature nationally and internationally.
He has 35 years of research experience in cardiovascular disease epidemiology. In the 1990s he led the development of New Zealand’s absolute risk-based clinical guidelines for managing CVD risk factors. For the past 15 years his research has been mainly based on using evidence about CVD risk and risk management from clinical practice to improve practice. He leads a big health data research programme that generates very large cohort studies from web-based clinical decision support systems linked to national health databases to implement, monitor and improve CVD risk assessment and management in primary and secondary care.