Not uncommonly, the issue of the generalisability or representativeness of a study population to the ‘real world’ is raised. There are some pretty big issues here, not least the idea that studies are undertaken in an unreal world, where troubles melt like chimney tops and sail above your lemon drops. Avoiding those sorts of mangled lyrics though – what’s the challenge of ‘representativeness’?

If we simply and split in into the patients, and the interventions, we can think about the threats to using study data to inform practice. As for patients – we need to ask the question – “In what ways do the patients in-study vary from the patients out of study?”. They may be older, sicker, whiter, or have fewer co-morbidities. In themselves, these things may have no effect. The follow-up question “How do these differences change how the illness/condition behaves?” holds the key. For if older patients have a greater response to treatment, or are more likely to spontaneously recover, we may not be able to use the results as they stand. If there are co-morbities in the general population which affect how intensively the therapy can be delivered, or increase the potential severity of the side effects, we should be careful.

When it comes to thinking about the intervention, there are some simple aspects to the questions we need to ask. Do we actually know what it was? (In drug studies we almost always do. In physio interventions, we often don’t really. Psychological ones may well be based on a manual but we may not have access to that.) If we know what it was, do we know how intensively it was monitored? The training and experience of the operators with the techniques? These may mean we won’t get quite the same results because of skill or craft differences. We shouldn’t forget the comparator in the studies either – if the New Thing is compared against something we think of as a Bit Rubbish then we really should be wary about taking results as they are given.

Some things we can’t get good evidence to guide us to answer. We may really not know if it matters if you were diagnosed when 6,12 or 24 weeks old for some things. We might only be guessing about interactions. We can only do what we can manage, with integrity and transparency, knowing our decisions are based on knowledge which is the best available at this time. The closest to truth we can get.