Rebecca Baines and John Donovan
As part of the revalidation process, doctors must collect patient feedback at least once every five years. While it is encouraging to see patient feedback included in this process, questions must be asked about what message, whether intentional or not, the requirement to collect patient feedback so infrequently sends to those involved. Are we unintentionally suggesting that patient insight is only worth exploring once every five years? As a research participant notes, our current model of “revalidation is so invalidating of the patient experience…”*
Currently, patient feedback for revalidation purposes must be collected using validated questionnaires. Our existing definitions of validity suggest that validated tools are those whose psychometric properties have been tested and evaluated, (often with limited patient involvement), leading to the conclusion that “high quality feedback will only be achieved if the instrument has good psychometric properties.” [1] This approach we argue is fundamentally flawed. Validity is more than psychometric validation. How stakeholders perceive, create, value, and respond to the process of patient feedback is also vitally important.
While validated, many patients, carers and doctors question the validity of existing patient feedback tools. Some perceive existing tools as a “tick box exercise” purposefully designed “to get reasonable results” with opportunities for doctors and patients to “play the game” through biased patient selection and biased patient responses, i.e. “to get proper feedback… you’re going to send the questionnaires to patients you know will respond and you know like you.” Patients are also hesitant to share critical and honest feedback due to a fear of repercussions e.g. “the fact is, I would actually lie, I would have too…” While structurally sound, existing patient feedback tools may not therefore be as valid as they seem.
This does not however mean to suggest that patient feedback is not worth collecting and engaging with. Quite the opposite. Our existing methods and processes just need refining. To begin with, we must clarify and agree what the purpose of patient feedback is. Why are we collecting it? What is it we are wanting to achieve? And are we asking the right questions in the right way to achieve this?
Are we looking to measure and compare patient feedback scores amongst doctors, or are we looking for learning, change, and development? If it is the latter i.e. a truly formative exercise, then the need for psychometric validation arguably becomes redundant. Other features that facilitate learning and reflection such as a tools ability to collect patient feedback in near real-time, share a patients’ story through narrative comments and support a feedback loop between feedback providers and users become more relevant. A broader and more flexible approach to defining what constitutes patient feedback validity is needed, preferably one that incorporates patient values, choice and voice.
Rather than validating tools based on their psychometric properties alone, we should also be critically exploring who defines and designs their content. Currently, our existing requirements for patient feedback in the context of revalidation conflates value with validity. This jeopardises its potential benefits leading to a “tick-box” exercise that is unnecessarily tokenistic for both doctors and patients. Patient feedback does not need, nor deserve, to be a tokenistic activity. While it is not our responsibility to validate, or indeed invalidate, a patients experience based on their perceived, or assumed vulnerabilities, it is our duty to listen, learn, and respond.
So in answer to our question of when is patient feedback “valid”? The answer is nearly always, provided its purpose, process, and design is valued by all those involved. A patients’ experience is valid to that individual and worthy of exploration regardless of its form, source, or external validation by a psychometric test. Patients are experts by experience and therefore, the patient voice is arguably one of our greatest, but at times, most unsupported learning resources.
Patient feedback is worth more than the internal structure of the questionnaire it is written on. Patient feedback can, and should, be a catalyst for change. We argue for a more flexible approach that focuses on value as opposed to validity, content as opposed to form, and co-design as opposed to dictation. By doing so, we remind ourselves that validity does not necessarily equate to value. The true value of patient feedback is its ability to inspire, celebrate, and educate. We must therefore ensure we are working in partnership with, and not in opposition to the patient voice.
*All quotes come from a research project.
Rebecca Baines is a researcher and PhD student in CAMERA (Collaboration for the Advancement of Medical Education Research and Assessment) at the University of Plymouth. Her research interests include patient and public involvement, patient experience and quality improvement. The research project that led to this opinion piece has been conducted in collaboration with a volunteer mental health patient research partner (John Donovan).
Competing interests: None declared.
Twitter: @Rebecca_Baines_
John Donovan is a designer who now volunteers as a patient research partner following a long history of auditory hallucinations, intermittent paranoia and hypomania. He has a strong passion for active patient participation and involvement.
Competing interests: None declared
References:
- Royal College of Psychiatrists. What is ACP 360? 2017 [Available from: http://www.rcpsych.ac.uk/workinpsychiatry/qualityimprovement/acp360/whatisACP360.aspx] accessed 07.07.2017.