Aalborg University Hospital – Psychiatry in the North Denmark region has pioneered a new approach to managing patients with schizophrenia and depression based on asking patients to routinely monitor and record their responses to a defined set of co-created patient reported outcome measures. Solvejg Kristensen, research leader of the ” PRO-Psychiatry” initiative and Maria Bonde, a teacher and former mental health service user, describe how it works. Jens Holmskov, doctor and chief medical manager, comments on the impact of covid-19 on the initiative.
National and local calls for greater patient involvement at the point of care, and more focus on patient outcomes (PROs) as a quality indicator, prompted the senior management at the psychiatric hospital in North Denmark Region to develop a new approach to management based on using patient reported outcomes in routine clinical care.  The initiative, called PRO-Psychiatry, was launched in 2016 and has grown to become a national research and action-oriented initiative within the Danish National Clinical Registries for depression and schizophrenia. The key components are:
- Self-monitoring by patients of PROs co-designed with patients.
- Use of these outcomes to steer discussion with clinicians (doctor, psychologist or nurse) at the point of care
- Incorporating PROs alongside existing quality assessment measures in the national Danish clinical quality registries for unipolar depression and schizophrenia.
- Using PROs data to inform research projects. 
PRO-Psychiatry stands out among other Danish patient reported outcome initiatives because of its extensive involvement of service users as co-leaders of the project and because patient representatives sit on the steering committee. In addition, the development and monitoring of the project is overseen by a Patient Peer Board which consists of 10 current and former users of psychiatric services across Denmark.
PRO-Psychiatry was designed for use with patients with unipolar depression and schizophrenia, but the approach has now been adopted by other psychiatric patients in contact with the hospital services, e.g. patients with anxiety disorders, eating disorders, post traumatic stress disorder, and stress and adjustment disorders.
Both inpatients and outpatients are invited to participate. Patients admitted to the hospital are informed by the admitting staff about the opportunity to use PRO-Psychiatry. Outpatients are informed at their first consultation. Patients who want to participate, are enabled to do so via registration in their electronic health record.
The PRO-data are collected electronically at www.pro-psykiatri.dk. Patients can input their data via their own smart phone, tablet or PC. In-patients are provided with a tablet by the hospital, and outpatients can use a kiosk in the waiting area in the outpatient clinics. The patient logs on to the PRO-questionnaire using his or her unique personal identifier and a code is sent to the patient’s mobile phone or handed to the patient by the hospital staff.
The patients PROs are transferred to their electronic health record in real time, to the individual electronic health portal (sundhed.dk ≈ eHealth) and later to the national clinical registries (currently only patients with unipolar depression and schizophrenia). The real time data transfer allows the patient to answer the PRO-questionnaire shortly before the consultation with the treating doctor, psychologist, or nurse. PRO-data are displayed in a simple easily readable graphical layout in the electronic health record allowing them to be seen, read, and used during the clinical consultation. PRO-data are provided at admission and discharge from hospital if the stay is 14 days or more. Likewise, data are provided from outpatients prior to every consultation given 14 days apart.
As of September 2020, ten psychiatric units in North Denmark Region had implemented or were in the process of implementing PRO-Psychiatry; 960 patients had agreed to participate. Of these 787 have already done so, but 173 have either not had the opportunity to answer yet, have changed their mind regarding participation or are too unwell to complete the questionnaire.
Figure 1. Flow of PRO-Psychiatry and differentiated use of the PRO-data
An evaluation of PRO-Psychiatry conducted in 2019 found that:
- Patients found the PROs questionnaires useful, but needed advise on how to fill in the forms and use them in the consultation
- In-patients who used PRO-Psychiatry reported improved quality of clinical encounters linked to use of the PROs: clinicians actively sharing, discussing, and addressing the PRO-results during the clinical consultation.
It’s been really gratifying that our hospital won the Danish Region’ 2020 user involvement prize for patient involvement at the hospital. 
I have been involved in the PRO-Psychiatry initiative since its outset and was involved in the co-creation process of the measurement concept, but despite this, during the pilot stage of the project in early 2019, I experienced how the developer’s and the patient’s perspective differed. Being the developer, I imagined the needs I would have as a patient. We looked for outcome measures which would capture what we as patients find important to let the clinicians know, information which reflect issues of health which has an impact on my daily life. We worked very thoroughly defining the measures in the Patient Peer Board and after the pilot study ensured we had measures which incorporated the link between medical side effects and quality of life.
I found that looking at my PRO-answers on the PC screen together with my “doctor” was a useful starting point for our dialogue during the consultation. It helped us focus on the issues which were most troublesome to me, get a common picture of why this was so, and discuss possible ways to tackle the problems. During the evaluation of PRO-Psychiatry, I interviewed about 20 patients and learned that other patients had the same positive experience as me. But I also learned that not all patients reported that their PROs had been addressed during the clinical consultation.
“PRO-Psychiatry has been a very positive experience for me. I used to simply passively sit on the other side of the table as a patient. Now I’ve worked in the engine room. I have gained a totally new insight into what is realistic for patients to expect and wish for in their contact with health care. My involvement has also enhanced my confidence and competences as a social worker/teacher/youth worker, which had faded away after many years of illness.
No changes have been made to PRO-Psychiatry initiative during the covid-19 crisis, but implementation has slowed as our initial priority was to build emergency preparedness. April 2020 was associated with a marked decrease in referrals to psychiatric care, but it’s clear that anxiety levels in the general population and in the psychiatric community in particular have been escalating throughout the pandemic and underline the need for strengthening psychiatric care. 
Changing the outpatient consultations to online video consultations has been technically challenging, but we know from individual clinicians that using PRO-Psychiatry in the online consultations is possible. So, we need to pay attention to this going forward. In the inpatient setting no changes have been made to PRO-Psychiatry as a result of covid-19, but again maintaining and starting new patients on the system is proving more difficult. Currently, the original measurement concept of PRO-Psychiatry has been deemed sufficient to embrace issues which might arise as a cause of covid-19. For now, we need to focus on using the PROs actively.
A decision has been made by the regional directors of health and the regional directors of psychiatry to pilot and use PRO-Psychiatry in all five Danish regions. Spread is being fostered by mounting implementation conferences in each region which are hosted jointly by regional health committees and the Danish Clinical Registries. Currently three other regions have started to implement PRO-Psychiatry. Broad use across the Danish health care system is set to improve the quality of care for patients with mental health conditions and underline the value of a unified and truly collaborative approach to providing patient centred care.
PRO-Psychiatry puts the patient’s own perception of his or her health in the centre of the dialogue around the patient’s needs, priorities and wishes for care, treatment and recovery.
It is still too early to assess the full effect of PRO-Psychiatry, but with the integration into the Danish Clinical Registries, it should be possible to monitor the quality of care as experienced by the patients over time.
PRO-Psychiatry is patient centred initiative for as one patient put it: “it allows me to describe my self-perceived health status directly in my electronic health record, without interpretation by a clinician or anyone else”!
We would jointly like to acknowledge the contribution of all participating health care professionals, leaders, The Obel Family Foundation, the Danish Clinical Registries and most of all the steering committee and all patient peer board members. Also thank you to the staff at the unit for co-creation at Aalborg University Hospital – Psychiatry for their contributions and cooperation. A special thanks to chief medical doctor Jens Holmskov and to the three directors of Aalborg University Hospital – Psychiatry; Anette Sloth, Tina Gram Hansen and Jan Mainz for their engagement in peer support and co-creation.
- Kristensen S, Mainz J, Baandrup L, Bonde M, Videbech P, Holmskov J, et al. Conceptualizing patient-reported outcome measures for use within two Danish psychiatric clinical registries: description of an iterative co-creation process between patients and healthcare professionals. NordJPsychiatry. 2018:1-11.
- Information about PRO-Psychiatry: 1. https://psykiatri.rn.dk/genveje/brugerinddragelse/individuel-brugerinddragelse/pro-psykiatri/in-english ; 2. https://pri.rn.dk/Assets/30057/PSYN01-114_PSYN01-114-Engelsk-vejledning.pdf ; 3. https://pri.rn.dk/Assets/30057/PSYN01-115_PSYN01-115-Questionnaire.pdf: Aalborg University Hospital – Psychiatry, North Denmark Region, Denmark; [
- Kristensen S HJ, Baandrup L, Bonde M, Haller L, Videbech P, Mainz, J. Evaluering af PRO-Psykiatri – Baseret på erfaringer fra Psykiatrien i Region Nordjylland. (Evaluation of PRO-Psychiatry – Based on the experiences from Aalborg University Hospital – Psychiatry) ISBN 978-87-993493-1-9. Aalborg: Aalborg University Hospital – Psychiatry; 2020. 42 p.
- TV2Nord. “Psykisk syge bliver væk: – Fik jeg ikke behandling, var jeg indlagt” (Psychiatric ill people stay away – If is did not get ambulatory care treatment, I would be hospitalized”. In: J BJH, editor. 2020. p. https://www.tv2nord.dk/coronavirus/drastisk-fald-i-psykisk-syge-patienter-bekymrer-regionen.
Other information about PRO-Psychiatry