Read the rest of this series of blogs about designing and planning population based systems of care here.

Step 4: Choosing criteria

Choosing criteria is a more time consuming process and may need to be addressed separately. For example it is important to reach an agreed definition by what is meant by “diagnose quickly” and “diagnose accurately.”

Each objective needs to have one or more criteria associated with it in order to measure progress. Without criteria, objectives are meaningless.   

Several terms are used as synonyms for criteria: “metrics,” “indicators,” or “measures.” Some people use the term “measures” to mean criteria that are considered to be of greater validity than indicators. For example, they would use the term “measure” about systematic surveys of patient experience, whereas they would use the word “indicator” for the number of letters of complaint or commendation that a hospital chief executive receives. Although the latter is less costly, it is less
valuable in terms of feedback for the system than the accurate measures used in the former.

The trade-off between validity and feasibility

When selecting criteria, the most important issue to tackle is the trade-off between their validity and feasibility (and cost) of collecting the data. Validity is the degree to which a criterion measures what it is intended to measure; feasibility is the ease with which the information can be collected. In general, the greater the validity of a measure, the greater the cost to collect it. In the course of designing an information subsystem to underpin a healthcare system, the aim must be to build the required data into the routine data collection process. For instance, at Dartmouth Hitchcock Medical Center, there is a well developed information subsystem where patient preferences are routinely recorded, as is the type of treatment given to patients. It is then possible to compare the treatment received with stated preference to identify whether patients’ preferences are being met.

How to gain ownership of the criteria

The same group of people who set the objectives for a system can also be involved in selecting criteria. In addition, it is useful to invite others, such as academics, who can bring rigour to pinpointing definitions and the meaning of specific criteria. It may be necessary to set up a subgroup to address the issues associated with selecting appropriate criteria.

Muir Gray is visiting professor of knowledge management, Nuffield Department of Surgery, University of Oxford.