Last week I considered the origins of the word “assessment”. It comes ultimately from an IndoEuropean root, SED, to put something down or sit. That in turn spawned the Latin verb sedēre, to sit, from whose past participle, sessum, English words ending in –sess derive. Assess is formed by adding the prefix ad–, in this case meaning beside. So to assess literally means to sit beside, and hence to advise, or by extension to judge, since an assessor was originally one who sat beside a judge or other official, as an assistant, giving advice.
We all get assessed from time to time. The process is sometimes called appraisal, but whatever you call it, it involves presenting yourself in some way or other to be judged.
But what happens when you have to judge yourself? Judging your own mental abilities is known as metacognition. The prefix “meta-” is used to designate, among other things, any higher science (actual or hypothetical) that deals with more fundamental problems than the original science itself. Just as meta-analysis is analysis of analyses, so metacognition is thinking about thinking—awareness and understanding of one’s own thought processes. Are we any good at it? The Dunning–Kruger hypothesis suggests not.
In 1999 Justin Kruger and David Dunning, from Cornell University, performed four studies, in which they tested University undergraduates, assessing a subjective attribute, sense of humour, and two objective attributes, ability to reason logically (two studies) and knowledge of English grammar; they then asked them to assess how they had performed in each test. The results in all four tests suggested that the subjects tended to overestimate their own abilities relative to those of their peers, except for a few individuals with high abilities, who tended to underestimate themselves. Some, in the middle, got it roughly right. I have combined the very similar results of the four studies in a single graph, showing the relation between perceived and actual ability (Figure 1). The results, often cited (Figure 2), suggest that we are not very good at self-assessment, thinking about how we think, at least in the domains that they tested. That we tend to overestimate our cognitive abilities, apart from a few talented individuals, and even they may overestimate their performance in some areas.
Figure 1. Combined results of the four studies carried out by Kruger & Dunning (1999); left-hand panel—the relation between perceived ability and measured ability, given as test scores; right-hand panel—a mean–difference plot, as recommended by Bland & Altman for analysing the relation between the results of two different measurements of the same variable; those with poor abilities greatly overestimate them, while those with very high abilities underestimate them, but not by as much; those of moderate ability may estimate their abilities quite well
Figure 2. The numbers of yearly citations of Kruger & Dunning’s 1999 paper (source Google Scholar)
These results and the hypothesis that accompanies them, have been criticized. Some, for example, have suggested that they are confounded by what has been called the better-than-average effect. This is an odd criticism, since that is exactly what Kruger and Dunning showed as a main outcome, not a confounder. Another, more cogent criticism is that they are due to regression to the mean, but Kruger and Dunning themselves considered that and rejected it as a main contributor to the effect, arguing that had that been problematic the difference between perceived and measured ability would have been equal across the board, which it was not. It has also been claimed that since simulations that incorporate these confounders produce similar outcomes, the Dunning–Kruger effect must be a statistical artefact; but different assumptions can produce similar outcomes, and simulations do not necessarily clarify interpretation of real-life results. Nor do results that appear to falsify the hypothesis shed much light when they relate to areas other than those originally studied.
It has also been argued that the difficulty of a task modifies the accuracy with which individuals assess their ability to perform it. However, Kruger, Dunning, and colleagues showed that difficulty was not a modifying factor. Others have published confirmatory studies in relation to sports coaching, sexism and racism, and information literacy.
That an individual’s own perception of their ability to perform a task seems to be poor in those at the incompetent end of the scale, Kruger, Dunning, and colleagues have suggested, is because “their incompetence deprives them of the skills needed to recognize their deficits”. So, those who feel certain of so many things about which we should be uncertain, and who pronounce upon them with apparent authority, no little of which is currently in evidence, might recall the words of Bertrand Russell, in New Hopes for a Changing World (1951): “One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision”. The Dunning–Kruger hypothesis seems to support Russell’s timeless assertion.
Jeffrey Aronson is a clinical pharmacologist, working in the Centre for Evidence Based Medicine in Oxford’s Nuffield Department of Primary Care Health Sciences. He is also president emeritus of the British Pharmacological Society.
Competing interests: None declared.
