Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments

This is a study reported in the December 1999 issue of the Journal of Personality and Social Psychology, and can be found at http://www.apa.org/journals/psp/psp7761121.html . It recieved a 2000 Ig Nobel prize.

The study tested a few groups of Cornell undergraduates in various areas (humor, grammar, and logic.) It also asked the students to estimate how well they did on each test. It then compared the students estimations of their scores with their actual scores, and came to the sensationalistic conclusion that those who scored lower were worse at judging their own abilities. It took this conclusion and claimed that incompetent people didn't know they were incompetent, that their incompetence impaired their "meta-cognittive abilities" to judge themselves accurately.

This conclusion plays to the biases of intelligent people, telling us that the foolish are even more foolish than we think they are, that they don't even know they're dumb. It discounts people's claims to competence by allowing people to think "if they were as incompetent as I think they are, they would be claiming to be this good" and maintain our preconcieved notions about people in the face of some amount of evidence.

I find the study deeply flawed, and horribly misleading. The further conclusions drawn from its sound bitey title and summary compound the problem.

The study implies that people who are more competent are also more accurate at estimating their ability. This is not born out by the data. Subjects generally estimated they fell into the third quartile (between 50 and 75%) with their estimates roughly correlating with ability within that range. Thus, the people in the third quartile were most accurate in their assessments. The top quartile underestimated their abilities, the bottom two overestimated. And the group that overestimated the most was the bottom quartile. But that is simple because that group has the most room to error (since the average student placed their ability at 66%.)

This is exactly the result we would expect if we took a group of people in a narrow, high ability range with accurate assesments of their ability. They would mostly predict themselves to be above average. The bottom quartile would be off by the most, and we could heap the triple-insult of incompetence, obliviousness, and hubris (for overestimating their abilities, rather than humbly understimating them) upon them.

The subjects here were Cornell students, a group you'd expect to estimate themselves as above average (I won't comment on whether that is justified or not.) I imagine if you were to do a similar study using the same methodology on a group that rated itself to be below average, you would find that the most competent had the most inaccurate estimates of their ability. The article might then be drawing the conclusion that the humble dumb at least know their limits.