Science & Social Science

Unskilled But Aware: Rethinking The Dunning-Kruger Effect

June 15, 2023 1563
Crumpled up blue piece of paper on a notebook with drawings of a lightbulb and other sketches.
(Photo: Fancycrave1/Pixabay)

John Cleese, the British comedian, once summed up the idea of the Dunning–Kruger effect as, “If you are really, really stupid, then it’s impossible for you to know you are really, really stupid.” A quick search of the news brings up dozens of headlines connecting the Dunning–Kruger effect to everything from work to empathy and even to why Donald Trump was elected president.

As a math professor who teaches students to use data to make informed decisions, I am familiar with common mistakes people make when dealing with numbers. The Dunning-Kruger effect is the idea that the least skilled people overestimate their abilities more than anyone else. This sounds convincing on the surface and makes for excellent comedy. But in a recent paper, my colleagues and I suggest that the mathematical approach used to show this effect may be incorrect.


What Dunning and Kruger showed

In the 1990s, David Dunning and Justin Kruger were professors of psychology at Cornell University and wanted to test whether incompetent people were unaware of their incompetence.

To test this, they gave 45 undergraduate students a 20-question logic test and then asked them to rate their own performance in two different ways.

First, Dunning and Kruger asked the students to estimate how many questions they got correct – a fairly straightforward assessment. Then, Dunning and Kruger asked the students to estimate how they did compared with the other students who took the test. This type of self-assessment requires students to make guesses about how others performed and is subject to a common cognitive mistake – most people consider themselves better than average.

Research shows that 93 percent of Americans think they are better drivers than average, 90% of teachers think they are more skilled than their peers, and this overestimation is pervasive across many skills – including logic tests. But it is mathematically impossible for most people to be better than average at a certain task.

After giving students the logic test, Dunning and Kruger divided them into four groups based on their scores. The lowest-scoring quarter of the students got, on average, 10 of the 20 questions correct. In comparison, the top-scoring quarter of students got an average of 17 questions correct. Both groups estimated they got about 14 correct. This is not a terrible self-assessment by either group. The least skilled overestimated their scores by around 20 percentage points, while the top performers underestimated their scores by roughly 15 points.

The results appear more striking when looking at how students rated themselves against their peers, and here is where the better-than-average effect is on full display. The lowest-scoring students estimated that they did better than 62 percent of the test-takers, while the highest-scoring students thought they scored better than 68%.

By definition, being in the bottom 25 percent means that, at best, you will score better than 25 percent of people and, on average, better than just 12.5 percent. Estimating you did better than 62 percent of your peers, while only scoring better than 12.5 percent of them, gives a whopping 49.5 percentage-point overestimation.

The measure of how students compared themselves to others, rather than to their actual scores, is where the Dunning–Kruger effect arose. It grossly exaggerates the overestimation of the bottom 25% and seems to show, as Dunning and Kruger titled their paper, that the least skilled students were “unskilled and unaware.”

Using the protocol laid out by Dunning and Kruger, many researchers since have “confirmed” this effect in their own fields of study, leading to the sense that the Dunning–Kruger effect is intrinsic to how human brains work. For everyday people, the Dunning-Kruger effect seems true because the overly arrogant fool is a familiar and annoying stereotype.

David Dunning On The Dunning-Kruger Effect
Listen to David Dunning describe the Dunning-Kruger Effect on the Social Science Bites podcast.

Debunking the Dunning-Kruger effect

There are three reasons Dunning and Kruger’s analysis is misleading.

The worst test-takers would also overestimate their performance the most because they are simply the furthest from getting a perfect score. Additionally, the least skilled people, like most people, assume they are better than average. Finally, the lowest scorers aren’t markedly worse at estimating their objective performance.

To establish the Dunning-Kruger effect is an artifact of research design, not human thinking, my colleagues and I showed it can be produced using randomly generated data.

First, we created 1,154 fictional people and randomly assigned them both a test score and a self-assessment ranking compared with their peers.

Then, just as Dunning and Kruger did, we divided these fake people into quarters based on their test scores. Because the self-assessment rankings were also randomly assigned a score from 1 to 100, each quarter will revert to the mean of 50. By definition, the bottom quarter will outperform only 12.5% of participants on average, but from the random assignment of self-assessment scores they will consider themselves better than 50% of test-takers. This gives an overestimation of 37.5 percentage points without any humans involved.

To prove the last point – that the least skilled can adequately judge their own skill – required a different approach.

My colleague Ed Nuhfer and his team gave students a 25-question scientific literacy test. After answering each question, the students would rate their own performance on each question as either “nailed it,” “not sure” or “no idea.”

Working with Nuhfer, we found that unskilled students are pretty good at estimating their own competence. In this study of unskilled students who scored in the bottom quarter, only 16.5 percent significantly overestimated their abilities. And, it turns out, 3.9% significantly underestimated their score. That means nearly 80 percent of unskilled students were fairly good at estimating their real ability – a far cry from the idea put forth by Dunning and Kruger that the unskilled consistently overestimate their skills.

Dunning-Kruger today

The original paper by Dunning and Kruger starts with the quote: “It is one of the essential features of incompetence that the person so inflicted is incapable of knowing that they are incompetent.” This idea has spread far and wide through both scientific literature and pop culture alike. But according to the work of my colleagues and me, the reality is that very few people are truly unskilled and unaware.

The Dunning and Kruger experiment did find a real effect – most people think they are better than average. But according to my team’s work, that is all Dunning and Kruger showed. The reality is that people have an innate ability to gauge their competence and knowledge. To claim otherwise suggests, incorrectly, that much of the population is hopelessly ignorant.

Eric Gaze is a senior lecturer of mathematics at Bowdoin College.

View all posts by Eric Gaze

Related Articles

Pandemic Nemesis: Illich reconsidered
News
June 14, 2024

Pandemic Nemesis: Illich reconsidered

Read Now
How ‘Dad Jokes’ Help Children Learn How To Handle Embarrassment
Insights
June 14, 2024

How ‘Dad Jokes’ Help Children Learn How To Handle Embarrassment

Read Now
How Social Science Can Hurt Those It Loves
Ethics
June 4, 2024

How Social Science Can Hurt Those It Loves

Read Now
Digital Scholarly Records are Facing New Risks
Research
May 21, 2024

Digital Scholarly Records are Facing New Risks

Read Now
Analyzing the Impact: Social Media and Mental Health 

Analyzing the Impact: Social Media and Mental Health 

The social and behavioral sciences supply evidence-based research that enables us to make sense of the shifting online landscape pertaining to mental health. We’ll explore three freely accessible articles (listed below) that give us a fuller picture on how TikTok, Instagram, Snapchat, and online forums affect mental health. 

Read Now
New Fellowship for Community-Led Development Research of Latin America and the Caribbean Now Open

New Fellowship for Community-Led Development Research of Latin America and the Caribbean Now Open

Thanks to a collaboration between the Inter-American Foundation (IAF) and the Social Science Research Council (SSRC), applications are now being accepted for […]

Read Now
Maybe You Can’t Buy Happinesss, But You Can Teach About It

Maybe You Can’t Buy Happinesss, But You Can Teach About It

When you deliver a university course that makes students happier, everybody wants to know what the secret is. What are your tips? […]

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments