Social Science Bites

Olivier Sibony on Decision-Making

May 17, 2021 6438

When human judgment enters the picture, so too will errors in human judgment. Think of this as “noise,” just as you might think of a signal-to-noise ratio in an audio signal. And just as in listening to music, this noise is not a feature, but a flaw.

In the context of human action, management professor at HEC Paris and former McKinsey senior partner Olivier Sibony defines “noise” as the unwanted variability in human judgment. “When you look at how people make a professional judgment, there is an average error … and that is what has traditionally been called bias in statistics and in the study of judgment. But when you have already identified bias, there is something left, and that is the unwanted variability of errors, that is noise.”

In a new book, Noise: A Flaw in Human Judgment, by psychologist and Bites alumnus Daniel Kahneman, Sibony, and Harvard law professor Cass Sunstein (author of Nudge), the trio look at the lottery that noise creates in social outcomes, and discuss ways to practice better “decision hygiene” to prevent noise from infecting important outcomes. Sibony spoke with interviewer David Edmonds in this Social Science Bites podcast about noise as a concept, the types of noise, why acknowledging it matters, and what we can do to avoid it. This is an area of great interest for Sibony, whose own research centers on reducing the impact of behavioral bias.

“Bias and noise,” Sibony explains, “are mathematically equivalent in the effect they have on error. Noise causes exactly as much error as bias does for the same quantity of noise or bias.

“And so, if you can reduce noise, you can reduce error.” Or put another way, make better decisions.

He gives the example of insurance claims adjusters. “When you look at how two of these people judge the same case, what price they set on the same insurance policy or the price they set on the same claim, and you ask them how much they expect to disagree, they say, ‘Of course we’re not going to be in perfect agreement; it’s a matter of judgment. But hey! We are all highly qualified, competent people, so we are more or less interchangeable depending on who is available.’”

If you ask the adjusters, or their bosses, about how much variability they expect, the answers come back around 10 percent. And if you ask business executives in general what they would expect the difference to be – and Sibony talked to hundreds — the answers came back at 10 to 15 percent.

In real life, he reveals, the differences vary by as much as 55 percent.

This isn’t just some peculiarity of insurance. “This was,” Sibony said, “something we found everywhere we looked!” He offers many examples: assessments by financial professionals, x-rays read by skilled doctors, professors grading essays, and many more. What he terms “big differences” appeared repeatedly

“More worrisome, perhaps, if you look at how judges sentence people who have been found guilty of a crime … [W]hen the average sentence is seven years in prison, the average difference, the mean difference between two judges is three-and-a-half.” And so, as Sibony notes, when appearing before two judges, you’ve already been sentenced to five years, or to nine years, “just based on the luck of the draw.”

This variability in outcomes, matters for several reasons: fairness (“when similarly located people are not treated similarly, it is unfair”); credibility of the underlying institutions; and because we’re routinely making bad (or at least not the best) decisions.

In addition to teaching strategy, decision making and problem solving at HEC Paris, Sibony is an associate fellow at Oxford University’s Saïd Business School. His book Vous Allez Commettre Une Terrible Erreur ! (published in English as You’re About to Make a Terrible Mistake!) was awarded the Manpower Foundation Grand Prize in 2019 for best management book. He is also a knight in the French Order of the Légion d’Honneur.

To download an MP3 of this podcast, right-click HERE and save.


For a complete listing of past Social Science Bites podcasts, click HERE. You can follow Bites on Twitter @socialscibites and David Edmonds @DavidEdmonds100.

Welcome to the blog for the Social Science Bites podcast: a series of interviews with leading social scientists. Each episode explores an aspect of our social world. You can access all audio and the transcripts from each interview here. Don’t forget to follow us on Twitter @socialscibites.

View all posts by Social Science Bites

Related Articles

R Sánchez-Rivera on the Hidden Legacy of Eugenics
Insights
August 5, 2025

R Sánchez-Rivera on the Hidden Legacy of Eugenics

Read Now
Ramanan Laxminarayan on Antibiotic Use
Public Policy
August 4, 2025

Ramanan Laxminarayan on Antibiotic Use

Read Now
My First Sage Book: The Gift of a Nocturnal Anthropologist
Insights
July 30, 2025

My First Sage Book: The Gift of a Nocturnal Anthropologist

Read Now
Why Men Have a Bigger Carbon Footprint Than Women  
Insights
July 8, 2025

Why Men Have a Bigger Carbon Footprint Than Women  

Read Now
Leor Zmigrod on the Ideological Brain

Leor Zmigrod on the Ideological Brain

Flexibility is a cardinal virtue in physical fitness, and according to political psychologist and neuroscientist Leor Zmigrod, it can be a cardinal […]

Read Now
When Clarity Isn’t Enough: Rethinking AI’s Role in Cognitive Accessibility for Expert Domains

When Clarity Isn’t Enough: Rethinking AI’s Role in Cognitive Accessibility for Expert Domains

The promise of artificial intelligence in accessibility work is often framed in hopeful terms. Large language models (LLMs) like GPT-4 are increasingly […]

Read Now
Degrading Sites of Punishment and Pain: The Case for Abolishing Prisons

Degrading Sites of Punishment and Pain: The Case for Abolishing Prisons

Prisons have been in crisis in England and Wales for 200 years. The state has responded with piecemeal, ‘pragmatic’ reforms which have […]

Read Now
5 2 votes
Article Rating
Subscribe
Notify of
guest


This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments