Obviously well-written from obviously accomplished authors and researchers. Overall, the book could've been much more concise.

Last night I had the pleasure of listening to William Tipper, Associate Editor of Books for The Wall Street Journal interview Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein regarding their new book, NOISE. These authors' credentials in terms of critical thinking, strategic planning, and decision-making are impressive: Kahneman (a Nobel Prize winner) also wrote Thinking Fast and Slow, while Sunstein co-authored Nudge and all three are professors at well-regarded institutions (Princeton, HEC Paris and Oxford, and Harvard). They subtitled their book "A Flaw in Human Judgment" and focus on how "noise" (random scatter or variability) along with bias (systematic deviation) each contribute error as humans make decisions. Citing numerous examples from diverse fields like the justice system (think about juries or sentencing), medicine (diagnoses, etc.), and the business world (hiring decisions, underwriting, and investment choices, for example), they explain the problem, including our resistance to believing how vulnerable we are to noise. They also devote considerable attention to suggestions for institutional design or "decision hygiene" such as aggregating independent views and consulting experts ("wisdom of crowds"); establishing guidelines (as is done for judges); adopting processes to structure decisions by breaking them down and keeping components separate; and delaying intuition and snap decisions. The book's appendix provides instructions on how to conduct a noise audit, a checklist for a decision observer, and information on correcting predictions. Extensive notes are also supplied. Given the disruption of the last year or so, this seems an especially valuable text worth consulting.

Noise by Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein is a non-fiction title that focuses on how noise and bias create error and impact our ability to make appropriate decisions in daily life. The book highlights the differences between noise and bias and how they differ in their effect on daily circumstances. This book also discusses how human error also impacts the circumstances and how that manifests. Finally, the book shares practical tips on how to minimize the impact of those variables to improve our ability to make decisions.

I found the premise of noise very compelling as someone who works with data daily. I found it interesting how the authors point to our current focus on bias and how we currently overlook the role of noise. Even as someone who works with data and can easily recognize bias vs noise, it was interesting to see how it plays out in real life rather than just numbers on a page. I found it fascinating that humans decisions are so flawed that even random models can produce better outcomes which is literally just chance! This book has made me reconsider some of my own assumptions and has strengthened my belief that many areas, including medical care, would benefit from greater implementation of technology and statistics to guide decision making as humans are so susceptible to external factors.

In addition, what really was interesting is that the effect of bias and noise is significant when it is an individual but is also amplified when in groups. This is not surprising based on theories like groupthink and the bystander effect, but it highlights how much we can get in echo chambers and go with the group rather than what the data tells us. The most valuable information I gained from this book is the importance of "decision hygiene" which focuses on preventing error beforehand, rather than during or after the situation. Overall I recommend this book for those looking to make more informed decisions in their daily life and to be more aware of the "data" around them.

Many thanks to the publisher Little, Brown Spark and Netgalley for the ARC in return for an honest review.

Would we all be better off if we got rid of human judges and used algorithms to make decisions? Most of us would say “No,” but this book might make some of us change our minds. There are many examples in the book, but let’s look at doctors. There is some evidence that entering symptoms, medical history, etc into an algorithm would give more consistently good diagnoses than human doctors provide. Why? The book gives many reasons but a couple are that doctors are more likely to order follow up diagnostic tests in the morning than in the afternoon, and that some doctors make an initial diagnosis within a few seconds and then are very reluctant to change that diagnosis, regardless of subsequent evidence. Algorithms don’t do either of these things, which are examples of “noise.”
This is one small facet of this eye-opening book. I would definitely recommend it. My one complaint is that the last half of the book was very heavy on business examples. “thinking Fast and Slow” seemed more generally social science-y and I did enjoy it more.

The sheer variety of ways judgment can be clouded is mind-boggling. The more closely we examine judgments, the more noise turns up as a factor. In Noise, an A-list team of celebrity psych stars, Daniel Kahneman, Olivier Sibony and Cass Sunstein pull together their confrères and evidence from the usual innumerable studies to delineate how bad it really is.

Noise, at least in psychology, is “unwanted variability”. In practical terms, that means even the most focused person might be swayed by unnoticed noise. Noise can be the home team losing the night before, lunch coming up in half an hour, miserable weather, a toothache – pretty much anything that has nothing to do with the issue at hand. This is all in addition to personal prejudices and the framework of bureaucratic rules that are always in play, restricting the range of possible decisions, and misdirecting them where they should not be going.

All kinds of studies show that trial judges are inconsistent when not totally wrong. The authors say two judges viewing the same evidence in the same case will come to two completely different decisions. So will the same judge given the same case on two different occasions. Sentencing is all over the place, which has led to enforced sentencing guidelines that often make things worse. It has also led to judge-shopping, as the decision patterns of judges builds up over the years. This is not based on evidence or argument, but in which way the judge’s decisions can be erroneous. Think political parties, religion, and stubborn pig-headedness.

The same goes for mere mortals, like supervisors. They all believe they do a creditable job, but the stats show the direct opposite. Even simple linear models do a far better job in every case. Not just sometimes – every time, according to Noise. Even randomly generated models do a far more accurate job of judging people correctly than people do. Artificial intelligence algorithms can also add a little more accuracy, though surprisingly, not significantly so. But people on their own perform miserably.

Still, no one, but no one, would trust a simple model to make a decision on their future; they feel better having personally tried with another human, regardless of the facts. It immediately reminded me of Lake Wobegon, where all the kids are above average. Doesn’t work like that. In the authors’ words, “Models of reality, of a judge or randomly generated models all perform better than nuanced, intuitive, insightful and experienced humans.” To which I would add: anyone who claims they can accurately size up a person on meeting them, can’t.

Errors occur far more frequently than people realize, because everyone trusts their own judgment foremost, and far too often, the judgment of others (their lawyers, doctors and managers, for example).

The worst example of this occurs in job interviews and performance appraisals. Everyone knows the single worst way to make a hire is through a personal, unstructured interview. Yet managers still insist on interviews, and so do candidates, thinking they can master the battle and win the job if they can simply deal with someone in person. Both are totally wrong, yet nonetheless, they both persist. Job interviews have become a nightmare for candidates, going back multiple times for essentially no good reason, as the more people interview them, the more inaccurate their ultimate decision will be.

As for quarterly, semi-annual and annual performance appraisals, those who have to work with the results know they are usually totally worthless. Managers burdened with multiple reports grind them out against a deadline, having little or nothing to do with an individual’s performance. Most everyone is “satisfactory”, especially when managers are required to rate them on a scale. No decisions can validly be taken from these exercises in frustration, but they are taken anyway. And while essentially no one in any organization likes or ever looks forward to the whole process, the noise persists, clouding futures.

Scales themselves are useless, as the authors show in examples such as for astronauts. A bell-curve distribution would show one or two excellent performers, one or two total failures, and most in the middle. But there are no total failures among astronauts. The yearslong training requires and ensures it. So grading on a scale against a bell-curve can be just more noise.

For the open-minded, Noise provides details, tips and tricks to leverage. For example, deliberation, the vaunted value of teams, actually increases the noise around a decision. The mere fact that team members discuss their reasoning before they make a decision increases the noise for everyone participating. The key to making teams work, ironically, is for everyone to do their own research in isolation, and once they have all come to a decision, they can then compare with others on the team.

They call this independent work “decision hygiene”. It cuts down noise in general, but no one can know what specifically, or by how much. The authors liken it to handwashing- no one knows what germs were there to kill. All they know is that handwashing kills germs, and that you can never get rid of all of them.

The authors show that noise occurs in almost any shape or form. The quality of the paper used for a business plan, and the font it is presented in, can tip the success or failure of a proposal in the hands of potential investors.

Another interesting noise source is called whitecoat syndrome. This is noise some people generate going to see a doctor, nurse or lab technician. Their blood pressure rises in anticipation, sometimes causing an erroneous diagnosis.

Things like prejudice are not so much noise as bias. When assessing decisions that go wrong, noise is the standard deviation of errors, while bias is the mean itself. The book is a thorough attempt to make a science of noise and errors in judgment.

Bias is a likely driver of noise. But the book is all about separating the two. It shows that biases, such as “planning fallacy, loss aversion, overconfidence, the endowment effect, status quo bias, excessive discounting of the future, and various biases against various categories of people” are all factors in erroneous decisions. But despite all this, sheer noise outweighs bias heavily.

They use Gaussian mean squared errors to demonstrate the effect of both bias and noise, with noise the clear winner, and dramatically so. Squaring the errors makes them visually arresting, But they still need to be stopped - somehow.

It transpires that errors do not cancel each other out, either. Instead, they add up, taking decisionmakers farther away from the right decision. And with the book piling on a seemingly infinite selection of noise factors and sources, it’s a wonder Man has made it even this far.

Speaking of erroneous judgments, it is difficult to decide what kind of book Noise is. It is steeped in psychology, but it is not a groundbreaking new discipline. People and firms have been actively trying to filter out noise since forever (the better ones, anyway). Nor is it a psych textbook, really, though there are exercises the reader can use right while poring over it. I think it is closer to a handbook of what to be aware of: forewarned is forearmed sort of thing. Though clearly, mere knowledge of the situation is far from enough to counteract it. The book includes how-tos like implementing an audit to identify and isolate noise, so the book definitely has practical applications. Handbook it is, then.

This noise thing is ego-deflating for all humans, who run their lives continually making decisions, not only on facts, but predictive judgments as well (Predictions provide an “illusion of validity”). That we are not equipped to pull this off successfully – at all – should cause a total rethink of where we go from here. Noise is pernicious. Trusting models looms heavily over us all.

David Wineberg

Everything I've ever read by Daniel Kahneman has been thoroughly researched, perfectly accessible, and incredibly engaging. So many practical aspects come out of Noise; I can use this in my classroom to guide students in decision-making.

Insightful analysis of why we make bad judgments
Noise is bad no matter where in life we find it. In their new book Daniel Kahneman, Olivier Sibony, and Cass Sunstein say there is too much of it in our judgments and explain how noise arises and what might be done about it.
“Judgment” is not “thinking”.The book defines “judgment” as “a form of measurement in which the instrument is a human mind.” Judgments may be less than optimal due to bias, which is systematic deviation from optimal, e.g.the group’s predictions are ALWAYS overly optimistic, or noise, which is a more random scatter. The main topic of the book is “system noise”, which is “unwanted variability in judgments that should ideally be identical.” (I should get the same jail sentence no matter which judge hears my case.) System noise has two main components. There is level noise ( A particular judge is lenient in granting bail.) and pattern noise. Pattern noise also has two components: stable pattern noise, (Such as a tendency to give women lighter jail terms), and occasion noise ( I just had a run-in with my boss)..
The book discusses each of these types of noise and their psychological aspects, drawing on earlier work such as Sunstein’s “nudge” and Kahneman’s “System 1 and 2” thinking. Readers who are not somewhat familiar with this work might find a quick google search helpful.
So once you know sources of noise in judgment, what do you do about it? The authors describe some remedies, such as a “noise audit” or a “decision observer” to help remove bias from judgments in groups or a judicious use of rules or standards.
There is a lot of good and thought-provoking insight in Noise, principles that everyone will recognize once they are pointed out but that interfere with good judgment unless we identify and address them. The authors show how to do this with extensive descriptions of judgments in a number of fields, like selecting new hires, setting bail or sentences in criminal cases, and medical decisions. As a result, this is rather a long book, and these descriptions can be skimmed if you are very focused on task, but they are interesting.
The applications described in this book are primarily decisions made by multiple people, whether they be judges setting bail or group recommendations on whether a company should acquire another company. It does not focus much on decisions people might make in their personal lives, but the principles certainly seem applicable there as well. I am sure the authors would recommend that I not review this book just before lunch and after an argument with my spouse!