Debiasing Decisions: Combat Confirmation Bias, Overconfidence Bias

Philip Meißner

Philip Meißner

Cognitive and behavioral biases can contribute to “blind spots” in decision-making, leading to less effective outcomes.
To improve decision outcomes, University of Marburg ’s Philip Meißner, Torsten Wulf of HHL Leipzig Graduate School of Management and HEC’s Olivier Sibony proposed a systematic checklist to identify potential decision derailment based on bias, along with rapid remedies.

Torsten Wulf

Torsten Wulf

They argues that two types of bias contribute to most decisions that lead to undesirable results:

  • Confirmation bias, the unconscious tendency to believe new information that is consistent with existing beliefs and recent experiences, and to discount contradictory data,
  • Overconfidence bias, the out-of-awareness likelihood to overestimate one’s skills, insights, and judgment.
    This leads to increased risk-taking based on illusory sureness of the decision and ability to mitigate adverse outcomes.
Olivier Sibony

Olivier Sibony

Previously, Lovallo and Sibony articulated four related decision biases:

  • Pattern-recognition biases, countered by changing the “angle of vision,”
  • Action-oriented biases, mitigated by recognizing uncertainty,
  • Interest biases, minimized by explicitly highlighting these interests,
  • Social biases, reduced by depersonalizing debate.

Debiasing techniques such as checklists, can limit the negative effects of biases in decision-making by offering a disciplined, comprehensive analysis of downside risks and by systematically considering multiple viewpoints.

Atul Gawande

Atul Gawande

However, effectively implementing checklists requires consistent discipline, noted Harvard’s Atul Gawande, who cited examples of partial adherence leading to costly oversights and failures.

One approach, suggested by Princeton’s Daniel Kahneman and Gary Klein of McKinsey, is a “premortem.”
Decision makers imagine that the decision has failed and analyze sources and reasons for adverse outcomes, to more thoroughly assess points of failure and possible mitigation strategies.
Formal scenario-planning is another way to expose assumptions underlying a plan, as well as a competitor’s priorities and potential strategy.

Massimo Garbuio

Massimo Garbuio

Using a variety of debiasing techniques significantly increased the Return on Investment (ROI) in a study by University of Sydney’s Massimo Garbuio and Dan Lovallo and Olivier Sibony of HEC.
As a result, Michael Birshan, Ishaan Nangia, and Felix Wenger of McKinsey, argued that debiasing techniques should be embedded in formal organizational decision-making processes, particularly for high-impact, repetitive decisions.

Michael Birshan

Michael Birshan

Decision biases may be out of awareness, or unconscious, so it’s more effective to evaluate the process of developing a proposal, rather than focusing only on the content and merits of a proposal.

Decision-making safeguards can be built into standard analysis processes by including questions to expose:

  • Multiple data sources,
  • Diverse opinions and perspectives,
  • Downside risk,
  • Potential negative outcomes for company, industry, and broader ecosystem.
Daniel Kahneman

Daniel Kahneman

Proposals are considered ready for a decision only when multiple perspectives are available to mitigate confirmation bias and risk analysis is available to reduce overconfidence bias.
Responses to decision checklist questions can be quantified to indicate one of four action steps, according to Daniel Kahneman:

  • Decide, based on inclusion of robust safeguards against both confirmation bias and overconfidence bias,
  • Screening MatrixReach out, suggesting the need for gathering additional perspectives, opinions, and perspectives to prevent narrow assumptions to reduce confirmation bias.
    The Vanishing-Options Test, proposed by Stanford’s Chip Heath and Dan Heath of Duke University, can generate new ideas by imagining that none of the current proposals are available.
  • Stress-test, by conducting a pre-mortem or analysis by external devil’s advocate or provocateur to reduce overconfidence risk by.
  • Reconsider when both more perspectives and risk analysis are required to reduce both overconfidence bias and confirmation bias.
    This screening matrix helps reduce related decision-making biases:
  1. Self-interest Bias
    -To what extent is the proposal motivated by self-interest?
Ishaan Nangia

Ishaan Nangia

-Assess for over-optimism

  1. Affect Heuristic
    -How strong is the team’s emotional attachment to a specific proposal?
    -To what extent were risks and costs fully considered for both preferred and non-preferred options?

-Assess for strongly-preferred outcomes
-Reintroduce analysis of all options

  1. Groupthink
    -How many dissenting opinions were analyzed?
    -How adequately were all options explored?
    -Was dissent discouraged? 
Felix Wenger

Felix Wenger

-Encourage substantive disagreements as a valuable part of the decision process
-Solicit dissenting views from members of the recommending team, through private meetings

4. Saliency Bias
     -To what extent are decisions made based on a potentially incomparable, but memorable success?
     -What about the proposed analogy is comparable to the current situation?
     -What are relevant examples from less successful companies? What happened in those cases?

Decision Making QuestionsRecommendation
-Carefully scrutinize analogies’ similarity to the current decision situation
Solicit additional analogies using reference class forecasting:

.Select reference class,
.Assess distribution of outcomes,
.Intuitively estimate project’s position in distribution,
.Assess estimate’s reliability,
.Correct intuitive estimate.

  1. Confirmation Bias
    -What viable alternatives were included with the preferred recommendation?
    -At what stage in the decision analysis were alternatives discarded?
    -What efforts were undertaken to seek information to disconfirm the main assumptions and hypotheses?

-Request two additional alternatives to the main recommendation, including analysis of benefits and drawbacks
-Acknowledge unknowns, risks

  1. Availability Bias

    Max Bazerman

    Max Bazerman

    If you had more time to gather date, what information would you seek?, asked Harvard’s Max Bazerman
    -How can you access similar data now?

-Use checklists to ensure comprehensive analysis of data required for each decision type

  1. Anchoring Bias
    -What data sources are used to analyze decision?
    -Which data are estimates? By whom? If so, from which data were estimates extrapolated?
    -To what extent could there be:
  • Unsubstantiated numbers?
  • Extrapolation from non-equivalent previous situations?
  • Attraction to specific anchors?

-Present data from other sources, benchmarks, or models
-Request new analysis

8. Halo Effect
     -To what extent does the analysis team expect that a person, organization, or approach previously successful in one context will be equally effective in different situation?

Phil Rosenzweig

Phil Rosenzweig

-Question potentially inaccurate inferences
-Solicit additional comparable examples
-Question attributions of success and failure to leaders’ personalities instead of chance factors, advised IMD’s Phil Rosenzweig.

9. Sunk-Cost Fallacy, Endowment Effect
     -To what extent are recommenders attached to past decisions?

Disregard past expenditures when considering future costs and revenues

  1. Overconfidence, Planning Fallacy, Optimistic Biases, Competitor Neglect
    -To what extent is the comparison case unwarrantedly optimistic?

-Adopt an outside view by using relevant simulations or war games

  1. Disaster Neglect
    -To what extent is the worst case scenario realistically and sufficiently negative?
    -How was the worst case generated?
    -To what extent does the worst case consider competitors’ likely responses?
    -What other scenarios could occur?

-Conduct a premortem, suggested by Gary Klein of Applied Research Associates:  Imagine the worst case scenario occurred, then propose likely causes, mitigations   

  1. Loss Aversion
    -To what extent is the evaluation and decision team risk averse?

-Realign incentives to share responsibility for the risk or to reduce risk

  1. Planning Fallacy focuses only on the current case while ignoring similar projects’ history and statistical generalization from related cases.
    -To what extent does the analysis rely on “top-down, outside-view” comparisons to similar projects?
    -Did the evaluators use a “bottom-up, inside-view” to estimate time required for each step?

-Statistically analyze a broad range of similar cases to avoid over-estimates from “top-down, outside-view” approaches and underestimates from “bottom-up, inside-view”
-Differentiate accurate forecasts from ambitious targets

  1. Loss aversion
    -To what extent are evaluators more concerned with avoiding loss than achieving gains?
    – How concerned are evaluators with being held responsible for a failed project?
    -To what extent has the organization specified acceptable risk levels?

-Seek risk tolerance guidelines from organizational leaders.

Decision-making tools like checklists can significantly reduce unconscious biases, provided that they are consistently and systematically applied.

-*What strategies have you found most helpful in reducing biases in decision-making?

Follow-share-like and @kathrynwelds

Related Posts:

Twitter:  @kathrynwelds
Google+LinkedIn Groups Psychology in Human Resources (Organisational Psychology)
Blog:  Kathryn Welds | Curated Research and Commentary

©Kathryn Welds



1 thought on “Debiasing Decisions: Combat Confirmation Bias, Overconfidence Bias

  1. kathrynwelds Post author

    Gary W. Kelly wrote:

    Excellent contribution.

    One small addition that may be useful is to state that when the input of multiple people is required to reach a decision, that the input be rendered independently, and in writing. Each person of a department or unit should provide their own list as to what to do. This leverages the wisdom of crowds, and helps to eliminate herd behavior.

    The importance of this technique arises from the fact that many of the list items are subjective. Many decisions have to be made more on subjective assessments than objective data, and often that objective data is interpreted, and that interpretation can be subject to debate. The debate is too often ended when the subordinates understand the preference of the manager. By having them arrive at their assessments independently, with justifications and arguments based on their perspectives, the dominant perspective can be obtained without the bias of wanting to defer to a perceived authority–which is not necessarily even the manager or actual authority in terms of the organization.

    Jane may be the one who is the perceived expert in a specific area, and she is confident, has her facts well organized, and articulates her position well. It is not surprising to see others nodding in agreement when she presents an assessment. They then dismiss their own conceptualizations as likely not relevant, or less significant. Jane carries the vote based on her perceived leadership more than the weight of her evidence and reasoning.

    By having the other team members render opinions separately from that of Jane, the total group knowledge and perspectives are preserved, and they may well contain important elements that alter the plan as argued by Jane. Herd behavior is mitigated.

    Kathryn Welds replied:

    Thanks, Gary, for your reminder that multiple, independent perspectives are crucial to reducing bias based on subjective assessment and perceived thought leadership by “high status” individuals in groups.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s