Confidence is a preference of the habitual voyeur of what is known as…

Recently, I’ve started asking for people’s confidence levels (from0% to 100%) of project-related events happening. And although in itsearly stages, I’ve been disappointed by the results thus far,confidence generally being way higher than the reality.

RichardFeynman beautifully exposed the flawed methodology behind riskassessment at Nasa in his role on the commission to investigate the1986 Challenger disaster. Nasa failed to realise (admit?) that if there arelots of uncorrelated bits of the Shuttle each with a near 100%probability of surviving the mission, each of which is critical toavoid disaster, then the probability of the Shuttle returning safelyto Earth can fall unacceptably short of 100%.

The realisation of risks in projects I manage has a lesser impact. But an impact nonetheless. So I intend to keep alog of all the confidence estimates I receive (column B), together with theperson whose confidence is being shared (column A) and the binary outcome of theevent in which they have confidence (column C, 1 meaning the predicted event happened, 0 meaning it didn’t). I figure that if people’sconfidence levels are true reflections of reality, then the sum of column B will equal the sum of column C. And sumifs based on people’s names will identify the optimists, realists and pessimists.

=SUMIF(A:A,"John Smith",B:B)/SUMIF(A:A,"John Smith",C:C) will give me an optimism quotient for John. I can then divide any confidence percentage I receive from him by the quotient to get a more realistic view of whether the event will happen.

Comments

3 Responses to “Confidence is a preference of the habitual voyeur of what is known as…”

  1. Shanahan on July 1st, 2008 01:31

    “Buildings in the sky and the air is sugar free…and everyone’s so friendly”

  2. art vandelay on July 1st, 2008 23:37

    sounds good, but you’ll need a big enough sample to obtain the original optimism quotient. without that, your quotients will be unacceptably affected by chance, and will be meaningless.

  3. Terry on July 2nd, 2008 23:29

    Didn’t Tom DeMarco cover this in one of his books how iterative estimation will converge on a fair estimate instead of people only estimating once?

Leave a Reply