Correcting for bias when estimating from evidence
Intuition patch
1. Base rate
2. Your guess
3. Est. correlation b/w evidence & outcome
4. Deviate #1 -> #2 proportional to #3.
Kahneman, TFAS
1. Base rate
2. Your guess
3. Est. correlation b/w evidence & outcome
4. Deviate #1 -> #2 proportional to #3.
Kahneman, TFAS
This is a tweet-packed nugget of wisdom from Thinking, Fast and Slow.
I love this book because it not only talks about what the evidence shows about our cognitive biases, but also what it says about how to correct for them, and how effective this correction can be.
What this^ bit is saying is that our guesses are largely based on substitution and intensity matching, which are great heuristics, but systematically biased to ignore regression towards the mean. That is, as he says frequently, we tend to assume that What You See Is All There Is (WYSIATI), when there's actually a lot of hidden factors that can influence the outcome other than the evidence we've seen.
So, when asked to estimate the college GPA of a child who could read by the age of 4, we first do substitution ("estimate future GPA" -> "estimate precocity"), then intensity matching ("quite precocious" -> ~3.7 GPA), and stop there.
Kahneman suggests a way to approximate the outcome of an actual statistical analysis by adding in two more things: The base rate (in this case average) GPA of any student, without any extra information about them, and our estimate of the correlation between precocity and GPA. You start by assuming the student is merely average, and then you walk in the direction of your intuitive number you got (~3.7) a distance proportional to how correlated you think GPA is to childhood precocity.