Lecture 5: Overconfidence
Lecturer: Andrew Grant
Introduction
› Overconfidence: Belief in one’s ability that is not justified by actual skill
› How do we identify overconfidence?
- Miscalibration in judgemental intervals
- Better-than-average effect
› Miscalibration can manifest itself in estimates of quantities that could potentially be discovered (e.g. the length of the Nile River)
› Or in estimates of not-yet-known quantities (e.g. the future price of a stock or the future value of a stock index).
› The following fractile method may be used to measure the degree of miscalibration in interval estimates:
- Please give the following estimates for your prediction.
- The true answer to the questions (e.g. a question about the length of the river Nilel or a question on the value of the Dow Jones Euro Stoxx 50 in one week) should
- Lower Bound: with a high probability (95 percent) of not falling short of the lower bound - Upper Bound: with a high probability (95 percent ) of no exceeding the upper bound.
Basic Facets
› Studies that analyse such assessments of uncertain quantities using this fractile method usually find that people’s probability distributions are too tight. › For example, studies that ask people to state a 90 percent confidence interval for several uncertain quantities
- (such as the above interval for the length of the river Nile)
› find that the percentage of surprises
- (i.e. the percentage of true values that fall outside the confidence interval)
› are higher than 10 percent, which is the percentage of surprises of an unbiased person in the context of estimating a 95 percent upper and lower bounds. 3
Basic Facets
› Such estimates of the quantiles of probability distributions are often elicited for uncertain continuous quantities, usually general knowledge questions. - Hit rates in many studies using 90 percent confidence intervals are less than 50 percent, leading to surprise rates of 50 percent or higher