Superforecasting: The Art and Science of Prediction

One of the problems with forecasts is the ambiguity of their verbiage. Studies have shown that people differ in their interpretations of words like “significant” and “likely,” and often, those making the predictions do not delineate their definition. Many forecasts also lack concrete timescales. While forecasters might assume that a timescale is implied, failing to directly specify a timescale precludes measurement of the forecast’s accuracy.

After the disastrous 1961 Bay of Pigs invasion of Cuba by the Kennedy administration, history professor Sherman Kent devised a scale to address such discrepancies. Highly regarded in the intelligence community, Kent left Yale in 1941 to join the newly created Coordinator of Information agency (a precursor of the Central Intelligence Agency, or CIA). Kent argued that people needed to quantitatively define such terms such as “serious possibility,” and in 1964, he published a scale where 100% was certainty and 0% was impossibility (55). Other values, such as 75% (give or take about 12%), meant probable. While Kent’s scale aimed to reduce ambiguity, people were reluctant to adopt it, preferring their familiar linguistic methods to the numerical ones that made them feel bookish. Tetlock and Gardner argue that such numerical grading is useful; when forecasters are forced to break down their thinking processes, they become better at identifying more minute degrees of uncertainty, “just as artists get better at distinguishing subtler shades of gray” (57).