Why Experts Are Often Wrong

I have been perplexed by the fact that so many smart and talented people with high IQs and advanced academic degrees from prestigious universities – have made so many wrong predictions about the future, have offered so much ill-advise for public policy, and generally have been proven wrong almost all of the time.

For an answer, consider a study by Philip Tetlock, a political scientist and psychologist at the University of California, Berkeley, who in Expert Political Judgment: How Good Is It? How Can We Know? (Princeton University Press, 2005) contends that there is no direct correlation between the intelligence and knowledge of the political expert and the quality of his or her forecasts.

Mr. Tetlock is very skeptical of our ability to predict future developments, as the odd assortment of path-dependency theorists, complexity-chaos theorists (recall the Butterfly Effect?), game theorists, and probability theorists demonstrate that the fundamental complex properties of the world make it impossible to achieve forecasting accuracy beyond crude extrapolation.

At the same time, the fundamental properties of the human mind – preference for simplicity, aversion of ambiguity and dissonance, belief in controllable world, misunderstandings of probabilistic processes – make it inevitable that experts will miss whatever predictability has not been precluded "in principle."

Mr. Tetlock has tried to prove, by gathering and analyzing more than 80,000 forecasts by academics, journalists, consultants and professional "futurists" about a variety of global issues, including the collapse of the Soviet Union, the end of the apartheid system in South Africa, and the outcome of the first Persian Gulf War (he explains the methodology he used in a long appendix to the study; which is rather complex itself) that the experts were wrong more often than blind chance.

What Mr. Tetlock considers to be an asset is the style of thinking, as there seems to be a direct link between how people think and what they get right and wrong. He applies in his study the prototypes of the "hedgehog" and the "fox" that the late British political philosopher Isaiah Berlin had proposed as a way of classifying political thinking and behavior.

Mr. Tetlock demonstrates the usefulness of classifying experts along a rough cognitive-style continuum anchored at one hand by Berlin's prototypical hedgehog and at the other by his prototypical fox.

The intellectually aggressive hedgehog knew one Big Thing and sought, under the banner of parsimony, to expand the explanatory power of that Big Thing to "cover" new cases. He or she toils devotedly within one tradition and imposes formulaic solutions on ill-defined problems.

The more eclectic foxes knew many little things and were contend to improvise ad hoc solutions to keep pace with a rapidly changing world. Drawing from a variety and sometimes contradictory array of ideas and traditions, he or she is better able to improvise in response to changing events and is more successful in predicting the future than the hedgehog.