Stewart Brand and the Long Now Foundation hold monthly invited talks.
Last evening's talk is summarized below. Of some relevent interest to
this group.
Richard
---------- Forwarded message ----------
Date: Sat, 27 Jan 2007 12:33:32 -0800
From: Stewart Brand <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Subject: [SALT] Ignore confident forecasters (Philip Tetlock talk)
"What is it about politics that makes people so dumb?"
From his perspective as a pyschology researcher, Philip Tetlock
watched political advisors on the left and the right make bizarre
rationalizations about their wrong predictions at the time of the
rise of Gorbachev in the 1980s and the eventual collapse of the
Soviet Union. (Liberals were sure that Reagan was a dangerous idiot;
conservatives were sure that the USSR was permanent.) The whole
exercise struck Tetlock as what used to be called an
"outcome-irrelevant learning structure." No feedback, no correction.
He observes the same thing is going on with expert opinion about the
Iraq War. Instead of saying, "I evidently had the wrong theory," the
experts declare, "It almost went my way," or "It was the right
mistake to make under the circumstances," or "I'll be proved right
later," or "The evilness of the enemy is still the main event here."
Tetlock's summary: "Partisans across the opinion spectrum are
vulnerable to occasional bouts of ideologically induced insanity."
He determined to figure out a way to keep score on expert political
forecasts, even though it is a notoriously subjective domain
(compared to, say, medical advice), and "there are no control groups
in history."
So Tetlock took advantage of getting tenure to start a long-term
research project now 18 years old to examine in detail the outcomes
of expert political forecasts about international affairs. He
studied the aggregate accuracy of 284 experts making 28,000
forecasts, looking for pattern in their comparative success rates.
Most of the findings were negative--- conservatives did no better or
worse than liberals; optimists did no better or worse than
pessimists. Only one pattern emerged consistently.
"How you think matters more than what you think."
It's a matter of judgement style, first expressed by the ancient
Greek warrior poet Archilochus: "The fox knows many things; the
hedgehog one great thing." The idea was later expanded by essayist
Isaiah Berlin. In Tetlock's interpretation, Hedgehogs have one grand
theory (Marxist, Libertarian, whatever) which they are happy to
extend into many domains, relishing its parsimony, and expressing
their views with great confidence. Foxes, on the other hand are
skeptical about grand theories, diffident in their forecasts, and
ready to adjust their ideas based on actual events.
The aggregate success rate of Foxes is significantly greater, Tetlock
found, especially in short-term forecasts. And Hedgehogs routinely
fare worse than Foxes, especially in long-term forecasts. They even
fare worse than normal attention-paying dilletantes--- apparently
blinded by their extensive expertise and beautiful theory.
Furthermore, Foxes win not only in the accuracy of their predictions
but also the accuracy of the likelihood they assign to their
predictions--- in this they are closer to the admirable discipline of
weather forecasters.
The value of Hedgehogs is that they occasionally get right the
farthest-out predictions--- civil war in Yugoslavia, Saddam's
invasion of Kuwait, the collapse of the Internet Bubble. But that
comes at the cost of a great many wrong far-out predictions--- Dow
36,000, global depression, nuclear attack by developing nations.
Hedgehogs annoy only their political opposition, while Foxes annoy
across the political spectrum, in part because the smartest Foxes
cherry-pick idea fragments from the whole array of Hedgehogs.
Bottom line... The political expert who bores you with an cloud of
"howevers" is probably right about what's going to happen. The
charismatic expert who exudes confidence and has a great story to
tell is probably wrong.
And to improve the quality of your own predictions, keep brutally
honest score. Enjoy being wrong, admitting to it and learning from
it, as much as you enjoy being right.
--Stewart Brand
(Iraq footnote. I asked Tetlock to opine on which experts were most
right about how things have gone in the Iraq War. He said the most
accurate in this case were the regional experts, who opposed the
invasion, and what they are predicting now is a partition of Iraq
into Kurdish, Shia, and Sunni areas.)
--
Stewart Brand -- [EMAIL PROTECTED]
The Long Now Foundation - http://www.longnow.org
Seminars & downloads: http://www.longnow.org/projects/seminars/
============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
lectures, archives, unsubscribe, maps at http://www.friam.org
