BROOKE GLADSTONE:: Daily our newspapers, our airwaves, our Internet news outlets are clogged with experts who make predictions. Omnipresent and presumably omniscient, they augur on demand on matters ranging from the war to Capitol Hill to Wall Street, kind of like the dessert after a heavy entrée of news. The McLaughlin Group, a Sunday morning talkfest, rewards viewers every week with the predictions of its four pundits.

MAN: Positions, Pat:

MAN: Your boy Netanyahu won't even run second.

WOMAN: Governor Schwarzenegger will commute the death sentence of Tookie Williams, founder -- co-founder of the Crips. [OVERTALK]

MAN: Congress won't pass a budget resolution.

MAN: Andrew Card will be the succeeding Treasury Secretary.

MAN: Ben Bernanke takes over from Alan Greenspan in February. He will ease back into this race, and that will lead to a stock market boom in 2006. Bye-bye!

BROOKE GLADSTONE:: But has anyone ever followed up on these professional prophets and calculated how often they're right? Well, one man has. University of California Berkeley professor Philip Tetlock has just published a new book with his findings, called Expert Political Judgment: How Good Is It? How Can We Know? Tetlock tracked 284 experts from politics, the academy, business and journalism for 20 years. He asked them to make predictions both in and outside their areas of expertise, and he tallied all their forecasts, both as probabilities and right or wrong guesses. By 2003, when the study concluded, he had analyzed more than 82,000 predictions. What he found out was that often, being an expert was the best predictor of being wrong.

PHILIP TETLOCK: When an expert has very, very strong opinions on an issue, when the expert places a high value on simplicity and has little patience with contradictions or ambiguity, and when the expert is making longer-term predictions, that expert's likely to go off the cliff.

BROOKE GLADSTONE:: [LAUGHS]

PHILIP TETLOCK: And the more knowledge that expert has, the worse, interestingly, it becomes, because the expert is using the knowledge very selectively to justify increasingly extreme predictions.

BROOKE GLADSTONE:: I guess in this case the experts do what the rest of us do when we consume media. They seek out information that confirms what they already believe.

PHILIP TETLOCK: Experts with that particular style of thinking are prone to that tendency, yes.

BOB GARFIELD:: Which brings us to the foxes and the hedgehogs. It's an old Isaiah Berlin metaphor that you use to divide your experts into two camps. What are foxes and hedgehogs and how do they differ in their forecasts?

PHILIP TETLOCK: Well, the fox/hedgehog distinction goes back at least 2,500 years to classical Greece. The basic idea is that the hedgehog personifies someone who knows one big thing, someone who tries to organize everything into a comprehensive overall framework, whereas the fox is someone who knows many little things, is much more intellectually opportunistic, is willing to pick ideas from wherever. It doesn't matter if it's a liberal idea or a conservative idea. They are very flexible.

BROOKE GLADSTONE:: You analyzed the political judgments of Winston Churchill in terms of this hedgehog/fox analogy and found that in some important situations, the distinction isn't just academic. What are some of the examples of how Churchill's being a hedgehog affected the real political landscape?

PHILIP TETLOCK: Well, Winston Churchill is, of course, best known for his remarkably far-sighted predictions about Nazi Germany that go back as far as 1933, when Hitler just came to power. He was on record very early as stating that Nazi Germany was a gangster state, that it would be essentially impossible for Britain to coexist with Hitler.

BROOKE GLADSTONE:: And the foxes felt differently.

PHILIP TETLOCK: Stanley Baldwin, Neville Chamberlain, those very sophisticated British politicians certainly disagreed, and so did most British intellectuals and political observers.

BROOKE GLADSTONE:: He also had a hedgehog view when it came to India.

PHILIP TETLOCK: Well, that's one of the interesting things about Winston Churchill. Of course, he's remembered for being spectacularly prescient about Nazi Germany, but we forget how far off he was on India. He was an adamant opponent of self-government from India very early on. He even once compared Gandhi to Hitler. They were similar, in his mental framework, because both leaders represented threats to the British Empire.

BROOKE GLADSTONE:: But, like Churchill, the hedgehogs aren't remembered very well for their wrong predictions.

PHILIP TETLOCK: By and large, that's right. It is true that if you wanted to identify the experts who have made the most spectacularly far-sighted predictions over the last 50 years, the hedgehogs would be disproportionately represented. But if you were computing batting averages, the hedgehogs would be clearly statistically inferior to the foxes.

BROOKE GLADSTONE:: [LAUGHS] And there's sort of a perverse inverse relationship between how spectacularly wrong some hedgehogs are and how much they're held responsible for those wrong predictions. The more wrong you are, the higher profile you are, the less you're called to account.

PHILIP TETLOCK: Well, that's an interesting feature of the political world. Hedgehogs are typically embedded in political movements or theoretical movements and they typically have people who will back them up. They can fall back on a base of supporters who will help them generate various types of excuses or belief system defenses that will neutralize the unexpected evidence. So they'll be able to argue, "Well, what I predicted didn't happen, but it will happen soon," or, "I predicted that country X had weapons of mass destruction, and, well, it appears that it didn't, but it was the right mistake to have made."

BROOKE GLADSTONE:: [LAUGHS]

PHILIP TETLOCK: It's better to have overestimated them than to have underestimated them.

BROOKE GLADSTONE:: You note in your study that hedgehogs are more frequently wrong than foxes, but you also note that when it comes to experts, the media love, not wisely, but too well, the hedgehog - I guess because they make better TV.

PHILIP TETLOCK: The hedgehogs tend to provide better sound bites. I think that's definitely true. They're much more likely to offer unequivocal predictions, whereas the foxes are much more likely to attach lots of linguist qualifiers to their predictions. Their speech is larded with things like, "but," "however," "although" - all signs that you're putting on the brakes. There's also a team sport aspect to punditry. Pundits are often there to represent a certain point of view so they want to make sure they've got an entertaining liberal hedgehog and an entertaining conservative hedgehog and pit them against each other and let the sparks fly, and so it has this public spectacle aspect to it. It's not particularly conducive to accuracy, but it is entertaining, and it really depends on what you want out of life.

BROOKE GLADSTONE:: And therein lies the problem, I guess, for the media. The entertainment value may be even more important than the accuracy, and yet your solution to the problem is to monitor all of these delightful hedgehogs and to rap the knuckles of the ones who are wrong. What exactly do you propose to deal with this problem?

PHILIP TETLOCK: Well, I think on balance, it would be a good idea to give some serious thought to systematically monitoring political punditry. I think we monitor professionals in many other spheres of life. I think we monitor weather forecasters, we increasingly monitor stock market analysts, we sometimes monitor doctors. I don't think it's unreasonable to suppose that when people offer opinions on extremely consequential issues, like whether or not to go to war or whether or not to have welfare reform, or tax policy, trade policy, it's not unreasonable to ask what are their predictive track records in the past as a guide for how much credibility to attach to what they're saying in the present.

BROOKE GLADSTONE:: It does seem a little more important than whether or not to take an umbrella.

PHILIP TETLOCK: Well, that's important too. [LAUGHTER]

BROOKE GLADSTONE:: Thank you very much.

PHILIP TETLOCK: Thank you.

BROOKE GLADSTONE:: Philip Tetlock is the Mitchell Professor of Leadership at the University of California at Berkeley, and author of Expert Political Judgment: How Good Is It? How Can We Know? [MUSIC UP AND UNDER]