Wilson Quarterly 30(2):7,9, Spring 2006.

Letter by Robin Hanson

"Reading the Future" featured thoughtful and piquant articles by intelligent, knowledgeable, and articulate authors. Still, one cannot help noting that the authors who specialize in future studies were careful not to make any predictions, while the others, knowing that a methodological review of their accuracy is unlikely, made dramatic, thought-provoking predictions that center on current concerns.

Why? Because those who have reason to think their judgments will be assessed for accuracy in the future have an incentive to be careful. Perhaps one day we will have more-systematic surveys of experts' predictive accuracy, such as psychologist Philip Tetlock undertook for his 2005 book Expert Political Judgment: How Good Is It? How Can We Know? Until then, I think our best hope comes from betting markets. Let's legalize betting on predictions and require pronosticators to place substantial bets on their own guesses. Prognosticators would have better incentives to strive for accuracy with their money on the line and with the knowledge that their track records would be followed.

Even better, once betting markets had odds to track, we could ignore who said what when and just take the odds established by all bettors as our best estimate. Not only are such estimates numberically precise and continuously updated, but they can directly reflect insights from millions of people. Betting markets emphasize the more accurate sources, whoever they are. And that is not just abstract theory; in all field comparisons made so far, such markets have been at least as accurate as competing institutions.

Robin Hanson
Associate Professor of Economics
George Mason University
Fairfax, VA

Above is the letter as it appeared. Below is the letter text I submitted to them.

When people say a description of the future “sounds like science fiction,” they are usually not praising its realism. They instead allude to a science-fiction-like temptation to tell vivid, dramatic, and sensational but unrealistic stories.

Reading the recent Wilson Quarterly issue on “Reading the Future” I find thoughtful and though-provoking articles by intelligent, knowledgeable, and articulate authors. But I have similar worries. The authors who specialize in future studies were careful not to make any predictions. And the others knew there is little chance of a systematic review of their accuracy; the game appears to be to make dramatic thought-provoking predictions that illuminate current concerns. So while I found their predictions interesting, I could not really believe them.

Who could I believe? I could better believe authors who expected to have their prediction track records checked for accuracy, especially if big consequences followed, or if authors were selected for expected high accuracy on interesting topics. Perhaps one day we will have more systematic surveys of accuracy, such as in Philip Tetlock’s recent book Expert Political Judgment.

But until that day, I think our best hope comes from betting markets. Let us make it legal to bet on these future predictions, and let us expect prognosticators to offer substantial bets on their predictions. Prognosticators would have better incentives, both because money would be on the line, and because track records can be compiled from settled bets.

Even better, once betting markets had odds to track, we could ignore who said what when and just take those odds as our best estimate. Not only are such estimates numerically precise and continuously updated, but they can directly reflect insights from millions who are not articulate enough to write for the Wilson Quarterly.

Not that I’m claiming that the inarticulate crowd is usually more accurate than articulate intellectuals. No, the point is that betting markets will emphasize the more accurate sources, whoever they are. And that is not just abstract theory; in all the field comparisons made so far, such markets have been at least as accurate as competing institutions.

Now it is true that prognosticators expected to offer substantial bets on their predictions would probably work harder to be clear about what exactly their predictions meant. But this would not be a cost required by betting markets; one could always bet on what some panel of judges would say about predictions so many years later. Instead, this would be the real cost of improved accuracy.

In other contexts, such as the furor over the Policy Analysis Market a few years back, people have been worried about manipulation, i.e., people losing bets on purpose to make market prices less accurate, or doing bad things in order to win bets. It turns out, however, that one cannot really reduce average price accuracy by losing bets. And the amount of money in such betting markets on the future would be negligible compared to other markets that one can influence by doing bad things.