Prediction Markets Meet Wall Street

Besides helping us determine the next President or Hollywood blockbuster, crowd wisdom may also help predict flu epidemics or whether a bailout will work

by Ben Kunz

With all the fuss about Wall Street lately, one fact has gone unobserved: The stock market has performed brilliantly as a form of artificial intelligence. That's right. The Dow Jones can see the future, and last month it tried to warn you.

Consider what happened on Monday, Sept. 29. A $700 billion bailout package was on the table in Congress, and many constituents were opposed to it, complaining that it would help fat cats on Wall Street but not mainstream America. The House of Representatives, where members are up for re-election in November, stopped the bill.

That same day, the Dow Jones industrial average plunged almost 778 points, its biggest one-day percentage decline in two decades. A vast group of investors bet instantly that the political logjam was very bad for the economy. And as the global stock sell-off continued in the succeeding days, Americans began realizing that failure to do something to help the credit markets was not helping their personal fortunes.

A Pool of Intelligence

How did Wall Street know what would happen? It acted like a prediction market, a pool of intelligence that can foresee the future. Prediction markets are simply bets on ideas: What do you think something is worth, and more important, what will it be worth tomorrow? When groups of people bet on something, their combined intelligence is often remarkably prescient. British scientist Francis Galton was one of the first to notice this in 1906, when he observed 787 people at a country fair try to guess the weight of an ox after it was slaughtered. The average of all guesses was 1,197 pounds, only one pound less than the correct answer.

James Surowiecki expanded on Galton's observation in his 2004 book The Wisdom of Crowds. The most obvious example of collective intelligence is sports wagers, where the odds are really set by all people placing a bet, he wrote. Think of a horse race where the odds are 3 to 1 that one horse will win. If suddenly more people start betting on that horse, the odds rebalance in that horse's favor. The tendency of sports fans to collectively set the right odds is how bookies make money. If it works in sports, "wouldn't people betting on other kinds of events be equally as good, as a group, at predicting them?" Surowiecki asked.

Recently new types of markets have emerged to specifically predict the future. The Iowa Electronic Markets is one of the most famous prediction markets, set up to foretell U.S. Presidential elections. Thomas Rietz, a University of Iowa finance professor who helps direct the IEM, notes it has accurately foretold each election result since 1988 with only a 1.33% error rate in voter totals.

Games for Adults

How does it work? Traditional markets such as stock exchanges, Rietz says, are "affected by a whole range of factors." Prediction markets, on the other hand, try to "untangle those factors that cause change," he says. The IEM lets you bet yes-no on simple questions, such as will Senator John McCain win the election? If the odds are 50-50, you could buy a 50¢ contract and get a $1 payout if you win (while someone on the losing side of that bet is out 50¢). As bettors change opinions, the odds shift. McCain has had a rough patch recently; on Oct. 11 the IEM "winner take all" contract for McCain had fallen to 15.5¢—the market's way of saying he has 15.5% odds of becoming President.

Some prediction markets act as games for adults. The Hollywood Stock Exchange lets you place bets on whether movies will succeed at the box office. The Popular Science Predictions Exchange puts odds on whether almost any event will happen—say, whether someone will hack the iPhone within two months of its release. (Those who bet yes won the prize.)

Corporations, too, are realizing the value of collective intelligence. In 2006 Yahoo! (YHOO) asked employees to dream up new products, but instead of leaving final judgment to top brass, it created a prediction market that let employees bet on which idea would thrive in real-world competition. Microsoft (MSFT) and Google (GOOG) also have tapped the wisdom of employees (BusinessWeek.com, 8/3/06) with similar markets.

Public Health Implications

But now, prediction markets may have even wider applications, bearing relevance to public health and global warming, for example. The IEM is now experimenting with a private flu market that would enable health-care professionals from different states to bet on when flu will erupt in the U.S. and how severe the outbreak will be. Rietz says the flu market may solve a thorny problem for the U.S. Centers for Disease Control, which until now had to watch flu outbreaks in a rearview mirror, too late for hospitals to prepare. "Flu is actually spread in a predictable pattern," Rietz says. "Kids get it first, parents stay home with the kids, and local clinics and nurses see it happening," he explains. "If we can get their information aggregated, hospitals can have an early warning."

Academics now hope to move prediction markets beyond single future predictions to evaluation of many alternative, possible futures. To understand this, think of the basic problem of setting public policy. Will the recently approved $700 billion bailout really unlock credit markets? Or, how about these questions: Would taxing CO2 emissions stop rising sea levels? Which form of energy exploration—oil, coal, wind, solar, nuclear—is most likely to lead to U.S. energy independence? Which Presidential candidate, Barack Obama or McCain, is least likely to start a new war? Each of these questions requires an "if A, then B" prediction.

The idea of scenario forecasts using group intelligence was created by Robin Hanson of George Mason University, who has noted public policy often fails because it is created by an "insider's club of pundits and academics." The trouble with humans, it seems, is that even when we're smart, we have access to imperfect information and follow the groupthink of our peers. Because we often disagree with other groups, we band together and end up agreeing too much with our own teams. No single leader can overcome such biases and data gaps to predict with certainty whether an action will succeed or fail. But Hanson suggests markets can do just that.

Bubbles Aren't New

Certainly all markets have their flaws. Bubbles and busts have been around since the Dutch tulip craze of the 1630s, when single tulip bulbs traded at values 20 times greater than the annual income of most workers. Wall Street's recent mania over questionable mortgages caused today's current crisis, and the manic sell-off backlash was eventually driven by fear, not logic.

But if markets are watched carefully they reveal a vast artificial intelligence, based on the individual data each human provides as he or she makes a best guess about the future. In a way, we are all like bees, frantically collecting pollen and rarely looking up to note the overall hive activity. Occasionally, something like the Dow falling almost 778 points in a single day—or a 936-point rebound a few days later—makes us see the big picture.

Ben Kunz is director of strategic planning at Mediassociates, a media planning and internet strategy firm. He is author of the advertising strategy blog.