More corporations are setting up their own markets for economic forecasts, hoping to tap into the wisdom of employees
A little over a month ago, a group of Yahoo! executives faced an enviable challenge. More than 200 employees had just completed the company's annual Hack Day, where small teams have 24 hours to create, refine, and then pitch new products or services that they hope Yahoo will bring to market. Judges picked their favorites, but the company needed to decide which ones merited further development.
In years past, that decision was left to Yahoo (YHOO
) executives. But this time, the company wanted help from the rank and file. So it doled out wads of play money and encouraged employees to place bets on which products would fly with the public. It used a system customized by Yahoo principal research scientist David Pennock, one that works a lot like a stock market. Yahoo used the data to help identify which ideas could be made into more complete prototypes. "As a product organization, you hope to remove the burden of predicting from one product manager," says Prabhakar Raghavan, head of Yahoo Research.
Yahoo's not alone in using in-house markets to shift that predicting burden. At least two dozen other companies, including Microsoft (MSFT
) and Google (GOOG
), are basing decisions at least in part on employees' betting patterns, using a tool known as a prediction market.
HOW IT BEGAN. One of the earliest prediction markets was the Iowa Electronic Markets, founded in 1988 by the University of Iowa, to guess the winners of presidential elections. In the 1990s, a few companies, including Hewlett-Packard (HPQ
), began to apply prediction market theories to corporate events.
Now they are gaining wider acceptance. Markets are being used to let employees weigh in on a range of matters, from product launch dates to sales growth. Companies such as NewsFutures and Consensus Point even offer software and services to help companies set up their own prediction markets.
If it sounds like gambling, to some extent it is—except corporate prediction markets typically deal in play money. While some companies hand out cash prizes to winners, most give nonmonetary awards. The markets are helpful because they aggregate knowledge from a wide base of participants, and they can give CEOs a better handle on employee thinking. Still, getting busy workers to play along can be a struggle, and companies are divided on whether markets are reliable enough to justify the costs and the setup hassles.
ARCELOR'S ATTEMPT. In September, 2005, Arcelor, the largest steel producer in Europe and Latin America, began testing prediction market software from NewsFutures in its flat products division. The company wanted to add another method of predicting quarterly variations in sales volume and prices of a key product, hot rolled steel.
"This is a very different concept," says Javier Llera, senior manager of market intelligence at Arcelor's flat products division. Collectively, the 30-40 employees who play the market have accurately predicted volumes, but pricing is more of a moving target. "During the three-month period prior to the start of each quarter, the collective forecast improves," says Llera.
In the Arcelor market, participants are asked a question, such as the expected quarterly change in price for hot rolled steel in a given region. Each employee makes a prediction and records their level of confidence. They are able to see colleagues' entries.
They're also allowed to change predictions as new information arises, and they score points based on the accuracy of projections. Those with the highest scores win prizes such as a certificate for a weekend at a luxury resort.
A "RADICAL NOTION." But can groups working in a system like that really make better decisions than a few well-informed individuals? It's a radical notion for many in the corporate world, where CEOs and a handful of other executives are paid top dollar for presumed expertise.
And yet the idea is commonly accepted in the stock market. "Firms are comfortable with markets; it's a bit puzzling why they don't use them more often internally," says Todd Henderson, a professor of law at the University of Chicago.
Experts say prediction markets can work well for companies because they give a voice to employees who might not otherwise speak up. The markets are particularly useful in areas such as consumer goods or technology, where change is rapid and companies need to adapt quickly or get left behind.
And when it comes to introducing new products, there aren't many other reliable ways to predict success, some experts say. Computer maker Hewlett-Packard (HPQ
) ran a series of internal prediction markets from 1996-1999, to better forecast computer workstation sales. "In six out of eight cases, it was more accurate than internal corporate forecasts," says Kay-Yut Chen, a principal scientist at HP Labs.
MICROSOFT'S METHODS. Microsoft, the world's biggest software maker, says prediction markets can be more reliable than other methods when it comes to key product milestones. "There is no other established forecasting mechanism," says Todd Proebsting, a director at Microsoft's Center for Software Excellence.
Companies often rely on a manager's estimates of how a project is going, but those aren't always accurate. Since August, 2004, Microsoft has run 15 to 20 markets studying when a particular feature will work, when a project is ready for testing, or the number of bugs that will be reported in a piece of software in a given period.
Google also uses prediction markets to forecast product launch dates. In the past 15 months, Google has run more than 130 markets on events such as launch dates and new office openings. The company even runs markets on how often particular products are likely to be used.
Executives won't divulge specifics. But they could use a prediction market, for instance, to uncover the number of 7-day active Gmail users. In all, more than 1,000 employees participate in Google's prediction markets, competing for T-shirts, gift certificates and cash prizes.
PUBLIC INTEREST. And companies don't just rely on their own internal markets. Prediction markets that are open to the public yield data on a range of matters affecting business plans. These include presidential election outcomes, gas prices, real estate values, a film's performance at the box office, and even the probability of a flu pandemic. "Bird flu is one such potential risk with massive economic consequences," says John Delaney, CEO of Dublin (Ireland)-based Trade Exchange Network, the parent company of both Intrade and TradeSports.com.
Intrade and TradeSports.com, along with HedgeStreet, are among public prediction markets that require participants to use real money. Other markets, including the Hollywood Stock Exchange, Foresight Exchange, NewsFutures Exchange, and the World Economic Forum's Global Risks market, use play money.
Stephen Roman, a currency analyst at Forex Capital Markets, started playing real-money prediction markets about eighteen months ago. He's trying to stay ahead of the world events that can have a profound impact on foreign exchange rates. "I'd always followed news events closely and the idea that you could see the odds of something happening was fascinating," he says.
A PREDICTION PAYOFF. Roman says that prices in a prediction market better reflect the probability of something happening than, say, odds on the winner of a horserace, since bookmakers build their transaction costs into odds.
Two recent bets that paid off for Roman: He called the resignation of Harvard President Lawrence Summers, and he predicted that former Iraqi Prime Minister Ibrahim al-Jaafari would leave office. "It is fun and it's a little more fast-paced than the stock market," he says.
Whatever the outcome, companies are cautious not to give too much credence to prediction markets. "This is a new concept, which is essentially being researched," says Bo Cowgill, an analytics specialist at Google, who started the company's prediction market. While several Google senior executives look at the prediction markets with some regularity, they don't manage the company with that information, he says.
POSSIBLE PROBLEMS. A company considering tapping employees' collective wagering will have to tackle any number of challenges. For starters, CEOs need to ensure that they really want to know what a market could reveal, such as the likelihood of missing a sales target or product launch date. "The risk of creating markets is that it's not fudgible, and if you ask the question, then you better want to know the answer," says Robin Hanson, a professor in the department of economics at George Mason University.
If a product or another project is behind schedule, and everybody knows it, employees could lose motivation. "Everyone jumps off a sinking ship," says Justin Wolfers, a professor at the Wharton School at the University of Pennsylvania.
Companies could also face a disclosure dilemma. Would executives would need to share with investors insights gleaned from these markets? "Prediction markets could make everyone in the company an insider from the SEC [Securities — Exchange Commission] perspective, if you don't manage it well," says Google's Cowgill. This is one reason Google doesn't use prediction markets to forecast revenue.
PROVIDING INCENTIVES. Then there's the participation problem. "Prediction markets aren't everybody's cup of tea," says Microsoft's Proebsting. If the mechanics of the market aren't clear when a participant goes to the Web site, there's a big chance of losing a potential trader right away, he says.
To combat this, Proebsting does a step-by-step presentation for groups who haven't used prediction markets before. Microsoft, like many other companies, finds that somewhere between 40% to 60% of those who are invited actually take part.
Google's Cowgill tries to stir up enthusiasm for his markets with fun events, such as taking bets on which team would win the World Cup. His market accurately picked Italy as the winner. But more importantly, for him, once employees started trading in the markets, they moved onto other business-related ones.
In the absence of fun events, executives may find it tough to give busy employees enough incentive to play the market. That was the case at HP in the late 1990s. Chen says that, due to budget constraints, HP could only give an average incentive of $50 per person per event.
IMPROVING THE SYSTEM. During its three-year trial, HP found that prediction markets offered an accuracy improvement over other forecasting methods, though in most cases the improvement was only a few percentage points. "The accuracy improvement was not high enough to be adopted," says Chen. "You need to be a lot more accurate before it's worth it to implement a new process."
HP has since moved on to a new game that Chen has invented. It also aggregates information from employees, but does it in a way that's not a trading scenario. Chen says this new method is much more efficient.
Meanwhile, Yahoo is just getting started. The company is considering using prediction markets in other parts of its business, such as sponsored search. Says Pennock: "It's the best known method we have for aggregating opinions and providing forecasts."
Rachael King is a writer for BusinessWeek.com in San Francisco
