Intelligent Machines
A Futures Market for Computer Security
A predictions market could help companies prepare for major security incidents before they happen.
Information security researchers from academia, industry, and the U.S. intelligence community are collaborating to build a pilot “prediction market” capable of anticipating major information security events before they occur.
A prediction market is similar to a regular stock exchange, except the “stocks” are simple statements that the exchange’s members are encouraged to evaluate. Traders will buy and sell “shares” of a stock based on the strength of their confidence about the future outcome—with an overall goal of increasing the value of their portfolios, which will in turn earn them some sort of financial reward. Traders may choose to buy or sell additional shares of a stock, and that buying and selling activity pushes the stock price up or down, just as in a real market.
Some of the stocks being considering cover a few months, such as: “The volume of spam e-mail will increase by 10 percent in the third quarter of 2011.” Others will ask participants to gauge the likelihood of far-off events, such as the chance that the U.S. House of Representatives will pass a bill with “cyber” and “security” in its title in the first session of the 112th Congress, or whether broadly used encryption algorithms will be defeated within the next 24 months.
Greg Shannon, chief scientist of the CERT program at Carnegie Mellon’s Software Engineering Institute, who is involved with the project, says the purpose is to provide actionable data.
“If you’re Verizon, and you’re trying to pre-position resources, you might want to have some visibility over the horizon about the projected prevalence of mobile malware,” Shannon said. “That’s something they’d like to have an informed opinion about by leveraging the wisdom of the security community.”
Predictions markets have effectively forecasted all manner of events and trends, from the success of sports teams to the sales of new products. The pilot project will rely on software and services provided by Consensus Point, a Nashville-based company that has helped to build employee-driven prediction markets for several major companies, including General Electric, Best Buy, and Qualcomm. Best Buy’s prediction market—called “TagTrade”—is designed to give management an early indicator of which new products or ideas are likely to succeed, and whether specific new stores will open on time.
The University of Iowa’s Iowa Electronic Markets, one of the earliest prediction markets, has significantly outperformed the polls in every presidential election when forecasting more than 100 days in advance: Compared to 964 polls over the five presidential elections since 1988, the Iowa market was closer to the eventual outcome 74 percent of the time. The University of Iowa also uses prediction markets to forecast seasonal flu outbreaks.
Prediction markets have a major built-in bias—those answering the questions are not polled randomly—but respondents also have an incentive to respond only to those questions they feel confident in answering with accuracy.
“Prediction markets aren’t just surveys that ask everyone to speak up,” Robin Hanson, chief scientist at Consensus Point. ” People tend to speak up only when they’re reasonably sure they know the answer.”
Consensus Point CEO Linda Rebrovick says the goal of the project is to attract a network of about 250 experts, although the organizers are still deciding how to compensate for correct answers.
“There will be some combination of rewards and financial incentives for participating,” Rebrovick says.
Even if questions generate only tepid responses, such responses can be informative, says Dan Geer, chief information security officer at In-Q-Tel, the venture capital arm of the Central Intelligence Agency (CIA). Geer is also involved in the project. “It may be that this tells us there is ambiguity, or that we are, in effect, measuring disagreement on a question that doesn’t have a quantitative aspect to it,” Geer says. “Straight-out surveys are vulnerable to idiot answers, and prediction markets are vulnerable to stupid questions.”
While the pilot project will be limited to invited information security experts, the consensus decisions reached by the group will be made public. “Even if we can’t find something useful in all of this, we feel that’s a valuable result. It’s the way you make progress,” Geer says.
