A "Standard & Poors 500 Index"
for Intelligence
Puong Fei Yeh
“Prediction markets can contribute to US Intelligence Community strategic and tactical intelligence work ”
In 2001, the Defense Advanced Research Project Agency (DARPA) started
experimenting with methods for applying market-based concepts to intelligence.
One such project, DARPA’s Future Markets Applied to Prediction (FutureMAP)
program tested whether prediction markets, markets in which people bet on the
likelihood of future events, could be used to improve upon existing approaches
to preparing strategic intelligence. The program was cancelled in the summer of
2003 under a barrage of congressional criticism. Senators Ron Wyden and Byron
Dorgan accused the Pentagon of wasting taxpayer dollars on “terrorism betting
parlors,” and that “Spending millions of dollars on some kind of fantasy league
terror game is absurd and, frankly, ought to make every American angry.”
Americans
need not have been angry about FutureMAP. It was neither a terrorism betting
parlor nor a fantasy league. Rather, it was an experiment to see whether
market-generated predictions could improve upon conventional approaches to
forecasting. Since 1988, traders in the Iowa Electronic Markets have been
betting with remarkable accuracy on the likely winner of the US presidential elections.[1] Eli Lilly, a major pharmaceutical company, found that prediction markets outdid
conventional methods in forecasting outcomes of drug research and development
efforts.[2] Google recently announced that it was using prediction markets to “forecast
product launch dates, new office openings, and many other things of strategic
importance.”[3]
The decision to cancel
FutureMAP was at the very least premature, if not wrong-headed. The bulk of
evidence on prediction markets demonstrate that they are reliable aggregators
of disparate and dispersed information and can result in forecasts that are
more accurate than those of experts. If so, prediction markets can
substantially contribute to US Intelligence Community strategic and tactical
intelligence work.
FutureMAP and the Policy
Analysis Market
When
the FutureMAP project began in 2001, DARPA solicited proposals for
“market-based techniques for avoiding surprise and predicting future events.”
[4] Two proposals were
selected for further funding, but Net Exchange’s Policy Analysis Market (PAM)
became the public face of the FutureMAP project until it was terminated.
PAM
would have offered trading on the following kinds of contracts: (1) political,
economic, and military indicators for Egypt, Iran, Iraq, Israel, Jordan, Saudi
Arabia, Syria, and Turkey; (2) global economic and conflict indicators; and (3)
on events as they came up, e.g. the likelihood of Hamas recognizing the state
of Israel.[5] It also would have offered contracts called conditional derivatives, which
would have allowed traders to speculate on events conditional on the occurrence
of other related events (e.g., a trader might bet on the likelihood that the
Saudi regime will fall if the United States withdraws from Iraq).[6] PAM’s creators
believed that the conditional derivative would have enhanced the “prediction
power” of the market.[7]
[Top of page]
Prediction Markets: Theory and Evidence
The theories underlying PAM and other prediction markets are
the Efficient Capital Markets Hypothesis (ECMH) and the Hayek hypotheses.[8] These hypotheses
explain how information is aggregated such that market prices provide accurate
estimates on the likelihood of future outcomes.[9] According to ECMH, capital markets are “extremely efficient in reflecting
information about individual stocks and about the stock market as a whole,”
such that no amount of analysis in an attempt to forecast future stock prices
can beat the market.[10]
Expanding on this hypothesis further was the idea of a
“random walk.” The logic of the random walk is that if information flows
without impediments and stock prices immediately reflect that information, then
tomorrow’s price changes will reflect only tomorrow’s news and are independent
of today’s price changes. But since news is unpredictable, then price changes are
also unpredictable. Consequently, prices fully reflect all known information,
and even uninformed investors buying a diversified portfolio at market prices
will obtain a rate of return as generous as that achieved by the experts.[11] Thus “[i]n an
efficient capital market, asset prices reflect all relevant information and
thus provide the best prediction of future events given the current
information.”[12] Oil futures prices, for example, have been demonstrated to act as a function of
the spot price and an estimate about the cost of carrying the commodity
until the time of delivery.[13]
For prediction markets, the theory that price
instantaneously reflects information is only part of the story. The other part
rests with the Hayek hypothesis. Hayek, criticizing central planning in 1945,
sought an answer to the following question: how does one effectively aggregate
disparate pieces of information that are spread among many different
individuals, information that in its totality is needed to solve a problem?[14]
Hayek’s answer was that market prices are the means by which
those disparate pieces of information are aggregated. “The mere fact that there
is one price for any commodity…brings about the solution which…might have been
arrived at by one single mind possessing all the information which is, in fact,
dispersed among all the people involved in the process.”[15]
Additionally, the market works even when people have limited
knowledge about their surrounding environment and the people with whom they
transact.[16] An interesting application of the Hayek hypothesis was the explosion of the
NASA space shuttle Challenger in 1986. Within minutes of that explosion, Wall
Street traders seemed to identify who would be held responsible for the crash
while a presidential commission took nearly four months to conclusively
pinpoint the cause of the tragedy.[17] The Challenger study authors’ note that “What the Challenger episode adds to
Hayek’s insights is that securities markets are vehicles for amalgamating
unorganized knowledge.”[18]
Trading Mechanics
Trading in prediction markets is similar to any haggling
kind of transaction: buyers and sellers exchange offers and counter-offers
until they agree on a price.[19] In a double auction, the most common mechanism used to clear prediction
markets, buyers submit bids and sellers submit asking prices, which are ranked
from highest to lowest to generate supply and demand curves. Trades are
executed when two prices match (i.e., bid-ask spread is zero or supply
intersects demand). In describing Eli Lilly’s 2003 experimental prediction
market, Vice President for Lilly Research Laboratories Alpheus Beingham noted
that, “When we start trading stock [in the drug], and I try buying your stock
cheaper and cheaper, it forces us to a way of agreeing that never really occurs
in any other kind of conversation.”[20]
In prediction markets payoffs are determined by the
occurrence (or lack thereof) of outcomes. Consider the following contract: Senator
Hillary Clinton will declare her candidacy for the 2008 presidential election
by 1 January 2007. If the contract has a share price ranging from 0 to 100
cents, the contract would pay 100 cents if, in fact, the senator declares
before then. In this case, a trader who bought 10 shares of the contract at 67
cents would realize a profit of 330 cents (1000-670=330); if she doesn’t
declare, that trader gets nothing. [21] The same trader could also profit by selling his shares to another trader at a
price higher than 67 cents before the closing period of the contract.
Prediction market proponents claim that market prices for
contracts can be interpreted as probabilities of an expected outcome. In the
above example, a contract closing at 67 cents would mean there is a 67 percent
probability that Senator Clinton will declare her presidential candidacy before
1 January 2007.
The contention that market prices can be interpreted as
probabilistic estimates of future events is not without controversy.[22] One specialist,
Charles Manski, argues that it is dangerous to read market prices as probabilities.[23] Others note that
little is known about why a trade occurs in prediction markets.[24]
Numerous studies have suggested, however, that markets do
lead to predictions that are more accurate than traditional forecasting
techniques, including those that rely on expert opinions. A study of the Iowa
Electronic Markets during the 1988 US presidential election concluded that
market predictions of the two candidates’ vote shares were closer to the actual
vote shares than were the polling data of six major organizations.[25] Orange juice futures
prices have been shown to be better predictors of weather than the National
Weather Service’s forecasts.[26] A preliminary study of the Goldman Sachs and Deutche Bank’s Economic
Derivatives market, which allows traders to hedge against surprises in economic
statistics like unemployment and GDP data, concluded that prediction markets,
“may be useful as a supplement to the other relatively primitive mechanisms for
predicting the future like opinion surveys, politically appointed panels of
experts, hiring consultants or holding committee meetings.”[27]
[Top of page]
From Orange Futures to Market Intelligence and Policy Analysis
While
some prediction markets outperform experts and polls in predicting
winners of presidential elections and weather in Florida, at least two
other experiments suggest the markets can perform intelligence and
policy analysis functions.
HP Labs and Market Intelligence
In 1996, HP Labs and Caltech conducted a three-year
experiment using an “information aggregation mechanism,” (IAM) or prediction
market.[28] Echoing Hayek’s information aggregation problem, the study noted that
responsibility in businesses for aggregating information in a timely way lies
with many different individuals throughout the company and that such efforts
have been costly and by most standards inefficient.[29] Moreover, “business practices
such as quotas and budget settings create incentives for individuals not to
reveal their information.”[30]
The
IAM experiment involved 12 predictions over a three-year period.
Traders were paid if and only if they owned the security that
corresponded to the actual sales outcome (e.g., trader owns stock that
forecasts the actual unit sales within a given range of units). The IAM
aggregated information from 20 to 30 people across different parts of
the United States and from HP business, finance, and market divisions.
They were selected because they possessed “different patterns of
information” (e.g., pricing strategies and client specific data) that
“were in need of aggregation.”[31] To “provide market liquidity” five participants from HP Labs who were ignorant
of HP business-related information also participated.[32]
The experiment was a success. In 75 percent of the predicted events for which there were HP official forecasts, IAM predictions
came closer to the actual outcomes than did the official forecasts.[33] The experiment also gave credence to the theory that prediction market prices
act as probabilistic estimates of future sales targets.
Thus, if a stock that corresponds to a sales projection
interval of 1,201 to 1,300 has a share price of 20 cents, it means that there
is a 20 percent probability that actual sales will fall within this range. The
study noted that the advantages of using an IAM lie in its ability to
“aggregate any type of information possessed by different people,” to quantify
and give weights “to the opinions of different people,” and in its scalability.[34]
The “Saddam Security” Policy Analysis
In contrast to the HP Labs experiments, the Saddam Security
study was an experiment to determine if decisionmaking could be informed in
real time by existing prediction, financial, and energy markets.[35]
One month prior to the US invasion of Iraq in March 2003, Wolfers and
Zitzewitz attempted an estimate of the effects of a US decision to go
to war with Iraq. The authors examined the relationship between equity
and oil spot and futures prices and the Saddam Security, a
Tradesports.com contingent security, that paid if and only if Saddam
Hussein was out of office by 30 June 2003.
In
the weeks preceding the invasion of Iraq, the authors reasoned that the
higher the price of the Saddam Security then the higher the probability
of the United States going to war. If during the same trading period
oil futures prices on contracts for delivery toward the end of 2003
were relatively high, then that would suggest investors expected the
war to cause medium disruptions in supply (i.e., no destruction of oil
fields). Similarly, if S&P 500 futures prices for one-year ahead
during the same period were negatively correlated with the Saddam
Security, that would suggest investors believed the war would
negatively affect the broader global economy. The rationale for using
equity and oil futures prices was that they reflected traders’ best
guesses on the economic and political conditions at the time of the
contract delivery date.
Researchers Looney, Schrady, and Brown performed similar
correlations, but on historical events. Observing that oil-futures prices
tended to sharply increase when a crisis breaks and steadily fall back to
pre-crisis levels once US naval forces arrived on the scene, the three
calculated that these price declines “produced significant cost savings to the
United States economy” in the range of $55.2 billion for the US economy in the
immediate aftermath of the Iraqi invasion of Kuwait in 1990.[36]
Exploring the possibilities of prediction
markets further, others have proposed that these markets should serve as
mechanisms to help decide which of several policies options should be
implemented. Hanson, for instance, hypothesized the creation of markets to
guide policymaking in which, “people could bet on future crime rates,
conditional on allowing concealed weapons.”[37] Hahn and Tetlock argue that the markets have the potential to provide informed evaluations
of policy proposals before they are adopted.[38]
[Top of page]
Using Prediction Markets to Enhance Intelligence Capabilities
How then can prediction markets improve the performance of
the US Intelligence Community? In many respects the challenge of intelligence
goes to the core of the Hayek hypothesis: How do you aggregate, in a timely
way, disparate pieces of information that are spread among and within 15 US
intelligence agencies into relevant products? Putting aside market design
questions for now, prediction markets can help address shortcomings in
analytical organization and processes, improve long-term intelligence
estimates, and perform real-time and ex-ante policy evaluations.[39]
Information and Analytical Aggregation
The 9/11 Commission, in its discussion of how to reorganize
the US Intelligence Community, cited the lack of unity of effort in information
sharing as the “biggest impediment to all-source analysis—to a greater
likelihood of connecting the dots.”[40] The lack of information sharing is further compounded by a culture that
emphasizes information compartmentalization, suffers from stovepipe
mentalities, and bureaucratic distrust.[41] One way to solve these problems is to work on IC-wide software and databases
and develop improved protocols for accessing classified information and for
providing better coordination of interagency analyses. Another way is to use
prediction markets to aggregate information and analyses. In the way HP’s IAM
fused together information and judgments from different corporate divisions
into probabilistic estimates of future outcomes, a prediction market could
perform the same function for the Intelligence Community.
A
good illustration of the way in which an Intelligence Community
prediction market might have worked in the months before the beginning
of the 2003 Iraq war is the case of the contested meaning of Iraq’s
purchase of specialized aluminum tubes in 2001. As is now well-known,
Intelligence Community analysts disagreed sharply about their
significance, some believing they were intended for Iraq’s putative
nuclear program. Irrespective of major disagreements, the conclusion
that the tubes were part of Iraq’s reconstituted nuclear program worked
its way into the case for war that Secretary of State Colin Powell made
before the United Nations in February 2003.[42] In hindsight, the judgment was
wrong.
 - This chart, showing the fluctuation in prices in the 2006 US Congressional Control Market of the Iowa Electronic Markets, illustrates the way in which market players’ collective judgment unfolded in the five months before the November mid-term election. (Image courtesy of University of Iowa, Henry B. Tippie College of Business.)
|
Turning back the clock, imagine that in February 2001
analysts throughout the community had the opportunity through an Intelligence
Community-wide prediction market to bid on the following yearly futures
contract (share price ranging between 0 and 100 cents): The Iraqi-purchased
high-strength aluminum tubes are for use in a uranium enrichment program.[43] The
specificity of the contract is noteworthy because it eliminates the ambiguity
surrounding a judgment about whether the aluminum tubes could be used in a
uranium enrichment program.[44]If demand (buyers) exceeds supply (sellers) for the contract (i.e.,
analysts believe that the tubes are destined for use in the Iraqi nuclear
program), then the share price rises. Conversely, if supply exceeds demand
(i.e., analysts believe the tubes are not destined for use in the Iraqi nuclear
program), then the share price decreases. In other words, the market price of
the contract depends on the price at which analysts are willing to bid and ask.
In this example a prediction market could have aggregated
the different information and analytic judgments of the different agencies into
a single market price. Hypothetically, share prices for this contract would
have fluctuated from a high of 87 cents in March 2001 when reports of attempted
aluminum purchases were first received to a low of 38 cents in February when
uncertainty remained about the end uses of the tubes. Thus when trading closed
in February 2003, the closing share price of 46 cents would have told
policymakers that the Intelligence Community believed that there was only a 46
percent chance that the aluminum tubes were for use in Iraq’s nuclear program.[45]
The judgment about the aluminum tubes was only one of many
inaccuracies that underlined the conclusion of the October 2002 National
Intelligence Estimate, which said that Iraq was reconstituting its nuclear
program. The commission investigating pre-invasion intelligence stated that,
“the NIE [October 2002 National Intelligence Estimate] too often failed to
communicate the paucity of intelligence supporting its assessments and also
contained several inaccurate statements.”[46] Prediction markets could have been especially helpful in the formulation of the
October 2002 NIE because with prediction markets, uncertainties and certainties
are expressed through a probabilistic collective judgment (the market price)
rather than through a consensus.
In this respect, because dissenting views affect market
prices they have considerably more value in prediction markets than they do in
intelligence estimates, where they may end up as unnoticed footnotes. Markets
also work better when traders disagree on what is the “truth” since trading by
its very nature means that an individual is attempting to profit from another
person’s perceived poor judgment.[47] Analysts buy (or sell) based on the information they possess. Those willing to
pay a higher price to engage in a transaction in expectation of a higher payoff
will do so, especially when they think they are right.
Long-Term Estimates (Avoiding Strategic Surprise)
Long-term intelligence estimates provide judgments on the
likely path of major issues affecting national security. These issues can range
from the proliferation of weapons of mass destruction to terrorism. However,
problems of ambiguity of judgment sometimes render an analysis useless and
ambiguity of evidence can further muddy the analysis.[48] Criticisms of past intelligence
estimates have also pointed out their lack of timeliness. For instance, as the
1979 Iranian revolution unfolded, a long-term estimate on the outlook for
Iranian political stability in the works deemed Iran politically stable.[49] Under these
circumstances, policymakers may find it difficult to draw any useful
conclusions from the intelligence.
How
would the IC arrive at a community-wide judgment if intelligence both
supports and undercuts the contention that the balance of power in the
Taiwan Straits will shift in favor of the Chinese against the United
States in 2008? Imagine that analysts bet on the following contract
(share prices ranging between 0 and 100 cents):
In
2008, China will prevail against the United States in a clash in the Taiwan
Straits, conditional on China successfully fielding supersonic sea-launched
cruise missiles.
Further
assume that policymakers want the long-term estimate to be completed in
three months and that trading occurs during the three-month time frame.
Therefore if the closing share price on the final day of trading is 87
cents then policymakers can interpret the closing share price as the
IC’s estimate that there is an 87 percent probability that the balance
of power in the Taiwan Straits will shift in China’s favor in 2008. The
closing share price of 87 cents also signals that: (1) the fielding of
supersonic ship-killing missiles by 2008 is a critical determinant in
estimating whether China will prevail in the straits; and (2) the US
Navy does not possess adequate countermeasures against sea-borne
cruise-missile attacks.
To
provide more depth to this analysis, analysts could bet on the
probability that China successfully fields sea-launched cruise missiles
by 2008, provided that China’s current rate of research and development
remains constant. They could also bet on China’s intentions by
speculating on the likelihood that Chinese leaders will seek to
forcibly reunify Taiwan by 2008, conditional on Taiwan introducing
another referendum on independence.
Prediction markets can function as powerful complements to
the traditional process by which long-term estimates are performed. Their power
is further multiplied when one imagines that the time and resources saved in
running such markets means that several long-term estimates can be run
concurrently and updated periodically. At the very least, had a prediction
market existed on Iran’s long-term political stability in 1979, fluctuations in
the share prices of the appropriate contract would have quickly reflected the
import of unfolding events and shifts in analytical judgments. Moreover, by
allowing analysts to hedge their estimates in the form of conditional
contracts, policymakers gain valuable probabilistic estimates, as opposed to
wishy-washy judgments which policymakers may easily ignore.
Attack Warnings (Avoiding Tactical Surprise)
Can prediction markets help avoid tactical surprise? Here
there are no clear answers. An attack that is truly surprising necessarily
results from a failure of strategic intelligence—“We had no idea they were
going to attack us.”
But a glance at the record of historical intelligence
failures suggests that such surprises are the exception and that failures most
often result from failure to communicate appropriate warnings or failure to
assess evidence correctly.
Thus one difficulty in using prediction markets to forecast
tactical problems is that the market requires contracts for explicitly
anticipated events.(e.g., what is the probability that Al-Qa’ida
will hijack planes and fly them into the World Trade Center, Capitol Hill, and
the Pentagon on or around September 11, 2001?)[50]
Such specificity is hard to come by, a fact aptly
demonstrated by the often ambiguous nature of the Department of Homeland
Security’s Threat Advisory warnings. Even if specificity were possible, there
remains the question of how much a share price needs to rise (e.g., 38 cents,
52 cents, 61 cents) before it is taken seriously by policymakers.
Where a prediction market might be useful is in speculating
on the probability that a certain method would be used in an attack. For example,
in the years leading up to 11 September 2001, analysts could have speculated on
a yearly futures contract associated with the likelihood of terrorists
hijacking planes and using them as aerial suicide bombs. In theory, an NIE on
terrorist threats against the United States would display a graph of rising
futures prices associated with the aerial suicide bomb contract. But again,
this presupposes that analysts had contemplated the method of attack and issued
the appropriate trading contract before 11 September 2001.[51]
Hanson
has proposed an alternative use for tactical prediction markets in
which trading revolves around the probability of red teams (US security
teams that act as terrorist cells) penetrating US homeland security
defenses (e.g, placing fake explosives on Capitol Hill). He suggests
that markets could trade on the rate of red team “wins” conditional on
the type of security measures (e.g., private vs. government airport
security screeners) used to thwart the mock terrorists.[52] The value of such a market is in identifying weaknesses in homeland defenses
without having to experience a market test of an actual terrorist attack.
Assessing Policy Choices
Is the United States winning in Iraq? Will the Andean Regional
Initiative decrease the supply of cocaine to the United States? These
are just some of the questions policymakers might ask that prediction
markets could help answer. If the United States goals in Iraq are to
quell the insurgency and to establish a bulwark of democracy in the
Middle East, futures contracts issued to the market might revolve
around a composite index of economic and political freedoms in Iraq and
in the broader Middle East, indicators of civil stability and economic
growth, or measurements of oil output and kWH of electricity
generation. If analysts believed that the United States was losing the
war in the short-term but winning in the long run, then one would
expect share prices for short-term contracts to be relatively lower
than those for longer-term contracts. These futures contracts could be
quarterly, yearly, or two-year contracts. Thus different share prices
at different points in time for different futures contracts would
provide policymakers with a more nuanced real-time evaluation of
whether US policy in Iraq is working.
Prediction
markets could also be used to make ex-ante evaluation of policies. Take
the question of whether the United States should continue to fund the
Andean Regional Initiative (ARI). Analysts could bet on two futures
contracts: (1) the tons of cocaine that will be exported from
the countries affected by the ARI to the United States in 2009, conditional on
the United States continuing ARI; and (2) the tons of cocaine that will
be exported if ARI is terminated. The difference in the two estimates would
tell policymakers how much of a reduction (or increase) in cocaine analysts
expect from the implementation of ARI. A more realistic assessment would most
likely involve analysts speculating on several futures contracts with different
expiration dates.
Prediction
Market Design Issues
Before a system of Intelligence Community prediction markets
could be implemented, key market design issues need to be addressed. For
example, does the number of traders in a market matter? The HP experiment was
successfully conducted with fewer than 30 participants, but contracts traded at
Tradesports or the Iowa Electronic Markets have participants many times that
number. Must traders be subject matter experts on the issue for which they are
betting, or can they be somewhat in the dark, like the traders in the Hayek’s
story or the HP Labs participants?
Public versus Private Prediction Markets
Prediction markets aggregate information and judgments, but
whose information and judgments should be aggregated for the best estimate of
future events? The report on the HP experiment noted that there is only limited
theoretical knowledge about the proper balance between participants with much
relevant information versus those without any or limited relevant information.[53] Hanson has suggested
that prediction markets “can be used to aggregate information from any given
set of participants.”[54]
Since the objective here is to effectively aggregate
information and analyses of the entire Intelligence Community, implementation
of prediction markets on a community-wide basis is preferable to intra-agency
markets. Ideally, anyone with the relevant information should trade. If the
traded contract relates to aerial suicide bombs, then even airport luggage
screeners, in addition to homeland security analysts, are potential market
participants. This necessarily means that expert knowledge on a particular
subject is not required before making a bet.
A more difficult question is whether there are circumstances
under which the general public should be allowed to trade. Certain issues might
require the aggregation of information and opinions on subjects intelligence
officers may know little or not enough about. On the other hand, making public
certain markets might be inadvisable because doing so might signal adversaries
about intelligence interests.
A compelling case can be made for making diversity a key
criterion. Diversity means that market participants have different pieces of
information about their surrounding environment and consequently different judgments
on the event for which they are betting. The HP experiment aggregated
information across several corporate divisions. Economic theory and empirical
evidence suggests that “thick” markets are preferable to “thin” ones.
Contract Specification and Determination
The market prices of prediction markets are only meaningful
if the contracts address the right questions and address them clearly. Wolfers
and Zitzewitz note that a prediction market works best when contracts are
clear, “easily understood and easily adjudicated.”[55]
Another consideration is avoiding situations in which
traders are punished for guessing correctly. This happens, for example, when
traders are asked to speculate on whether Boeing’s Future Combat Systems will
deliver a battlefield communications network to the Army on time, and in
response to sagging market prices the Army extends the deadline. The solution
to this example is to specify two conditional contracts: (1) what is the
likelihood that Boeing will deliver the product on time, conditional on a
contractual change; and (2) the same question but conditional on no contractual
changes.
The
final consideration in contract specification is in determining whether
the contract is realized when it expires. Someone has to act as the
final adjudication authority in deciding whether, in fact, the balance
of power in the Taiwan Straits has tilted in China’s favor against the
United States—short of the market test of an actual conflict. For
contracts involving measurements such as the real-time or ex-ante
evaluation of policies (e.g., US cocaine imports), the methodology of
measurement should be fixed in advance. Traders need certainty that
they will be rewarded for advice that is correct. Serious thought needs
to be given to deciding who in the Intelligence Community should set
the contracts for trading and judge whether the contractual outcomes
are realized.
Soundness of the Theoretical Basis
The fact that prices in prediction markets fully and
instantaneously reflect and aggregate all known information is an extension of
the Efficient Capital Markets and Hayek hypotheses. In recent years behavioral
finance theory has challenged the efficient markets hypothesis, which holds
that rational actors account for stock market volatility.
[56] Behavioral finance theory asserts
that human psychology affects financial markets. It argues, for instance, that
the feedback phenomenon in which enthusiasm begets enthusiasm explains the rise
and burst of the Internet stock market bubble. In response, proponents of ECMH
argue that markets are efficient in spite of irrational human behavior because
in the long-run “true value” overcomes the “voting mechanism.”
[57]
For intelligence consumers the concern is that speculative
bubbles will drive prices away from the “true price,” thereby misleading
policymakers. And even if speculative bubbles eventually burst, policymakers do
not always have the time to wait for that to occur. There are also the problems
of recognizing a speculative intelligence bubble and what to do if one occurs.
Could one establish, for example, an instrument like the Federal Funds rate
that an Intelligence Community equivalent to the Federal Reserve chairman could
use to deflate a bubble? Wolfers and Zitzewitz note that further lab
experiments are central to learning more about bubbles in prediction markets
since “it is possible for the experimenter to know the ‘true price’ and, hence,
to observe deviations.”[58]
Regardless, the possibility of speculative bubbles in
prediction markets should not be the sole basis for a decision not to implement
prediction markets if, on average, prediction markets outperform conventional
forecasting methods. Certainly, the October 2002 NIE was prone to a form of
speculative intelligence bubble. The stock market, in spite of its drawbacks,
still manages to allocate hundreds of billions of dollars of equity capital to
industry sectors more efficiently than any other social institution, especially
those that rely on central planning.
Market Manipulation and Bias
In the summer of 2003, one criticism of PAM was that market
manipulation would render its results useless. Analysts might engage in trading
behavior to fit a certain policy outcome (a specialized form of politicization)
or worse, terrorists could manipulate the market to mislead the IC or even use
the market to finance attacks. “Historical, field, and laboratory data,
however, have failed to find substantial effects of such manipulation on
average price accuracy” and instead attempts to manipulate markets actually
increase the accuracy of information markets.[59]
Rhode and Strumpf noted that attempts to manipulate
presidential betting markets in the early 20th century as well as their own
attempts to manipulate prices of presidential candidates during the 2004
election year had a negligible impact on prices.[60] Empirical evidence
notwithstanding, one simple preventative would be to limit participation in
prediction markets.[61] The key consideration in implementing this measure is similar to considerations
in deciding the scope of the prediction market: what scale and level of
participation is required for information aggregation to work?
In
addition to market manipulation, there may be concerns that trader’s
judgment or behavioral bias might influence market prices. This bias
occurs when traders trade according to the outcomes they desire rather
than a dispassionate assessment of what is likely. An analogy is that
in the run-up to the Iraq war, intelligence analysts were so convinced
that Iraq had reconstituted their WMD programs that any evidence,
regardless of its veracity, only served to harden their earlier
convictions.
Forsythe, Nelson, Neumann, and Wright examined the
phenomenon of judgment bias in their study of the Iowa Presidential Stock
Market in 1988 and concluded that these biases were prevalent.[62] However, despite those biases,
market predictions proved remarkably accurate on account of the marginal-trader
hypothesis. Under this hypothesis, the marginal trader determines market
prices. The authors noted that marginal traders essentially act as arbitrageurs
by profiting in buying stocks from one set of biased traders and selling them
to another set of biased traders. And by engaging in arbitrage, the marginal
traders set the market price despite the fact that the average trader was
biased.[63]
Real- vs. Play-Money: Accuracy, Motivation, Legal, and Moral
Issues
The evidence on whether real-money prediction markets lead
to forecasts that are more accurate than those of play-money markets is
inconclusive. Some experts believe that markets in which traders have to “put
their money where their mouth is” produce better results than markets in which
traders do not risk their money.[64] Essentially, these experts argue that the profit-motivation in real-money
markets contributes to a working market.
One study that compared the predictions of the two markets
(Tradesports, a real-money market, v. NewsFutures, a play-money market)
concluded that the play-money markets performed as well as the real-money
markets.[65] The implications of this finding go beyond the accuracy issue since there are
also legal, financial, and ethical issues involved in setting up a real-money
market. PAM, for instance, was forced in part to consider conducting a public
market trial due to restrictions on government inter-agency transfers of money.
In any event, PAM presumably would have had to comply with US gambling laws;
TradeSports, which deals in real-money trades, is based in Ireland so as not to run afoul of US gambling laws.
If real-money markets are set up, then decisions are needed
on the value of the payoff per outcome (e.g., 100 cents is paid if event “A”
occurs) and whether to allocate cash to market participants, and if so how
much.[66] Morally, one might limit the value of the payoff per outcome to as low as 100
cents to avoid the appearance of rewarding analysts for correctly predicting
bad outcomes (e.g., US troop deaths in Iraq will exceed 3,000 by some date). In
a public real-money market payoff limits could mitigate concerns of “bad guys”
using the market to finance their illicit activities. If play-money markets
were implemented, other incentives might be needed, for example, mechanisms for
granting “community bragging rights.”[67]
[Top of page]
Conclusion
The record of prediction markets is impressive. For the US
Intelligence Community, prediction markets offer a method by which to improve
analytical outcomes and to address some of the deficiencies in analytical
processes and organization. In the realm of intelligence analysis, prediction
markets can contribute to more accurate estimates of long-term trends and
threats and better cost-benefit assessments of ongoing or proposed policies.
Further study is needed on how prediction markets can
improve tactical intelligence, and much more thought is required to ensure that
policymakers and intelligence chiefs will value the results of prediction
markets if they are attempted. Without their engagement, there would be no
motivation to trade, and market performance would suffer.[68]
Despite everything that prediction markets can do to enhance
US intelligence capabilities, at the end of the day, prediction market results
are just probabilistic estimates of future outcomes. A stock price that shows a
15 percent probability of a Sino-Japanese clash over disputed territory in the
East China Sea in 2010 still only means that there is a chance, albeit a low
one, that the outcome will occur. Policymakers still must decide on the
threshold for action. And as often is the case, human intuition will carry the
day when definitive intelligence is lacking.
[1]http://www.biz.uiowa.edu/iem/media/su
mmary.html.
[2]Rana Foroohar, “A New ‘Wind Tunnel’ for Companies: Testing economic theories through
experiments,” Newsweek, October 20,
2003. http://msnbc.msn.com
/id/3087117.
[3]http://googleblog.blogspot.com/2005/09/putting-crowd-wisdom-to-work.html.
[4]The
FutureMAP Web site or DARPA’s Information Awareness Office no longer
exists.However, numerous other Web sites have snapshots of the
original, displayedabove, including their content. See http://hanson.gmu.edu/policyanalysismar ket.html and http://www.ratical.org/ratville/CAH/linkscopy/PAM/
pam_home.htm
[5]John
Ledyard, Robin Hanson, and Takashi Ishikida, “An Experimental Test of
Combinatorial Information Markets,” (February 2005): 4. http://hanson.gmu.edu.
[6]http://www.ratical.org/ratville/CAH/links
copy/PAM/pam_home.htm. See also Robert Looney,
“DARPA’s Policy Analysis Market for Intelligence: Outside the Box or Off the
Wall?”, Strategic Insights II, Issue 9 (September 2003), 3 (on
http://www.ccc.nps.navy.mil.) and Robin D. Hanson, “Impolite Innovation: The
Technology and Politics of Terrorism Futures and Other Decision Markets,” at
HTTP://hanson.gmu.edu/PAM/HansonTal ks/ImpoliteInnovation, 10–11.
[7]http://www.ratical.org/ratville/CAH/links
copy/PAM/pam_home.htm.
[8]Justin
Wolfers and Eriz Zitzewitz, “Prediction Markets in Theory and Practice,” in The New Palgrave Dictionary of Economics, eds. Lawrence E. Blume and
Steven N. Durlauf, November 15, 2005 Draft (London: Palgrave Macmillan): 4. See
also Robert Forsythe, Forrest Nelson, George R. Neumann, and Jack Wright,
“Anatomy of an Experimental Political Stock Market,” The American Economic
Review, 82, No.5 (December 1992): 1143.
[9]Theoretical
explanations for prediction markets have been made on the basis of either hypothesis
but not both at the same time. See Charles F. Manski,
“Interpreting the Predictions of Prediction of Markets,” NBER Working Paper
10359 (March 2004): 1.
[10]Burton G. Malkiel, “The Efficient Market Hypothesis and Its Critics,” Journal of
Economic Perspectives 17, No.1 (Winter 2003): 59.
[11]Ibid.
This is essentially behind the proposition that stock market index funds, on
average, will out perform actively managed funds.
[12]Paul
W. Rhode and Koleman S. Strumpf, “Historical Presidential Betting Markets,” Journal
of Economic Perspectives 18, No.2 (Spring
2004): 136.
[13]Anthony
E. Bopp and George M. Lady, “A comparison of petroleum futures versus spot
prices as predictors of prices in the future,” Energy Economics 13, No.
4 (October, 1991): 274–76.
[14]F.A.
Hayek, “The Use of Knowledge in Society,” The American Economic Review 35,
No.4 (September, 1945): 520.
[15]Ibid.,
526.
[16]Ibid.
[17]Michael
Maloney and J. Harold Mulherin, “The complexity of price discovery in an
efficient market: the stock market reaction to the Challenger crash,” Journal
of Corporate Finance 9, No. 4 (2003).
[18]Ibid.,
474.
[19]The
Iowa Electronic Markets (http://www.biz.uiowa.edu/iem/trmanual/I
EMManual_3.html), InTrade (http://www.intrade.com), and NewsFutures
(http://news.us.newsfutures.com/guide.ht ml) provide good users manuals
detailing how their respective market platforms operate. Although there are
differences, generally they rely on the principle of a double auction. Note
that at InTrade, the users guide confuses the meaning of bid and ask.
[20]Barbara
Kiviat, “The End of Management?” Time,
July 6, 2004. http://www.time.com/.
[21]At
NewsFutures, traders are given the option to trade two contracts: outcome
occurs and outcome does not occur.
[22]Justin
Wolfers and Eriz Zitzewitz, “Prediction Markets,” Journal
of Economic Perspectives 18, No.2 (Spring 2004): 109.
[23]Charles
F. Manski, “Interpreting the Predictions of Prediction of Markets,” 6.
[24]Justin
Wolfers and Eriz Zitzewitz, “Prediction Markets in Theory and Practice,” The New Palgrave Dictionary of Economics, eds. Lawrence E. Blume and Steven N. Durlauf, November 15, 2005 Draft (London: Palgrave
Macmillan): 4.
[25]Robert
Forsythe, Forrest Nelson, George R. Neumann, and Jack Wright, “Anatomy of an
Experimental Political Stock Market,” The American Economic Review 82, No.5 (December
1992): 1148-1149.
[26]Richard
Roll, “Orange Juice and Weather,” The American
Economic Review 74, No. 5 (December 1984).
[27]Justin
Wolfers and Eriz Zitzewitz, “Prediction Markets,” 125.
[28]For
a survey of experiments related to the information aggregation problem (testing
of the Hayek Hypothesis) see Shyam Sunder, “Experimental Asset Markets: A
Survey,” The Handbook of Experimental Economics,
eds. John H. Kagel and Alvin E. Roth (Princeton: Princeton University Press:
1995).
[29]Kay-Yut
Chen and Charles Plott, “Information Aggregation Mechanisms: Concept, Design
And Implementation For A Sales Forecasting Problem,” California Institute of
Technology Social Science Working Paper No. 1131 (March 2002): 3. See
also Ajit Kambli, “You Can Bet on Idea Markets,” HBS
Working Knowledge (December 1, 2003): http://hbswk.hbs.edu/pubitem.jhtml?id=3808&t=innovation.
[30]Kay-Yut
Chen and Charles Plott, 3.
[31]Ibid.,
5.
[32]Ibid.,
10.
[33]Ibid.,
11–13.
[34]Chen
and Plott, 17.
[35]Justin
Wolfers and Eriz Zitzewitz, “Using Markets to Inform Policy: The Case of the Iraq War,” NBER Working Paper (June 2004).
[36]Robert
E. Looney, David A. Schrady, and Ronald L. Brown, “Estimating the Economic
Benefits of Forward-Engaged Naval Forces,”Interfaces No. 31 (July-August 2001), 83-86.
[37]Robin
D. Hanson, “Decision Markets,” IEEE Intelligent
Systems (May/June 1999):16-17.
[38]Robert
W. Hahn and Paul C. Tetlock, “Using Information Markets to Improve Public
Decision Making,” AEI-Brookings Joint Center for Regulatory Studies, Working
Paper 04-18 (Washington, DC: March 2005): 44-45.
[39]Richard
K. Betts, “Analysis, War, and Decision: Why Intelligence Failures Are
Inevitable?” Strategic Intelligence: Windows into a Secret World, eds.
Lock J. Johnson and James J. Wirtz. (Los Angeles: Roxbury Publishing Company):
97–99.
[40]The
9/11 Commission Report, “Final Report of the National Commission on Terrorist
Attacks Upon the United States,” 416.
[41]Mark
M. Lowenthal, Intelligence: From Secrets to Policy 2nd Edition (Washington, DC: CQ Press, 2003): 76.
[42]Implementing
prediction markets and other reform measures should not be mutually exclusive.
[43]This
example sidesteps the question of how the initial allocation of shares of this
contract and money are conducted in this hypothetical prediction market. The
following section on market design issues will explore this issue. One way to
do an allocation is to distribute a fixed amount of shares and money equally
among “traders” within the 15 intelligence agencies so that the starting share
price is at
50 cents.
[44]The
Commission on the Intelligence Capabilities of the United States Regarding
Weapons of Mass Destruction (March 31, 2005): 49.
[45]With
a 46 percent likelihood of this outcome, policymakers would have had to decided
if that was a high enough probability to cause them to act.
[46]The
Commission on the Intelligence Capabilities of the United States Regarding
Weapons of Mass Destruction, 74.
[47]Wolfers
and Zitzewitz, “Prediction Markets,” 121.
[48]Betts,
101.
[49]Lowenthal,
“Intelligence: From Secrets to Policy,” 103.
[50]Looney.
[51]The
importance of issuing the “right” contracts is addressed in more depth in the
Market Design section of this article.
[52]Robin
Hanson, Designing Real Terrorism Futures (August
2005): 9. http://
hanson.gmu.edu/.
[53]Chen
and Plott, 9.
[54]Robin
Hanson, “Chapter 6: Foul Play in Information Markets,” Information Markets:
A New Way of Making Decisions in the Public and Private Sectors,
ed. Bob Hahn and Paul Tetlock (Washington DC: AEI-Brookings Press, 2006): 92.
http://hanson.gmu.edu.
[55]Wolfers
and Zitzewitz, “Prediction Markets,” 120.
[56]See Robert J. Shiller, “From Efficient Markets
Theory to Behavioral Finance,” Journal of Economic Perspectives 17, No.1
(Winter 2003).
[57]Malkiel,
“The Efficient Market Hypothesis and Its Critics,” 61. Other economists have
suggested that speculative bubbles are more prone in stock markets because of
restrictions on short-selling; not all prediction markets restrict
short-selling. See also Andrei Shleifer and Robert Vishny,
“The Limits of Arbitrage,” Journal of Finance 52, No.1 (1997) and
Chapter 11 of Surowiecki, The Wisdom of Crowds.
[58]Wolfers
and Zitzewitz, “Prediction Markets,” 119.
[59]Robin
Hanson and Ryan Opera, Manipulators Increase Information Market Accuracy (July
2004, revised): 9. http://hanson.gmu.edu.
[60]Cited
in Wolfers and Zitzewitz, “Prediction Markets in Theory and Practice,” 5.
[61]Robin
Hanson, “Chapter 6: Foul Play in Information Markets,” 92.
[62]Robert
Forsythe et al., “Anatomy of an Experimental Political Stock Market,” 1156–57.
[63]Ibid., 1157–60.
[64] Robin Hanson, “Impolite Innovation: The Technology and Politics of ‘Terrorism
Futures’ and Other Decision Markets,” 4.
[65]Emile
Servan-Schreiber, Justin Wolfers, David M. Pennock, and Brian Galebach,
“Prediction Markets: Does Money Matter?” Electronic Markets 14, No. 3 (September 2004): 9–10.
[66]At
the very minimum, the costs of setting up either a real-money or play-money
prediction market include costs related to developing and fielding a trading
software platform.
[67]See Emile Servan-Schreiber et al., “Prediction
Markets: Does Money Matter?,” 10. NewsFutures, which is a US-based prediction
market, uses play money and awards prizes to the market’s top predictors (top
play money earners) and ranks its “richest” players.
http://us.newsfutures.com/topwin.html.
[68]Wolfers
and Zitzewitz, “Prediction Markets,” 121.
All
statements of fact, opinion, or analysis expressed in this article are
those of the author. Nothing in the article should be construed as
asserting or implying US government endorsement of an article’s factual
statements and interpretations.
