pageok
pageok
pageok
When Do Prediction Markets Beat the Common Wisdom?:
Over at the Chicago Faculty Blog, Cass Sunstein has an interesting post on prediction markets. Commenting on the broad coverage of the fact that prediction markets didn't predict that Hillary would win the New Hampshire primary, Cass notes the possibility that skepticism about prediction markets may result from the availability heuristic. That is, when a prediction market predicts a low probability of something and that event occurs, observers may see that as the prediction market not "working" rather than it being an instance in which a low probability event did indeed occur. He concludes:
To be sure, we are continuing to obtain information about how prediction markets perform and when they do well and poorly. Perhaps they will turn out to be less reliable than they seem -- and in all likelihood, we will obtain a better understanding of when they work. And of course no one has a crystal ball. But . . . if you want to have a sense of the probabilities, you'd probably do best to consult Intrade.
  Perhaps that's right. At the same time, I would think prediction markets will tend to beat the common wisdom only when there are people actively participating in the market who have better knowledge than a reasonably informed member of the public. And for the most part that usually won't be the case when the future event is a subject of intense media speculation like an election.

  The problem with prediction markets in elections is that the media and the public are already obsessed with predicting the outcome of elections. Everyone loves the horse race. Those who have particularly deep insights into who will win elections -- assuming those people exist -- already are eager to share them with the world, and the media is eager to broadcast their views.

  If the market for experts is competitive, the experts who are the best predictors will end up having the most impact on the common wisdom about the election. In that setting, there's no particular reason to think prediction markets will beat the common wisdom instead of just reflecting it.
SIG357:
Isn't "predictions market" just a polite term for what we used to call "bookies"?

In any case I agree that they have no useful insight about political races. The people making the bets base them on the polls and their own gut feelings.
1.12.2008 3:10pm
formerbeltwaywonk (mail):
That's the wrong question. The right question is "how do we know what the common wisdom really is?" That is what prediction markets and polls each in their own ways can tell us.

Sunstein has also pointed out that, especially in these days of media fragmentation, the "common wisdom" of one particular social community can diverge radically from the "common wisdom" of another one. An example we are currently witnessing: Beltway people have no clue what is going on in the minds of Ron Paul supporters, and vice versa. Markets and polls allow us to aggregate these widely divergent and mutually ignorant views into estimates of overall "common wisdom".
1.12.2008 3:46pm
Thoughtful (mail):
There is an excellent book, Expert Political Judgment: How Good Is it? How Can We Know, by Philip Tetlock (who's an expert in these things...
:-> ), which argues convincingly that most experts are wrong in their predictions most of the time.

I always thought the difference between "experts", whom you can see on major and cable news channels all year pontificating, and prediction markets is that the experts on TV are not at any financial risk if they're wrong. Why, they even get continually invited back; that is, they don't even lose "TV expert" income for being wrong. While, of course, those investing in prediction markets are confident enough of their own opinion, for whatever reason, that they're willing to bet their own money on the outcome. This distinction still holds even when all the points made by Orin are true, does it not?
1.12.2008 3:58pm
Sean O'Hara (mail) (www):
The market wasn't wrong; it underwent a correction.

Seriously though, Clinton and Obama tied in delegates, so the market wasn't totally wrong in predicting he would win.
1.12.2008 4:11pm
Sasha Volokh (mail) (www):
The key assumption is that better predictors have a greater effect on the common wisdom because the market is competitive. Is there reason to think this is true? It's not clear that bad predictors are punished for this; rather, the race may go to those people who are effective communicators, good writers, and producers of effective sound bites, which may be uncorrelated with good predictive skills.

Relatedly, I recall reading that when it comes to predicting the value of stocks, buying a diversified sample of the market did better than following the most-quoted experts. That's old news, of course, but the interesting twist was that those experts who did better at prediction systematically did worse at getting quoted by the media because they hedged their predictions with lots of qualifications rather than brashly predicting something spectacular.
1.12.2008 4:21pm
Mr. Liberal:
I think the word "prediction market" is misleading.

Better, would be information aggregation market.

In computer science, there is a well-known acronym. GIGO. Garbage-In, Garbage-Out. Basically, the idea is that there are no processes that are going to take garbage input, and somehow output useful information.

Now, I would not go so far as to say the inputs into a prediction market is so bad as to constitute garbage. And it is in fact sometimes true that combining the results of multiple artificial intelligence algorithms can result in more accurate predictions. But, when you have a lot of bets on prediction markets that are not any better informed than the conventional wisdom, you are not going to expect the aggregation of those bets to be better than the conventional wisdom.

Of course, there is the idea that if the conventional wisdom is wrong, there is a hugely profitable arbitrage opportunity. The question is, why wasn't their some genius arbitrager who knew that Clinton was going to win before everyone else, who then made money on that superior information?

Basically, a prediction market is never going to be more accurate than the best informed arbitrager who is able and willing to put his own money at risk on this beliefs about the future.
1.12.2008 5:47pm
Mr. Liberal:
I should ad one more sentence.

Basically, sometimes that best informed arbitrager who is able and willing to put his money at risk based on beliefs on the future just isn't going to know much more than the conventional wisdom.

That is what explains the failure of prediction markets with respect to the New Hampshire primary.
1.12.2008 6:04pm
Alan Gunn (mail):
There's a good post on this, and some interesting comments, on "Overcoming Bias": www.overcomingbias.com The discussion there is somewhat more technical than Sunstein's.
1.12.2008 6:34pm
Will Conway (mail) (www):
This is a problem, and we're working on it... (I love that commercial)
1.12.2008 6:48pm
bla bla (mail):
I think Cass was right. On the morning of the primary, 9 or 10 different polls all had Obama leading by more than the margin of error. For some of the polls, the lead was 10 or 12 percent. Everyone knows that polling is an imperfect science, especially because it's difficult to predict who is likely to vote, but what were the chances that all of the polls were wrong? Pretty low, I'd think. To add to that, reports on the campaigns themselves tended to state that Obama's crowds were large and boisterous, while Clinton's crowds were said to be reacting with ambivalence to her and Bill. Finally, Obama supporters were supposed to be giddy, while Clinton's supporters were said to be sullenly preparing for a loss and shakeup. Given all of that information, it seemed pretty unlikely that Hillary would win, and the market's assessment of the situation may have been essentially correct given the available information.
1.12.2008 7:17pm
TruePath (aka logicnazi) (mail) (www):
I think part of the problem with the media is what sort of theories/opinions it favors. The media rewards the most interesting theories or stories. If you give an explanation in terms of a class that people find interesting (young people, gender, race) or the political bell weather of the moment (soccer moms) it's probably going to attract more interest than a more boring or technical explanation. On the other hand the prediction market favors those theories that are most likely to be right (or at least those which the people investing money believe are most likely to be right).

I think a huge reason the prediction markets end up working better than other forms of information aggregation is the cost of irrationality. As the "myth of the rational voter" convincingly argues people often prefer to believe certain irrational things but will give up those beliefs when they cost too much. I suspect we see this in political pundantry (even if the beliefs aren't partisan favoritism) and the prediction markets minimize the irrationality in prediction.

Of course they still can't do better than the data they receive but assuming the conventional predictions embody closely held irrationalities then the prediction markets can do better than that.
1.12.2008 7:18pm
Paul Allen:
Something subtle is going on here. I don't think free-market supporters ever claimed that a market mechanism beats the common wisdom. This is a straw-man that individuals of a certain persuasion have latched on to in order to discredit the idea that 'markets work'. This trouble arises from confusing the common wisdom with the wisdom of the common man.

Second, the 'political markets' are quite different beasts from real financial markets. Political markets are toys (leisure) whereas real financial markets implicitly weight people's influence over the price by their historic success (how much money they have and thus how much volume they produce). One man one vote creatures they are not.
1.12.2008 7:53pm
Jerrod Ankenman:

Those who have particularly deep insights into who will win elections -- assuming those people exist -- already are eager to share them with the world, and the media is eager to broadcast their views.


...and the views of a bunch of people who have no better information than anyone else, but are skilled at making it appear that they do.
1.12.2008 8:05pm
Loophole1998 (mail):
I'm not clear what you mean by "common wisdom." I take it to mean the popular view of individual experts (i.e., talking heads, pundits).

You surmise that the prediction market will not outperform "conmon wisdom unless" individuals with "better knowledge" (i.e., inside traders) are participating in the market.

This conclusion reckognizes one of the advantages of a prediction market, which is the inclusion of "insider" knowledge.

But it ignores another, more significant, advantage of prediction markets, which is the power of collective thought. There is a theory that prediction markets work because the consensus views of a large group are more reliable than the individual views of experts. I am not in a position to explain the logic behind the theory, but it needs to be acknowledged when considering whether prediction markets work.

Removing the participants with "better knowledge" from the prediction market does not remove the "collective thought" advantage, so a prediction market should still outperform "common wisdom."
1.12.2008 8:17pm
SIG357:
I think a huge reason the prediction markets end up working better than other forms of information aggregation is the cost of irrationality.




Is there any evidence that they do work better? They seem to have a worse record at political prediction than the conventional polls.
1.12.2008 8:34pm
SIG357:
How much of what these "futures markets" do is an attempt at prediction, and how much is an attempt at controlling the future? Perception is a large part of reality in politics. Any sensible campaign with some money to on hand might want to make it look as if they are the wave of the future.

That brings up a slightly off-topic question I've been wondering about - to what extent do normal polls skew the results of the political process? If everyone did not know that candidate X was polling better than candidate Y, how often would candidate Y win? There seems to be a large degree of measurement induced change in the results. It's a topic for some bright sociology researchers to look into.
1.12.2008 8:43pm
TruePath (aka logicnazi) (mail) (www):
SIG357:

I'm not sure if there is. Perhaps I should have been more careful and said one way they might work better.

That's an interesting point about the cost-effectiveness of buying the top spot in the intrade markets. I'm not sure if these markets attract quite enough interest yet for that to be a good deal but if they start receiving more media attention that certainly is an interesting question.

But even if you are a partial cause of the result doesn't mean you didn't predict it.
1.12.2008 8:47pm
Guest 1L:
SIG357, &Logicnazi: A paper on manipulating political futures markets (apparently it's not so easy):
http://www.unc.edu/~cigar/papers/ManipHIT_Jan2007.pdf
1.12.2008 10:02pm
Recovering Law Grad:
"there's no particular reason to think prediction markets will beat the common wisdom instead of just reflecting it"

I don't think anyone argues that "prediction markets" exist to "beat the common wisdom." They simply reduce to a number the collective wisdom of those participating in the particular market. If there are an insufficient number of participants or if there is an insufficiently diverse group of participants or if a host of other conditions aren't met, then the resulting price won't be very helpful.
1.12.2008 11:00pm
Lev:
So how many people from outside NH were bused into NH to vote? Would the polls account for that? Would the prediction market?
1.13.2008 12:35am
OrinKerr:
Paul Allen wirtes:
I don't think free-market supporters ever claimed that a market mechanism beats the common wisdom. This is a straw-man that individuals of a certain persuasion have latched on to in order to discredit the idea that 'markets work'.
Really? I don't think that's right. In this instance, for example, I'm much more market oriented than Sunstein, and yet he's much more an enthusiast of prediction markets than I am. Of course, if the claim is that prediction markets are just serving a price function of aggregating net values, then I don't think anyone disagrees that they do that. But if so, we should stop calling them prediction markets.
1.13.2008 1:07am
American Psikhushka (mail) (www):
Kerr summarizing Sunstein:

That is, when a prediction market predicts a low probability of something and that event occurs, observers may see that as the prediction market not "working" rather than it being an instance in which a low probability event did indeed occur.

Or, the observers criticizing the prediction market could be quite right and the market miscalculated the probability of the "low probability" event occurring in the first place. Perhaps the predicted probability for Clinton should have been much higher. Financial markets routinely miscalculate the probability of "low probability" events occurring.
1.13.2008 2:03am
David Prince (mail) (www):
As I understand them, the "prediction" markets allow you to bet on the outcome of a political race. So, while you may want one candidate to win, you would be most likely to wager on who you actually think has the best chance of winning, whether your preference or not.

When calculating a potential winner, you have to use the information available to you. In my case, being in the South, I have the news media, the polls, and my acquaintances to gather information from. I do not have access to the same data as people voting in a New England primary.

I assume that the price of a candidate is determined by the number of people who believe, based on their individual sources of information, that the candidate will win.

It makes sense that the markets should echo the information available to the people who wager.

In the case of New Hampshire, it would have been interesting to have had a market that was open only to eligible New Hampshire voters. I believe it would have been more predictive of the outcome because those are the people with the best information.

I think the markets have an advantage over the polls because they are confidential. I can wager without having to publicly air my view. I am also making a decision on who I think will win, as opposed to who I am voting for.

I think polls are invalid to some extent because there are, as far as I know, no built in validity scales of the type found in psychological tests such as the MMPI. These types of questions are designed to catch random answering and lying. It would be interesting if something could be developed for pollsters to help them determine if their results are valid. If these exist, I'd certainly appreciate some education. The research I've done shows psychometricians discussing validity scales with regards to polling for at least 50 years.

My belief is that the prediction markets are better than experts because they aggregate more information, and I think they are better than polls because there is no reason to lie, and decisions are based on most probable winner(from the gambler's perspective) instead of most favored candidate.

But the key is the quantity and quality of information available. If I were betting on say, the South Carolina primary, I would want to know who the people in South Carolina are betting on. I think they probably have the best information. (If I were a Democrat, I'd want to know who Diebold was betting on. ;))
1.13.2008 3:55am
Toby:
Somehow this thread makes me think of the company I recently heard about that is setting up on-line bidding for weather futures...
1.14.2008 12:52pm
formerbeltwaywonk (mail):
a prediction market is never going to be more accurate than the best informed arbitrager who is able and willing to put his own money at risk on this beliefs about the future.

This is silly. A liquid market will reflect and aggregate information from many such arbitragers with many different sources of information.
1.14.2008 9:12pm