Pollsters Battle it Out in Primaries.--

It's not just Obama v. Clinton today. Among pollsters, it's Zogby v. Insider Advantage and Survey USA.

Zogby has Obama up a staggering 14% in NC and, perhaps more surprising, Obama also up 2% in Indiana.

Insider Advantage shows Obama up only 4% in NC and down 4% in Indiana.

Survey USA has Obama up 5% in NC and down a staggering 14% in Indiana.

My guess is that Zogby will be farther from the final results than Insider Advantage (and perhaps than Survey USA as well). But then, I haven't done any polling in either state.

In the November 2004 general election, Survey USA got the state-by-state presidential results closest of 11 polling organizations studied.

UPDATE: As noted in the comments, Daily Kos has a nice review of 6 pollsters this primary season. Survey USA (4.7% av. error) has a wide lead over 2d place Rasmussen (8% av. error). In fifth and sixth (last) place are Zogby (9.8%) and Insider Advantage (10.1%).

2d UPDATE: In NC, with a large African-American turnout and heavy pre-election voting, most of the online commentators are predicting a huge win for Obama. So Zogby might be right after all . . . .

3d UPDATE, Tuesday evening: Right now Indiana is too close to call and NC is going overwhelmingly for Obama. At least in the closeness of estimates, it appears that Zogby is going to be a big winner as well tonight.

Can you explain why you think "Zogby will be farther from the final results" than the other organizations? That was just sort of a blanket statement there. Is that based on Zogby's (poor) past performance, the other entities' (better) past performance, or some aspect of the polling styles?
5.6.2008 11:27am
MarkField (mail):
SUSA has had, I believe, the best record this primary season too. Zogby has generally been waaay off.
5.6.2008 11:31am
James Lindgren (mail):
My GUESS is based on what I would guess voters in each state will do. Zogby did well in 2004 but not as well as Survey USA. Mark Field (above) says that Survey USA has been good and Zogby bad this year.

I don't know about that, but my unsystematic impression is that Zogby has indeed performed relatively poorly this season. I note that Survey USA averaged off 2% in Ohio and PA, while Zogby averaged 5% off in those two states. Zogby was also the farthest off of the late polls in the NH race (13% lead for Obama).
5.6.2008 11:43am
ChrisIowa (mail):
Furthering the theme of nothing new under the sun, in 1856 the Davenport,Iowa Democrat was publishing the results of its weekly polling of steamboat passengers on the Presidential campaign.
5.6.2008 11:45am
There's a post on this at DailyKos that goes state by state, showing how accurate each pollster has been. Zogby has done really, really badly. SUSA has been terrific.
5.6.2008 11:49am
Blue (mail):
Zogby has been appalling this cycle, just appalling...and I don't think it is by accident or incompetence either.
5.6.2008 11:50am
Rhode Island Lawyer:
Blue, are you suggesting that Zogby is deliberately doing a terrible job? Why would he want to do that - even if he privately favored a particular candidate it would be crippling to his business to intentionally skew the polling to achieve a desired result. I don't see how that makes sense.
5.6.2008 11:55am
John Thacker (mail):
Admittedly, this is a more difficult race to predict. It's not too difficult to get a decent poll result of who people are willing to say that they would support. It's considerably more difficult to both predict:
1) Whom they'll actually support in the privacy of the voting booth,
2) Who will actually turn up to vote out of the people you surveyed,
3) Whom the people who refused to talk to you or you otherwise couldn't reach but who will vote will support.

Regarding 2), the general rule of thumb is that people are pretty accurate when you ask them if they actually voted in other elections, but when you ask them whether or not they'll vote this time, they're unreliable. However, if you throw in someone who excites people who don't normally vote in primaries, it becomes a difficult decision of trying to figure out if no, this time those people who don't normally vote are really motivated by this new person.

This year it appears to be odd because Sen. Obama has a large group of highly committed people who have not participated in previous years, but OTOH Sen. Clinton seems to have an edge in many states in the more casual voter/soft support, as seen by her wins in primaries versus caucuses. (Washington and Texas provide stark examples, with Sen. Clinton significantly outperforming in the primaries and Sen. Obama the caucuses in each state, even though the WA primary didn't count for any delegates.)
5.6.2008 12:00pm
Perhaps Zogby's most egregious polling of the year, as highlighted by Drudge the day of the primary: SHOCK POLL: OBAMA TAKES 13-POINT LEAD IN CALIFORNIA; ROMNEY UP 7...

Clinton won California by 9; McCain by 8.
5.6.2008 12:21pm
I know nothing about scientific polling. All I can go on is past performance. In this area, Zogby has really floundered since around 2002. I used to view his polls as the most reliable. But now I just don't pay much attention.

A friend who does polling has explained to me some of the problems with his methodology. In particular, my friend thinks, Zogby can't work out how to control for /exclusive/ cell-phone (i.e., voters with no land-line) usage from county to county.

I don't have any reason to doubt this. But does anyone have another take?
5.6.2008 12:33pm
BTW--If SUSA is right, this campaign continues, no?
5.6.2008 12:34pm
If Zogby is right, this campaign continues.

At this point it doesn't really matter who wins primaries by how much. It's going to the convention. Or, at the very least, the end of the primaries.
5.6.2008 12:42pm
The Unbeliever:
It's worth noting that in any general survey of polls after recent elections, if you remove Zogby polling from the mix, a lot of the volatility disappears. In fact I'm pretty sure he is the reason got the final prediction wrong in 2004.

Normally I'd say "take what you will from this" and leave conclusions to others, but it's also worth noting that Zogby is unabashedly in the tank for the Democrats, just like he was for Kerry in 2004. Watch some of his interviews on the Daily Show and you start to get an idea of where he's coming from.
5.6.2008 12:51pm
Blue (mail):
I'll be blunt--I think Zogby is whoring out his name to specific candidates in order to try and "spook the herd."
5.6.2008 12:52pm
"In particular, my friend thinks, Zogby can't work out how to control for /exclusive/ cell-phone (i.e., voters with no land-line) usage from county to county. "

on a pure speculative basis, and following the "obama appeals to elites/urbanites/younger people", one would guess the cell phone only crowd disproportionately skews obama. they are not quite "early adopters" at this point ... cellphones are hardly new technology, but you could probably draw nice contrasts between cell phone only people, and those with landlines and/or exclusively landlines.
5.6.2008 12:55pm
A. Zarkov (mail):
The surveys don't have a sample size problem, they have a sampling frame problem and non-response problem. The sampling frame population must be representative of the target population who are the people who will actually vote. Telephone surveys don't necessarily reach even the population of likely voters let alone those who will actually show up and vote. Then there is the problem of those who respond as an undecided (actually a type of non-response). I don't know the specific details on how they handle these problems because the companies make such things proprietary. Nevertheless I'm not impressed with their results. My own prediction of the 2004 presidential popular vote was more accurate than any of the published polls I saw. I used a Bayesian approach which is a little offbeat, but gives superior results.

It's worth noting that the world of survey sampling exists apart from the statistics world. They have their own methods and terminology and theories. It's a bit of a hodge podge.
5.6.2008 1:30pm
MarkField (mail):

it's also worth noting that Zogby is unabashedly in the tank for the Democrats, just like he was for Kerry in 2004.

Except that (a) the issue is the Democratic primary, so any such bias seems to be irrelevant; and (b) Zogby has been heavily criticized for his inaccuracy even (maybe even "especially") by kos (from today: "it continues to spit out weird-ass numbers").
5.6.2008 1:58pm
The Unbeliever:
MarkField, I meant to say he can in no way be thought of as an impartial observer. Besides which, I think he actually came out and endorsed Obama on the Daily Show, but I can't get the clips to work at the moment to confirm my notoriously bad memory.

I hesitate to admit agreeing with Kos on anything, so I will facetiously correct him: "weird-ass numbers" is putting it much too kindly when referring to Zogby polling.
5.6.2008 2:11pm
MarkField (mail):

MarkField, I meant to say he can in no way be thought of as an impartial observer.

That may very well be the reason his results have been so poor this cycle (and maybe last cycle too). But it's odd, if that's the case. As I said, he's polling a Dem primary, so "mere" partisanship wouldn't seem to affect the outcome. Maybe he's not just partisan but biased in favor of particular candidates. In that case, though, his inaccuracy causes him to be dismissed even by supporters of the same candidate (kos, for example, supports Obama, as do I).
5.6.2008 2:19pm
Zogby has actually done really well in the past couple of contests: they were right on in Pennsylvania, and very close in Texas. Apparently this is because their different method of polling is more accurate for these later contests. I'd bet the results end up pretty close to Zogby's predictions.

here's a link.
5.6.2008 2:22pm
Syd Henderson (mail):
In one of their recent polls in North Carolina, Insider Advantage was counting Blacks in their proportion as part of the general population rather than as part of the Democrats, and underestimated Obama's support. That's why they had Clinton up when everyone else had her down by 7-14 points.

My guess is Obama takes North Carolina by 10 and Clinton takes Indiana by 6-10.
5.6.2008 3:39pm
Syd Henderson (mail):
I should mention that North Carolina had a lot of early voting, which helps Obama since he was ahead by 20 points in the polls for a good part of that. He should do better than the exit polls.
5.6.2008 3:41pm
LM (mail):
I have a slightly OT question. If my pro-Obama bias is blinding me to reality, please let me know. Because when I look at the last few months of Democratic primaries, I see a manufactured news event of Hillary's reported resurgence.

The trend since sometime in February when Obama was first declared to have a virtually insurmountable lead in pledged delegates has been for almost every primary to follow form. Obama wins the states he's supposed to, and he cuts into Hillary's early lead in the states she wins (Texas, Ohio, Pennsylvania). Yet Hillary is reported to have staged one comeback after another, and Obama is repeatedly characterized as failing to close the deal. In reality he's done exactly as was predicted when the foregone conclusion of his final delegate lead became generally acknowledged, and better than predicted in terms of his margins of victory and defeat.

Hillary's election day results have generally been a few points higher than the final ingoing polls. I could see the news outlets accounting for that with any or all of the following:

1. Undecided primary voters are generally more pro-Hillary than pro-Obama;

2. There's a modest Bradley effect;

3. Rush's campaign to skew the primaries toward Hillary with Republican voters may be adding a point or two to her results.

And despite those modest spikes, the longer trend in each primary was clearly pro-Obama. For example, Hillary was polling a double digit lead in the Texas primary, she ended up winning by two or three points, and that was heralded as an heroic comeback. In Pennsylvania she was ahead by 18-22 points, but winning by 9 purportedly signified a dramatic upturn in her chances for the nomination. How can the media keep breathlessly reporting these results as come-backs when she's won exactly was she was expected to win, and by smaller margins than she was expected to win them?

Put aside the Wright, Ayers-type issues, which understandably make good sport for Obama's opponents, splashy news, maybe even make plausible arguments to super delegates about perceived momentum and other intangibles. But I'm talking here about primary results to date only.
5.6.2008 8:29pm
arbitraryaardvark (mail) (www):
My predicton: Obama up 14 points in NC, down 4 in IN, so Zogby's doing as well as the others. How is the Guam recount going?
5.7.2008 12:09am