Non-Gallup/Rasmussen tracking polls thread

[J. J.]

IDB/TIPP:         -2.6
Gallup:             -4.9
R2K:                 -0.9
Zogby:              -4.1 (It's Zogby!)
Hotline:            -1.9
ABC/WP:          -3.9
Ras:                 -0.4
Ipsos:              -4.9

Every one of these polls under projected McCain, though many were in the MOE.  Obama was undercounted in some, but not all.

Okay, now I'm about ready to start banging my head against the wall too.  The only number that's worth anything whatsoever from a poll that doesn't either push all the leaners or allocate all the undecideds is the margin.  Under no circumstances should you be comparing the absolute percentages of any single candidate in a poll which has undecideds with those from an actual election which doesn't.   This isn't a Bradley effect thing, this isn't a methodology thing...this is the obvious fact that even the best poll which only allocates 94% of the voters will undercount McCain. 

The way to "fix" this undercount isn't to give up and claim that other pollsters are better.  It's to stop looking at the McCain totals and focus on the margin instead!

Then we shouldn't be getting positive numbers in any case, but we are in some cases involving Obama, or other candidates that are black.   We did on ABC/WP.   We came close (and might have with rounding) on Gallup.  I can understand Zogby.

I could understand everybody being undercounted, but in some cases, black candidates are being overcounted.

[J. J.]

One thing to watch out for:  The CNN numbers you gave add up to 100%, so I assume you're completely ignoring third parties.  The polls don't. 

I am currently, but it shouldn't change the gap too much.

You must be logged in to read this quote.

I will expect the Obama/McCain number to increase.

Just to be clear, you are the one that cited Ipsos, initially.  It isn't one that we generally look at on this thread.

I am suggesting that in races where there are white and black candidates, the white candidate tends to underpoll.  This was historic and can be seen, arguably in the Bradley (1982), Wilder (1989) and in Blackwell, Steele, Patrick  (but not Ford) in 2006.  It has dropped rather dramatically from 1989 to 2006, but it's still present.

In this race, at least on the national polls, it seems to be present, roughly like 2006.  It's there, but it's dropped.  It may be lessened because of something that some pollsters do.

[J. J.]

It looks like it occurred in some states, UT, AR, GA, WV , where there was a strong McCain vote.

Two other possibilities are CA and more probably NY.  It didn't make a difference, but it's present.

IA, ironically, seems to have been a Bradley Effect state. 

The national gap is currently 6.38

[J. J.]

I guess if you keep narrating authoritatively...whatever.

Obviously the best explanation for Utah polling (of which there was little, and virtually none of it was respectable) is the Bradley Effect.  Utah, a state known for its rich history of racial tension. Utah.

Good god.

Utah has one characteristic that it shares with Iowa; both are very white states.  I've got to admit, I was surprised at Iowa, considering Obama's electoral record there and McCain's stand on ethanol.  It is the underpolling, which is strange for a Republican in Utah.

And what made you think it has anything to do with racial tension?  I also looked at MA, VA, and NC, where it didn't occur.  They have had a "history" of racial tension.

One of things I indicated about 2006, we don't have across the board polling in state polls.  We do have the nationals, and you see the results.

Do you know anything about Hotline's polling methods?

[J. J.]

No, but I do know that defaulting to the Bradley Effect in every instance is dumb.

Perhaps the lack of any racial tension correlation whatsoever is indicative of something, hmm.

Yes, no black people in the state (well very few).

We have the nations polls, and frankly Gallup is behaving like Zogby.  We have two where Obama underpolled.  I'm interested in what, if anything, is in common between those two that isn't in common with Gallup or ABC/WP.

[J. J.]

Statistically Obama should overperform his poll margins 50% of the time

Actually, Obama should over perform about 1 in 20 times out of the MOE.  We have Zogby numbers on Gallup and ABC/WP.  Come on, a 2 point MOE and the polls are off 4.5 points.

We know two things about Rasmussen, that make it different from Gallup and ABC/WP, it pushes for answers, and it uses robocalls.  I'm wondering if Hotline does one (or both) of those things.

I frankly was expecting a slight overpolling for Obama, but within the MOE.  I would have expected TIPP like numbers across polls.

Does anyone actually have any information on how Hotline conducts it polls.

[J. J.]

As I mentioned above, Hotline, along with most of the firms the were within 1 point of the correct margin (Rasmussen being an exception), does not use robot polling. 

Okay, do they push as strongly as Rasmussen (or more strongly than Gallup)?

[J. J.]

Statistically Obama should overperform his poll margins 50% of the time

Actually, Obama should over perform about 1 in 20 times out of the MOE.  We have Zogby numbers on Gallup and ABC/WP.  Come on, a 2 point MOE and the polls are off 4.5 points.

1. Statistical error does not include artificial sampling error, which obviously exists in every poll.

You're going have to go into greater detail.

You must be logged in to read this quote.

Both Gallup and ABC/WP had a 2 point MOE.

You must be logged in to read this quote.

I'm not ignoring that, but this occurs in white/white or black/black races, and we don't get these numbers.

However, the polls that didn't push them as hard as, say, Rasmussen or Pew did in the end, ended up underestimating him at the end by a couple of points.  That's the extent of the Bradley Effect now (not much).

I think the key is "not much." I was suggesting a 1-2 point Bradley Effect, which wasn't much, but present.  It was never, "Obama's polling will be like Doug Wilder's."  In 2006, it looked like 2-4 points in three races.  Maybe, if we could get into the minds of every voter in MO, it made a difference, but it is impossible to tell.  Right now, I'd say present, but only 1-3 points.  It also looks like it is declining over time.

I'd really like to know more about Hotline.

You must be logged in to read this quote.

Gallup and ABC/WP are now less than a half point off Zogby.

[J. J.]

1. Statistical error does not include artificial sampling error, which obviously exists in every poll.

You're going have to go into greater detail.

MoE assumes a perfectly representative sample.

It could be a weighting problem, but they would have to hugely overweight Democrats.  R2Kos did, but they got a better result than Gallup.

[J. J.]

Statistically Obama should overperform his poll margins 50% of the time

Actually, Obama should over perform about 1 in 20 times out of the MOE.  We have Zogby numbers on Gallup and ABC/WP.  Come on, a 2 point MOE and the polls are off 4.5 points.

Didn't Obama constantly overperform his polling number in MANY states by >4%?

What do you attribute that to?  Why is it that every time he underperforms it's because people are afraid of being racist to the interviewer?

Actually, no on the national polls.  Do you have some state polls were this this occurred?

I found a one, someplace, where Obama and McCain both over performed their poll numbers out of the MOE.  I did attempt to look at that as well.

[J. J.]

Pennsylvania?  Nevada?  New Mexico?  Michigan?


He beat the MoE of the polls in all of these states.  Why is it the fricking Bradley Effect whenever the polls are wrong in the inverse?

*hand trembles*

PA:

From the 30th two Rasmussen, 1 ARG and a crap tracking poll  out of the MOE, 3 out of 12.

NV:

3 polls in the last week, two were out of MOE one was on.

NM:

3 polls in the last week, two in MOE, one out.  One overcounted Obama.

MI:

Five polls in the last week.  4 were in the MOE.

*suggest Lunar puts down the crack pipe*

[J. J.]

Lunar, don't bother.  please.

*suggest Lunar puts down the crack pipe*

Ugh.

Alcon, if Lunar's going to make the comments, I'll make the comments.

So when McCain underpolls within the MOE it's the Bradley effect, when Obama underpolls within the MOE, it's just the MOE. Ok, I got it.

No, we're getting some that are outside of the MOE even at the state level.  We definitely do at the national level, beyond question.

[J. J.]

So if 9 out of 12 polls are out of MoE, it's nothing notable and these polls have a solid methodology.  But if a few polls that show the inverse, it's caused by race.

Lunar, in PA, most of the polls undercounted both candidates, but all of them did so in the MOE.  That's why I'm saying that there was not a B.E. in PA.  I frankly expected there to be one, but it wasn't there.

In MI, we have one poll where Obama did better than the MOE, out of five.  It that poll McCain did worse that he did in the election.  I'm not calling that a Bradley Effect either.

In NM, one poll had McCain one point lower and Obama one point higher than actual.  I'm not calling that a Bradley Effect, because the other polls don't show it.  There is not consistent overcounting.

In NV, you have the only state where Obama was consistently undercounted.  On the last poll, McCain was undercounted as well (I'm wondering if the Latino vote might be in play there).

In other states, there is consistent overcounting and/or results out of the MOE.

[J. J.]

So, 538's poll average-based model doesn't show the effect you allege because...?

1.  Would you provide a link?

2.  Would you explain why, in terms of national polls, none were outside the MOE, except those showing a lead for Obama?

[J. J.]

1.  Would you provide a link?

fivethirtyeight-dot-com

They weight by sample size, and time elapsed since release, and pollster record.  But unless you can explain why doing so removes the Bradley Effect, your suggestion that Obama under-performed vs. state polls is unfounded.

Please provide a link.  The only thing I've found was this:  http://www.newsweek.com/id/165030/page/2

You must be logged in to read this quote.

n=2?
[/quote]
[/quote]

n=3, if I understand your point.

We've been looking at a number of tracking polls:

Zogby (understandably)
Gallup
ABC/WP
(These overcounted and are outside the MOE)

TIPP
(overcounting, but inside the MOE)

Rasmussen
R2K
(Undercounted, but inside the MOE)

[J. J.]

Sigh.

fivethirtyeight-dot-com = www.fivethirtyeight.com

C'mon man

The section the particular point you are making, "538's poll average-based model."

The Supertracker?

[J. J.]

Sigh.

fivethirtyeight-dot-com = www.fivethirtyeight.com

C'mon man

The section the particular point you are making, "538's poll average-based model."

The Supertracker?

"Trend-Adjusted" under the right-hand side.  Did you bother to read the FAQ, or look through the entire page?  It's pretty self-explanatory.

Yes, that method is used with the Supertracker.

Well, one difference is that I look at only the last week of polling, in the case of tracking, final polls.  I treat polls as a snapshot of the electorate and I want the photo to be close to the event.  So that would be one difference, probably the main one.

A also look for trends, basically where McCain underperformed and Obama didn't under preform, or where Obama was in the MOE and McCain wasn't.

[J. J.]

Wait, so the Bradley Effect was contingent upon the place in the election cycle?

Besides, the weighting was done in such a way that polls in the last week were hugely over-weighted relative to earlier ones.

Alcon, what I am looking at is basically the number of voters who don't answer the polls accurately.  In many cases, they answer undecided.  There were also true undecideds out there, but they tend to make up there minds before election day.  For this reason, I look at ending polls and really don't want to to include polls that are 30 days old, or 15 days old, or 10 days old.

There were also legitimate shifts in opinion within the last 30 days.  PA is a good example.  The polls closed about a week out, then bounced back.

No weighting just the last snapshot.

Gustav,  in the national tracking, there was.  We had three national tracking polls (out of six) that over counted Obama and did so outside outside of the MOE.  Two of them were not Zogby.    I suggested that there would be a small Bradley Effect, 1-2 points.  That small effect seems to be there.  Now, did it make a difference?  No.  Was it a polling artifact?  Yes.  A large one? No.  An effect that has been measured over time and is declining?  Yes.

If I had to say what it was, at this point, 1-3 points, in this election.

[J. J.]

We should remember in the midst of glorifying 538 that all his talk about cell phones and young voters emerging turned out to be bull.

True, but not related to the model.

In any case, I guess we're just not understanding J. J. on the following:

1. Weighting based on pollster quality, age and poll conduct time somehow gets rid of the Bradley Effect, in a way I assume J. J. will refuse to explain.

No, I'm suggesting that looking at two week old polls isn't that good.  I have explained the snapshot effect.  Going back, even with reducing the weighting, of polls 2-4 weeks ago, might show trending, but not the snapshot.

You must be logged in to read this quote.

So far, we've had one state where Obama overperformed and more than a half dozen McCain overperformed.

You must be logged in to read this quote.

No, I've been looking both national tracking polls and state polls.

You must be logged in to read this quote.

What makes you think that it's a "racial tension" issue.  You are assuming something about the Bradley Effect that has never been claimed.

You must be logged in to read this quote.

I didn't have up the national polls.  It is intellectually dishonest to make claims about "racial tensions" or claims that I'm not looking at state polls, when I've been discussing them.  :rolleyes:

It certainly is intellectually honest when I said, prior to the election, I thought PA could exhibit a Bradley Effect, and now say that the data doesn't show it.

[J. J.]

Didn't you suggest that PA is likely to have the Bradley Effect because of racial tension?  (Caveat:  I was wrong to say that Utah has never had racial issues)

No, the only thing I suggested was that it might be cultural.  Phil, Ice Hockey, and a slew of others (Murtha) basically said the same things.  It was more of "they're rednecks."

You must be logged in to read this quote.

Okay.  All polls from 10/28.  We can argue if the poll is complete crap or not later. 

I'm looking at:

1.  Polls where McCain underpolled and Obama didn't underpoll.  For example:

Poll A:  Obama 48, McCain 44.  Actual result:  Obama 52, McCain 46.  This result would not show a true underpolling.

Poll B:  Obama 52, McCain 40.  Actual result:  Obama 52, McCain 48.  This result would show a true underpolling.

2.  Polls where McCain polled below MOE.

We'll start with that.

I'm looking for a polling artifact.  Except in really close races, it shouldn't make a difference in the result.

I really thought that we would not see it in national polls, but we did.  It didn't occur in some states where I thought it would occur, either.

[J. J.]

No, you can tell me which polls are "complete crap" now.  You don't do an experiment and then toss out data points afterwards.  Why would you?  That serves no purpose whatsoever other than to potentially introduce bias.

Alcon, I'm saying that I'm willing to look at it in the opposite direction.  You can have a poll that is terrible, but can still show an accurate result.  I'm will to say, even though I don't really think that poll is not particularly good, it got it right.  It's only going to be when that poll shows an undercounting for McCain, that I might we willing to say, "Well this might be a polling problem."

Let me put it this way.  In PA, we have Marist, which wouldn't be counted because it was out 10/27, but let's say it came out on 10/28.  It didn't undercount Obama, but did McCain.  It fits my criteria otherwise.  You could look at that poll and say, "Here is an example."  I'd be will to look at that and say, "That is not a particularly good poll." 

In other words, if the effect seems to be present, I'd say, "Let's go back and look at the specific polls."

If this shows up, I'd also like to look at margins, but let's look at this first.

[J. J.]

That's fine, J. J., but I'm not going to let you throw out polls after I do the analysis, lol.  That would be incredibly unscientific.  My argument isn't with the concept that "some polls are crappy," it's with ex post facto toss-outs.

Are you willing to list which polls you consider acceptable so as to look at this objectively, or not?

No, I'd to look at every state and research every poll.  I want to look to see which states seem to have a B.E. and then see if some of those polls can be factored out.

If we were looking at the seven national tracking polls, for example, we'd get four polls that undercounted McCain, three that had it out of the MOE, the that undercounted McCain and none that undercounted him outside of the MOE.

We might discuss matters and say that we shouldn't count Zogby and Y2Kos (possibly due to the weighting, and I still don''t get how it undercounted Obama).   We'd then have 2 outside of the MOE, one inside underrcount, and two overcounts.

[J. J.]

Alcon, look at the last week of polling.  That would be step one.

Looking at the polls afterward will probably be favorable to the proposition that there is no Bradley Effect.  Now, if you don't want to do it, fine, but I don't want here, "Oh, you have to look at these bad polls," afterward.

On this thread, I've noted that the national tracks show something with Zogby, and posters get annoyed because I'm including Zogby, then I remove Zogby, and I still hear, "Well what about these other polls."

[J. J.]

I can include the national trackers too, if you like, but in proportion to their interviews.  Again, there is no logical reason reason to weight them any more strongly than state polls with an equal number of interviews.

I don't want to throw out polls afterward no matter who it is favorable to.  It is unscientific.  It will enter our own experimental bias into the mix.  There is no reason whatsoever not to throw them out beforehand.  Which of the listed polls do you want to remove?

The fact that you're defending something by arguing I should do it because it might show the results I like, is exactly why people are skeptical of your analytical abilities.

I'd like to look just at state polls for now; the tracing polls were just an example.

Okay, look at all the polls in a given state from 10/28 onward, or you can limit that to 10/29.

[J. J.]

All right.  Again, which pollsters do you want to throw out of the above list?

And again, none.  Let's look at the last week of state polling, from 10/28 onward.  The only ones (and I can only think of one of these) that shouldn't be counted would be state tracking polls.