Non-Gallup/Rasmussen tracking polls thread

[Grad Students are the Worst]

I can include the national trackers too, if you like, but in proportion to their interviews.  Again, there is no logical reason reason to weight them any more strongly than state polls with an equal number of interviews.

I don't want to throw out polls afterward no matter who it is favorable to.  It is unscientific.  It will enter our own experimental bias into the mix.  There is no reason whatsoever not to throw them out beforehand.  Which of the listed polls do you want to remove?

The fact that you're defending something by arguing I should do it because it might show the results I like, is exactly why people are skeptical of your analytical abilities.

[Sam Spade]

This thread is hilarious.

[J. J.]

I can include the national trackers too, if you like, but in proportion to their interviews.  Again, there is no logical reason reason to weight them any more strongly than state polls with an equal number of interviews.

I don't want to throw out polls afterward no matter who it is favorable to.  It is unscientific.  It will enter our own experimental bias into the mix.  There is no reason whatsoever not to throw them out beforehand.  Which of the listed polls do you want to remove?

The fact that you're defending something by arguing I should do it because it might show the results I like, is exactly why people are skeptical of your analytical abilities.

I'd like to look just at state polls for now; the tracing polls were just an example.

Okay, look at all the polls in a given state from 10/28 onward, or you can limit that to 10/29.

[Grad Students are the Worst]

All right.  Again, which pollsters do you want to throw out of the above list?

[J. J.]

All right.  Again, which pollsters do you want to throw out of the above list?

And again, none.  Let's look at the last week of state polling, from 10/28 onward.  The only ones (and I can only think of one of these) that shouldn't be counted would be state tracking polls.

[Grad Students are the Worst]

All right.  Again, which pollsters do you want to throw out of the above list?

And again, none.  Let's look at the last week of state polling, from 10/28 onward.  The only ones (and I can only think of one of these) that shouldn't be counted would be state tracking polls.

I don't see why not?  State tracking polls are just three-day polls that roll.  In its last instance, Muhlenberg was just a three-day poll.  Why not use it?

Other than that, I still need you to pick one of the following:

Either pick margin-vs.-final (margin minus final margin) or relative candidate-showings-vs.-final ([Obama minus final Obama]-[McCain versus final McCain]).  Then, I will run a poll compilation and test for statistical significance.

Then we'll get to 'er.

[Torie]

So have we figured out yet why McCain did a couple of percent better than the polls in general, apparently?  You know what?  We might not really know. Oh the horror, the horror!  By the way, Obama will be close to a 7% lead before this is all over I suspect.

[Verily]

So have we figured out yet why McCain did a couple of percent better than the polls in general, apparently?  You know what?  We might not really know. Oh the horror, the horror!  By the way, Obama will be close to a 7% lead before this is all over I suspect.

Obama is now at a 6.5-point lead, and that will probably still grow a bit with Oregon, Washington, Georgia (Fulton County, a heavily Dem area), New York and Illinois still not fully counted (plus absentees and provisionals).

http://www.politico.com/electionmap2008/index.html

[Lief]

So have we figured out yet why McCain did a couple of percent better than the polls in general, apparently?  You know what?  We might not really know. Oh the horror, the horror!  By the way, Obama will be close to a 7% lead before this is all over I suspect.

Historically, it's common for the candidate with a large lead to overpoll a small amount. Yes, even the whites ones, JJ.

[Grad Students are the Worst]

So have we figured out yet why McCain did a couple of percent better than the polls in general, apparently?  You know what?  We might not really know. Oh the horror, the horror!  By the way, Obama will be close to a 7% lead before this is all over I suspect.

Historically, it's common for the candidate with a large lead to overpoll a small amount. Yes, even the whites ones, JJ.

This is true, and an argument to be had if Obama overpolled to statistical significance.

[J. J.]

All right.  Again, which pollsters do you want to throw out of the above list?

And again, none.  Let's look at the last week of state polling, from 10/28 onward.  The only ones (and I can only think of one of these) that shouldn't be counted would be state tracking polls.

I don't see why not?  State tracking polls are just three-day polls that roll.  In its last instance, Muhlenberg was just a three-day poll.  Why not use it?

I think it has a smaller sample size.  It's also, as a tracker, designed to do something slightly different that a standard state poll.

Other than that, I still need you to pick one of the following:

Either pick margin-vs.-final (margin minus final margin) or relative candidate-showings-vs.-final ([Obama minus final Obama]-[McCain versus final McCain]).  Then, I will run a poll compilation and test for statistical significance.

Then we'll get to 'er.

The first thing I'd like to look at are the states where McCain underperformed on the final set of polls verses those where he over performed (and the same date on Obama). 

I'd then like the subset where McCain underperformed outside the MOE (and Obama underperformed outside of the MOE).

I'll look at margin later.

[Grad Students are the Worst]

No, as a tracker, it's the same as a state poll.  It's still interviews being conducted over a three-day period.  Muhlenberg's three-day sample size was 618, bigger than Insider Advantage, ARG, Zogby, and only 7 fewer responses than Mason-Dixon.  No reason to throw it out.  It's a minor point, though, but unless you know something about Muhlenberg I don't, I have to include it for consistency if we're including all polls.

So, in other words, you want option B?  That's ([Obama minus final Obama]-[McCain versus final McCain]).

And to clarify, you want all polls, not just the recognized national ones I listed?

Once I get your response on that, I'll compile the results and check for statistical significance.

[J. J.]

So have we figured out yet why McCain did a couple of percent better than the polls in general, apparently?  You know what?  We might not really know. Oh the horror, the horror!  By the way, Obama will be close to a 7% lead before this is all over I suspect.

Historically, it's common for the candidate with a large lead to overpoll a small amount. Yes, even the whites ones, JJ.

This is true, and an argument to be had if Obama overpolled to statistical significance.

Actually, that wasn't true in a race I looked at a while back, Casey in 2006.  He underpolled slightly.  Both Swann and Rendell underpolled

[Grad Students are the Worst]

Actually, that wasn't true in a race I looked at a while back, Casey in 2006.  He underpolled slightly.  Both Swann and Rendell underpolled

It is not an exceptionless rule (what is -- ever?).  However, FiveThirtyEight did a table and determined that it does tend to happen presidentially.

You didn't really answer either of my questions.  I can't start the procedure until you do.

[J. J.]

No, as a tracker, it's the same as a state poll.  It's still interviews being conducted over a three-day period.  Muhlenberg's three-day sample size was 618, bigger than Insider Advantage, ARG, Zogby, and only 7 fewer responses than Mason-Dixon.  No reason to throw it out.  It's a minor point, though, but unless you know something about Muhlenberg I don't, I have to include it for consistency if we're including all polls.

Other than being on the top of my list of appallingly bad polls, I would prefer to to compare trackers (in any state) to non trackers.

So, in other words, you want option B?  That's ([Obama minus final Obama]-[McCain versus final McCain]).

And to clarify, you want all polls, not just the recognized national ones I listed?

Once I get your response on that, I'll compile the results and check for statistical significance.

I think that is it, from roughly 10/28 to the final; it would be state by state, of course.  I'll readily agree that it doesn't occur in every state.  After that, I'd like to see where Obama and McCain's final were outside of the MOE.  Plus or minus.

[Grad Students are the Worst]

But trackers are the same as a three-day poll.  Do you understand that?  There is no basis for considering them different entities.  They are the same as a three-day poll.  There's no difference.

Anyway, you didn't answer either of my questions, again.  Do you want all polls, regardless of pollster?  Is option B -- ([Obama minus final Obama]-[McCain versus final McCain]) -- acceptable?

I can even arbitrarily exclude Muhlenberg if you really really want.  Just realize that it makes no sense!

[J. J.]

But trackers are the same as a three-day poll.  Do you understand that?  There is no basis for considering them different entities.  They are the same as a three-day poll.  There's no difference.

Anyway, you didn't answer either of my questions, again.  Do you want all polls, regardless of pollster?  Is option B -- ([Obama minus final Obama]-[McCain versus final McCain]) -- acceptable?

I can even arbitrarily exclude Muhlenberg if you really really want.  Just realize that it makes no sense!

Acceptable but please exclude Muhlenberg.  Yes I want the polls regardless of pollsters, but by state and after 10/28.

[Grad Students are the Worst]

OK.  While I'm working on this, you can explain to everyone how the Muhlenberg tracking poll (612 phone interviews conducted over three days) is worth excluding while Insider Advantage (588 phone interviews conducted over three days) should be included.

go!

[Grad Students are the Worst]

Too slow!  (But, seriously, keep working on that explanation.)

Anyway, done.  Excluding Muhlenberg and the UNH tracker, which I think is ridiculous and arbitrary but whatever, we have 135 state polls during the specified time period.

On average, they underestimated Obama -- yes, Obama -- by a 1.02% margin.  This was not statistically significant.

Of the 50 states, 37 had polling during this period.  Obama was under-estimated in 15, and over-estimated in 21.  Montana was an exact statistical hit.  More of McCain's 21 were small margin errors (i.e., <2.5%) or based on a single poll -- hence why all other measures showed Obama being the one under-estimated.

None of the aforementioned 36 were statistically significant, save for the following:

* Obama under-estimations in Massachuestts, Michigan, Nevada, New Mexico and Pennsylvania and Vermont.

* Obama over-estimations in Alaska (serious pollster meltdown there), Arizona, Arkansas, Iowa and Wyoming.

So, we have a statistically insignificant overall under-estimation of Obama.  There were five six states where Obama was under-estimated to statistical significance, and four where he was over-estimated.

Now, speaking of individual polls, 26 of 135 (19%) were off to statistical significance, which is actually a little bit better than a perfectly-conducted batch should be (the joys of weighting)

In any case, conclusions:

* On average, state polls slightly under-estimated Obama, but not to statistical significance.

* As one would expect with his overall being slightly under-estimated in the state polls, Obama's under-estimation achieved statistical significance in more states than did McCain's (6 vs. 4).

Iowa Arizona and one Obama-underpoll state (closed the window Kansas) narrowly missed SS.

* While Obama was over-estimated in more states than McCain, they tended to be states with few polls, and thus more potential for noise.  Of Obama's statistically significant under-estimations, all but MA had a good number of polls; only two of McCain's did (AK, AZ IA).

In closing, there is no strong evidence that polls under-estimated Obama.  However, there is no evidence whatsoever of a Bradley Effect, or any kind of Obama over-representation.  If it happened, it was eaten up by noise, an opposing pro-Obama effect, or perhaps was all a dream.

Raw data, for your own experimental pleasure, is available here.

[Grad Students are the Worst]

And just for fun, to further drive the point in, I looked at 538's time/sample size-adjusted model.  This is a bit more friendly to your thesis.  Obama was only under-estimated by 0.13% there, which is a lot further from statistical significance than 1.02%.  The biggest botch here wasn't Alaska, though; it was D.C., where the polls were off a whopping 20 points.  (Hawai'i was second-worse, another Obama under-estimation, then Alaska barely before another Obama under-estimation in Vermont.)

Convinced that you might be wrong about the Bradley Effect, yet?  Even a little?  My guess is "no."

(How's the tracker poll explanation going, btw?)

This is fun

[Lunar]

I think it's time to retroactively throw some polls out.

[Grad Students are the Worst]

I think it's time to retroactively throw some polls out.

Amusingly, throwing tracking polls out (very barely) increases Obama's under-estimation, since Muhlenberg was a little bit less pro-McCain in PA than UNH was pro-Obama in NH.

[J. J.]

Alcon, my thought was a 1-2 point overestimation of Obama, nationally and that the states could give mixed results. 

Now, in what  states where Obama overpolled did McCain underpoll?

In what states did McCain overpolled did Obama underpoll?

[Grad Students are the Worst]

Still waiting to find out why a three-day tracker is different than a three-day poll, btw.

[Lunar]

Still waiting to find out why a three-day tracker is different than a three-day poll, btw.

...wait, you mean that polling accuracy fluctuates up and down, possibly due to poor weighting, and it's stupid to apply analysis to tiny amounts of error without statistical significance, especially if it doesn't stand out from the overall sample?