Fox/OD sub-polls

[millwx]

...oops... title should say "sub-polls"!  Sorry.  

Apparently, Fox/OD oversampled some states.  Getting this from the PollingReport subscription service...

Florida:  Bush 48%, Kerry 38%, Nader 3%  - 750RV
Michigan:  Bush 42%, Kerry 40%, Nader 5% - 750RV
Ohio:  Bush 45%, Kerry 41%, Nader 4% - 750RV
Pennsylvania:  Bush 46%, Kerry 41%, Nader 3% - 750RV

I try not to be partisan when I look at these polls, especially with a respected firm like Fox/OD, but this stuff looks as bad as Zogby, or worse.  Bush up in Michigan?  Maybe, but doubtful.  Bush up 5% in PA??  I'll put money on this one... A LOT.  10% Bush lead in FL?  Maybe.  SUSA agreed.  But this is absolute maximum, as other polls disagree... Fox/OD could be right, but debatable.  Ohio is the only one that looks close to right to me.  Bush is likely marginally ahead there, IMO and according to MOST other polls (though ARG disagrees).

[ATFFL]

I brought this up in the big thread on the poll.

They all do seem to be on the high possible edge for Bush on the MoE.  Would love to see the full internals on them.

Oh, they are not oversamples, the main poll is 900 RV and each state is 750 RV.  THey are 4 seperate polls in the states.

[The Vorlon]

...oops... title should say "sub-polls"!  Sorry.  

Apparently, Fox/OD oversampled some states.  Getting this from the PollingReport subscription service...

Florida:  Bush 48%, Kerry 38%, Nader 3%  - 750RV
Michigan:  Bush 42%, Kerry 40%, Nader 5% - 750RV
Ohio:  Bush 45%, Kerry 41%, Nader 4% - 750RV
Pennsylvania:  Bush 46%, Kerry 41%, Nader 3% - 750RV

I try not to be partisan when I look at these polls, especially with a respected firm like Fox/OD, but this stuff looks as bad as Zogby, or worse.  Bush up in Michigan?  Maybe, but doubtful.  Bush up 5% in PA??  I'll put money on this one... A LOT.  10% Bush lead in FL?  Maybe.  SUSA agreed.  But this is absolute maximum, as other polls disagree... Fox/OD could be right, but debatable.  Ohio is the only one that looks close to right to me.  Bush is likely marginally ahead there, IMO and according to MOST other polls (though ARG disagrees).

The 4 Fox State polls were seperate samples, so they can't ALL be statistical blips (or the odds are soooo low it's not worth calculating)

Something STRUCTURAL is at play here in how they are drawing up their samples

OR

there actually has been a real shift in the electorat in the last 2 or 3 days...?

This is NOT statistical - It is a polling flaw OR a real shift.

I have a "theory" I am working with that I am going to investigate...

Will report back later

[millwx]

I brought this up in the big thread on the poll.

They all do seem to be on the high possible edge for Bush on the MoE.  Would love to see the full internals on them.

Oh, they are not oversamples, the main poll is 900 RV and each state is 750 RV.  THey are 4 seperate polls in the states.

Yes, the 900 RV isn't enough to simply "oversample" for 750/state.  You're right.

The question then is, since these samples are separate, does this mean no impact on the 900 RV poll?... it is, after all, the most Bush-friendly poll we've seen recently (the 10% lead in Harris was a LV poll).

Frankly, I'm not as impressed with Fox/OD as other people are.  Yes, they nailed 2000 pretty well, but, just as people say about Zogby (who also did well in 2000)... one poll doesn't make the company.  O.k., o.k., yes, comparing Fox/OD to Zogby is unfair.    But, my point is, I've conducted a pretty long-term examination of Fox/ODs favorability-unfavorability ratings for Bush.  Fox/OD shows the highest favorability gap... and they are above the mean by a solid few percent.  I know I'm in the minority in this opinion here, but I cannot get around the apparent pro-Bush bias in Fox/OD.  This recent set of polls just reinforces my opinion... right or wrong.

And as The Vorlon notes, these are all different samples... Highly unlikely that they're all randomly wrong.  Either I missed some real good news for Bush in the past few days or the Fox/OD polls do function in some biased way or I'm crazy.  I know you'll all subscribe to the third theory but I'm going with the second.

[millwx]

Something STRUCTURAL is at play here in how they are drawing up their samples.

I have a "theory" I am working with that I am going to investigate...

Are you saying they're wrong, they're the only one right, or just different?  Or don't you want to give away your secret yet?    

[The Vorlon]

Something STRUCTURAL is at play here in how they are drawing up their samples.

I have a "theory" I am working with that I am going to investigate...

Are you saying they're wrong, they're the only one right, or just different?  Or don't you want to give away your secret yet?    

The five polls (1 national + 4 State) either ALL stand, or ALL fall.

The fact they are all SEPERATE samples means it is NOT a statistical blip (5 polls ALL being outside the MOE and ALL high has about 1 chance in 102 million, so we can be fairly sure it is not statistical)

Even if we say Ohio is ok, the odds are still 800,000 to 1

This is structural - it's a flaw in their sampling, or the electorate has actually shifted.

[© tweed]

You can change the title

[ATFFL]

F: All of the above is also possible.  Even if there is a flaw in teh polling, there may be a real trend toword Bush.  

Finally, I think we all agree Millwx is crazy.

EDIT: How did this reply to the Vorlon end up over his post?

[The Vorlon]

Either I missed some real good news for Bush in the past few days or the Fox/OD polls do function in some biased way or I'm crazy.  I know you'll all subscribe to the third theory but I'm going with the second.

Actually, in a multible choice question there are extra options...



How about:

A)     Shift for Bush
B)     Flaw in Polling
C)     Millwx is crazy
D)     Shift for Bush AND Millwx is crazy
E)     Flaw in Polling AND Millwx is crazy....

Lots of possibilities...

Unrealated to a statistical theory I have but...

Last couple days has been news of the terrorists beheading westerners in Iraq and Saudi Arabia.

At a gut deep down level there has be be a bit a a desire for revenge for these especially brutal crimes.  This is a human instinct that I think trancends party ties.

I think deep down many people, even a lot of Democrats, think Bush is more lkely to extract a heavier revenge of the terrorists than Kerry...

Maybe this revenge/bloodlust factor is reflected in these polls?

[Smash255]

Something STRUCTURAL is at play here in how they are drawing up their samples.

I have a "theory" I am working with that I am going to investigate...

Are you saying they're wrong, they're the only one right, or just different?  Or don't you want to give away your secret yet?    

The five polls (1 national + 4 State) either ALL stand, or ALL fall.

The fact they are all SEPERATE samples means it is NOT a statistical blip (5 polls ALL being outside the MOE and ALL high has about 1 chance in 102 million, so we can be fairly sure it is not statistical)

This is structural - it's a flaw in their sampling, or the electorate has actually shifted.

I think its a problem or flaw in their sampling.  Just look at all the other polls that show opposites of Fox/OD during the saame period.  Fox 6/22-6/23 Bush up 6without Nader 7 with Nader, Gallup 6/21-6/23 Nationwide Bush 1/1.

PA state poll Fox 6/22- 6/23  Bush up 3without Nader 5with Nader,  Quinnipiac Pol 6/21-6/22 Kerry 6without Nader 1 with Nader

OH poll Fox 6/22- 6/23  Bush up 4  ARG 6/21-6/22 Kerry up 6

FL  Fox Bush 10, ARG Kerry 1

MI  Fox   Bush 2???

This Fox poll just seem insane.  The Most insane part is Blue States Bush 43 Kerry 42???

[ATFFL]

Something STRUCTURAL is at play here in how they are drawing up their samples.

I have a "theory" I am working with that I am going to investigate...

Are you saying they're wrong, they're the only one right, or just different?  Or don't you want to give away your secret yet?    

The five polls (1 national + 4 State) either ALL stand, or ALL fall.

The fact they are all SEPERATE samples means it is NOT a statistical blip (5 polls ALL being outside the MOE and ALL high has about 1 chance in 102 million, so we can be fairly sure it is not statistical)

This is structural - it's a flaw in their sampling, or the electorate has actually shifted.

I think its a problem or flaw in their sampling.  Just look at all the other polls that show opposites of Fox/OD during the saame period.  Fox 6/22-6/23 Bush up 6without Nader 7 with Nader, Gallup 6/21-6/23 Nationwide Bush 1/1.

PA state poll Fox 6/22- 6/23  Bush up 3without Nader 5with Nader,  Quinnipiac Pol 6/21-6/22 Kerry 6without Nader 1 with Nader

OH poll Fox 6/22- 6/23  Bush up 4  ARG 6/21-6/22 Kerry up 6

FL  Fox Bush 10, ARG Kerry 1

MI  Fox   Bush 2???

This Fox poll just seem insane.  The Most insane part is Blue States Bush 43 Kerry 42???

Most OH polls have had Bush up, ARG has been the exception.

Most FL polls have Bush up, ARG has been the exception.  Survey USA backs Fox.

Most PA polls have shown Bush very close, with the trend being the state tightening.  

MI is just plain freaky, though most polls have shown it tightening, a Bush lead is out there.

If you compae the FOX/OD poll with jsut one other poll, they look odd.  That does not tell you which is the outlier though.  

I think the Fox/OD polls may be overpolling Bush by 2-3, but not more than that

[Smash255]

Something STRUCTURAL is at play here in how they are drawing up their samples.

I have a "theory" I am working with that I am going to investigate...

Are you saying they're wrong, they're the only one right, or just different?  Or don't you want to give away your secret yet?    

The five polls (1 national + 4 State) either ALL stand, or ALL fall.

The fact they are all SEPERATE samples means it is NOT a statistical blip (5 polls ALL being outside the MOE and ALL high has about 1 chance in 102 million, so we can be fairly sure it is not statistical)

This is structural - it's a flaw in their sampling, or the electorate has actually shifted.

I think its a problem or flaw in their sampling.  Just look at all the other polls that show opposites of Fox/OD during the saame period.  Fox 6/22-6/23 Bush up 6without Nader 7 with Nader, Gallup 6/21-6/23 Nationwide Bush 1/1.

PA state poll Fox 6/22- 6/23  Bush up 3without Nader 5with Nader,  Quinnipiac Pol 6/21-6/22 Kerry 6without Nader 1 with Nader

OH poll Fox 6/22- 6/23  Bush up 4  ARG 6/21-6/22 Kerry up 6

FL  Fox Bush 10, ARG Kerry 1

MI  Fox   Bush 2???

This Fox poll just seem insane.  The Most insane part is Blue States Bush 43 Kerry 42???

Most OH polls have had Bush up, ARG has been the exception.

Most FL polls have Bush up, ARG has been the exception.  Survey USA backs Fox.

Most PA polls have shown Bush very close, with the trend being the state tightening.  

MI is just plain freaky, though most polls have shown it tightening, a Bush lead is out there.

If you compae the FOX/OD poll with jsut one other poll, they look odd.  That does not tell you which is the outlier though.  

I think the Fox/OD polls may be overpolling Bush by 2-3, but not more than that

Actually Ohio is pretty much split evenly on the polls, not just ARG has shown Kerry ahead

The Survey USA FL  poll was shown to be Republican Heavy in its sample.  Every other poll has shown it close either way.

PA  7 of the 8 previous PA polls showed Kerry ahead, and the only one that showed Bush ahead (Rasmussen) was by 1

MI 9 of the previous 10 polls showed Kerry ahead including the last 5

[Beet]

In May ARG and Rasmussen had Bush +1/tied. This month Bush is up +7/+10. It looks like Florida is trending to Bush by 6-10 points over the past month for some reason, which is unclear since there has been no similiar national trend outside of this latest Fox poll.

[Smash255]

In May ARG and Rasmussen had Bush +1/tied. This month Bush is up +7/+10. It looks like Florida is trending to Bush by 6-10 points over the past month for some reason, which is unclear since there has been no similiar national trend outside of this latest Fox poll.

ARG Florida Poll taken from 6/21-6/23 shows Kerry up 47-46

[Beet]

In May ARG and Rasmussen had Bush +1/tied. This month Bush is up +7/+10. It looks like Florida is trending to Bush by 6-10 points over the past month for some reason, which is unclear since there has been no similiar national trend outside of this latest Fox poll.

ARG Florida Poll taken from 6/21-6/23 shows Kerry up 47-46

I must be looking at an outdated source then, though it seems to be updated every day.

[ATFFL]

Only Ohio polls I know of having Kerry up are ARG, LA Times and Zogby Internet.  The LA Times polls were garbage and the Zogby polls have terrible methodology.  I would not consider them woth much of anything.

Until the recent Quinnepac poll the trend in PA had been very tight.  It is likely it still is and both polls are running on the edge of the MoE.

Even if Kerry has been ahead in MI in the past it does not mean he still is.  He was almost always up only 2-4 points.  When you consider that it looks less like an outlier out of the MoE and more like a tossup that went to Bush, but still in the MoE.

I'm not willing to throw them out completely until I see some internals.  They do seem to be slightly Bush favorable, but not to the extent that the Times was for Kerry.

[Smash255]

Only Ohio polls I know of having Kerry up are ARG, LA Times and Zogby Internet.  The LA Times polls were garbage and the Zogby polls have terrible methodology.  I would not consider them woth much of anything.

Until the recent Quinnepac poll the trend in PA had been very tight.  It is likely it still is and both polls are running on the edge of the MoE.

Even if Kerry has been ahead in MI in the past it does not mean he still is.  He was almost always up only 2-4 points.  When you consider that it looks less like an outlier out of the MoE and more like a tossup that went to Bush, but still in the MoE.

I'm not willing to throw them out completely until I see some internals.  They do seem to be slightly Bush favorable, but not to the extent that the Times was for Kerry.

Granted they are older polls, but an older Rasmussen & Ohio poll showed Kerry ahed in Ohio, the most recent polls have gone back & forth

As far as PA goes their has been NOTHING to suggest its anywhere near Bush +5

[ATFFL]

Only Ohio polls I know of having Kerry up are ARG, LA Times and Zogby Internet.  The LA Times polls were garbage and the Zogby polls have terrible methodology.  I would not consider them woth much of anything.

Until the recent Quinnepac poll the trend in PA had been very tight.  It is likely it still is and both polls are running on the edge of the MoE.

Even if Kerry has been ahead in MI in the past it does not mean he still is.  He was almost always up only 2-4 points.  When you consider that it looks less like an outlier out of the MoE and more like a tossup that went to Bush, but still in the MoE.

I'm not willing to throw them out completely until I see some internals.  They do seem to be slightly Bush favorable, but not to the extent that the Times was for Kerry.

Granted they are older polls, but an older Rasmussen & Ohio poll showed Kerry ahed in Ohio, the most recent polls have gone back & forth

As far as PA goes their has been NOTHING to suggest its anywhere near Bush +5

Agreed, PA is definitely the poll that is the most pushing it.  Still could easily be well with in the MoE though.

[millwx]

Finally, I think we all agree Millwx is crazy.

Well, even if Fox/OD isn't right, at least YOU are!!

And just so y'all know I'm not totally off my rocker, here's the data I'm talking about w.r.t. my concern over bias in Fox/OD.  Here are the job approval (approve minus disapprove) poll averages for the past 18 months averaged together...

AP/IPSOS = +13.2%
Quinippiac = +14.6%
Time/CNN = +14.8%
Newsweek = +17.0%
Gallup = +19.0%
Pew = +19.1%
CBS = +20.3%
ABC/WP = +21.5%
Fox/OD = +22.8%

Fox/OD is also the ONLY poll showing NO monthly averages being negative.  Moreover, Fox/OD has one of the highest percent undecided.  If you understand how that works, you'll know that this means the 22.8% approval gap is relatively larger than it looks compared to the other polls.  For example, among only the decideds Fox/OD is +25.3%, while the next in line, ABC/WP is only +22.3%.  That's a rather whopping 3% difference between the top two numbers, considering the entire table above, with nine polls, spans less than 10%!  And that 10% is aided by the timing of the March '03 polls for AP/IPSOS and Quinippiac; that skews their numbers... they would otherwise be between 16% and 17%, shrinking the full range slightly.

Since, in a poll like this, it's impossible to say who's "right", you can at least see my concern.  All the praising up and down of Fox/OD's methodology is unconvincing to me.  It's numbers simply look biased.  I'm NOT saying it's horribly biased!  The above numbers, in fact, show Fox/OD only a bit higher than the next highest poll numbers.  So, don't get me wrong, I'm not blasting Fox/OD as Zogby-like randomness.  BUT, in such close numbers in many states and nationally, a few percent can make a lot of difference.  And based on the above numbers, Fox/OD could be "off" (aka, biased) by 2-5%, roughly.

[Smash255]

Finally, I think we all agree Millwx is crazy.

Well, even if Fox/OD isn't right, at least YOU are!!

And just so y'all know I'm not totally off my rocker, here's the data I'm talking about w.r.t. my concern over bias in Fox/OD.  Here are the job approval (approve minus disapprove) poll averages for the past 18 months averaged together...

AP/IPSOS = +13.2%
Quinippiac = +14.6%
Time/CNN = +14.8%
Newsweek = +17.0%
Gallup = +19.0%
Pew = +19.1%
CBS = +20.3%
ABC/WP = +21.5%
Fox/OD = +22.8%

Fox/OD is also the ONLY poll showing NO monthly averages being negative.  Moreover, Fox/OD has one of the highest percent undecided.  If you understand how that works, you'll know that this means the 22.8% approval gap is relatively larger than it looks compared to the other polls.  For example, among only the decideds Fox/OD is +25.3%, while the next in line, ABC/WP is only +22.3%.  That's a rather whopping 3% difference between the top two numbers, considering the entire table above, with nine polls, spans less than 10%!  And that 10% is aided by the timing of the March '03 polls for AP/IPSOS and Quinippiac; that skews their numbers... they would otherwise be between 16% and 17%, shrinking the full range slightly.

Since, in a poll like this, it's impossible to say who's "right", you can at least see my concern.  All the praising up and down of Fox/OD's methodology is unconvincing to me.  It's numbers simply look biased.  I'm NOT saying it's horribly biased!  The above numbers, in fact, show Fox/OD only a bit higher than the next highest poll numbers.  So, don't get me wrong, I'm not blasting Fox/OD as Zogby-like randomness.  BUT, in such close numbers in many states and nationally, a few percent can make a lot of difference.  And based on the above numbers, Fox/OD could be "off" (aka, biased) by 2-5%, roughly.

Another interesting Fox poll tidbit.  Looking at ALL of the Fox Polls with Nader involved they have NEVER shown Kerry ahead,most the other polls have shown Kerry leading at some poimt.  Out of all the pollsters that have had a bunch of polls Fox/OD is the ONLY one not to show Kerry up with Nader at ANY point, AP has also shown Bush ahead the whole time,but they have  fewer total polls than the other big ones.  So out of the pollsters with more than a couple polls only Fox & AP have shown Bush leading all the way, and with all the pollsters that have come out with more than 5 polls ONLY Fox has not shown Kerry leading at any points in the Nader polls.

[ATFFL]

Another interesting Fox poll tidbit.  Looking at ALL of the Fox Polls with Nader involved they have NEVER shown Kerry ahead,most the other polls have shown Kerry leading at some poimt.  Out of all the pollsters that have had a bunch of polls Fox/OD is the ONLY one not to show Kerry up with Nader at ANY point, AP has also shown Bush ahead the whole time,but they have  fewer total polls than the other big ones.  So out of the pollsters with more than a couple polls only Fox & AP have shown Bush leading all the way, and with all the pollsters that have come out with more than 5 polls ONLY Fox has not shown Kerry leading at any points in the Nader polls.

TIPP/IBD has not had Kerry up since March.

The CNN/Gallup/USA Today poll have also not had Kerry up since early March.  Yes, this is different from other Gallup polls due to CNN and USA Today altering the question wording/methodology.

AP/Ipsos has had 3 polls, all with Bush up.

Zogby has not had Bush up since early April.

Fox has had several ties.

[millwx]

And just so y'all know I'm not totally off my rocker

I should have also added to my diatribe that I've not selectively cherry-picked here.  This is every polling organization that has polled at least 80-90% of the months for the past 18 months.  And I've limited it only to those asking the job approval question in a consistent manner (e.g., Zogby is left out because they ask it in terms of "excellent", "good", "fair", and "poor").

Furthermore, I did not selectively cherry-pick 18 months because it fit my point.  It runs Dec '02 through May '04.  And, in case you're curious, for the June '04 numbers in thus far, Fox/OD has the second highest favorability gap.  And running before Dec '02... in Nov '02 Fox/OD was WAY above all of the others (by about 10% in most cases); in Oct '02 is was also the highest... etc, etc.

So, I truly am trying to be "fair and balanced" here, no pun intended :-)  I did not pick and choose the data, massage it, or anything else.

[classical liberal]

Actully, by what you said you massaged it in the favor of your opposition.  Leaving it clear would have strengthened your point.

[millwx]

Actully, by what you said you massaged it in the favor of your opposition.  Leaving it clear would have strengthened your point.

Not sure what you mean.  I did no massaging... straight-up monthly average of the favorability gap of all available polls.

Also, just FYI, added six previous months to my data to make it a two-year average... Fox/OD actually got worse.  If that's what you meant by massaging (i.e., just selecting 18 months), I merely stopped there because I didn't feel like going back any further, and I felt the point was made.  I just added the additional 6 months.  I could add more, but, frankly, don't feel like it at this point.  Anyway, stretching it to two years (had to drop Quinnipiac to do this) showed:

AP/IPSOS = 18.9%
Time/CNN = 20.4%
Newsweek = 22.3%
Pew = 23.8%
Gallup = 24.8%
CBS = 25.4%
ABC/WP = 26.6%
Fox/OD = 29.0%

The 2.4% difference between ABC/WP and Fox/OD is the biggest jump between any two polls on that list.  And while 2.4% may not sound bad, keep in mind that this is a two YEAR analysis... not exactly any small sample of data.

[classical liberal]

so FOX is wrong by roughly their margin of error, ie most statistical ties are probably in Kerry's favor slightly but may show Bush due to a slight bias.  I think that that's reasonable.