Non-Gallup/Rasmussen tracking polls thread

[J. J.]

No, I'm saying that whatever the problem, it doesn't seem to effect the result.  Please don't put words into my mouth.

I'll let you put words in your own mouth, then.  There are three options:

1) The poll's sample is not as "off" as it seems. Obama is performing worse than Kerry among youths -- probably much worse.

2) The one-in-many-million event occurred, and the poll was just off by chance.

3) The poll had a methodological flaw.

You need to pick one.  There are no other possible options here -- either the poll isn't off, it is off by random chance, or it is off by error.  Which option are you arguing?

4)  The poll is not designed to accurately measure the subsamples.  That isn't a methodological flaw because it's not designed to get that type of information.

[J. J.]

Thanks for the daily tracker...tracker, Sam.

Absolutely!  Thank you.

[J. J.]

IBD/TIPP
Obama 45.8%(+1.0)
McCain 42.3%(-1.4)

Probably a bad sample dropping.

[J. J.]

IBD/TIPP
Obama 45.8%(+1.0)
McCain 42.3%(-1.4)

Probably a bad sample dropping.

Yeah, a good Mac day fell off.

Probably not the methodology.

[J. J.]

IBD/TIPP
Obama 45.8%(+1.0)
McCain 42.3%(-1.4)

Probably a bad sample dropping.

Yeah, a good Mac day fell off.

Probably not the methodology.

Is that your unsubtle substitute for responding to my last post? 

Well, since you've declined to respond to a few ... .

In this case, looking at the rest of the polls, along with TIPP's record, it was fairly clear it wasn't methodology.

[J. J.]

Well, since you've declined to respond to a few ... .

What posts were those?  It was not intentional.  I will respond to any post you link me to.

I would appreciate the same in return.  Here is the post in question.

If you say it's "fairly clear that it [wasn't] methodology," you're saying it's fairly clear that it was a one-in-many-million event.  It must be one, the other, or an accurate observation.  I am still waiting for you to pick one of the three.  They are the only three options.

Your infamous claims of "evolution is proven."  I never received an answer. 

I am saying, however, that at a subsample really is not indicative of the rest of the poll.

[J. J.]

Your infamous claims of "evolution is proven."  I never received an answer. 

I don't know how "infamous" it is, but I found the topic again.  I never claimed that "evolution is proven" anywhere in that topic; if I did, I should not have, it's a theory.  Our debate was over whether the development of metacognition was an aberrant pattern that indicated intelligent design.  In the last post I made, I conceded that I didn't really have any way of proving my thesis, or you, yours, but that I thought you were looking for evidence in your own conclusions.

Looking over your last post, you just re-iterated points you had already made.  To me, it looks like that was a mutual agreement to close the subject with mutual disagreement.  You didn't ask any question in the last post that I left "hanging," other than ones which I answered but that you had different interpretations on.

The question was on the ability, basically, or another species to "ask wny."  You never posted the information.

You must be logged in to read this quote.

First off, that's factually incorrect.  As Verily pointed out, a swing to Obama 60-40 (closer to in line with national polling)

Second off, you have still failed to pick one of the following options.  Again, one of the following must be true.  Which one do you believe is most probable?

1. The sample is accurate; all other polls are significantly off.

2. The sample is off by random chance, but with no methodology flaw; an event of infinitesimal likelihood occurred.

3. The sample is off by design, and the methodology is flawed, thus the poll is questionable.

"The subsample really is not indicative of the rest of the poll" is invalid because:

1. It affects the topline significantly; and,

2. It shows that their methodology was flawed.

I await your answer -- again, option one, two, or three.
[/quote]
4.  The subsample does not necessarily affect the rest of the poll.  As I've pointed out, there were unusual results in that subsample before and the poll was in line with the rest of the polls at that time.  Does it, any more than those fluctuations we see on Gallup or Rasmussen, mean anything out the methodology?  Does it cause the poll result to be outside of the MOE?  I would say no.

[J. J.]

Only those three options are logically possible.  There is no option #4.

If Gallup had a poll with a subsample as wildly off as this poll, then yes, we would presume that either there had been a huge swing in the subsample or that Gallup had a flawed methodology.

Or that the subsample selection was bad, i.e. unrepresentative.  The question is, does a bad subsample knock the poll out of MOE?  I say, with TIPP so far, no.

[J. J.]

J. J., your option four is not mutually exclusive to the other three.  It demands one of the other three be true.

No matter whether the subsample reflects the poll or not, it must be one of the following:

I. Correct
II. Incorrect, by random chance
III. Incorrect, by poll design flaw

Do you think (I) is likely?  I doubt it.

So, the question is:  Do you think either (II) or (III) is more likely?

I think I've said that if it would be III (and I suspect II) it isn't enough to effect the results.  Why.  Because we had similar results in the subsample and the poll behaved the same as the other major polls.

[J. J.]

Well, since you've declined to respond to a few ... .

What posts were those?  It was not intentional.  I will respond to any post you link me to.

I would appreciate the same in return.  Here is the post in question.

If you say it's "fairly clear that it [wasn't] methodology," you're saying it's fairly clear that it was a one-in-many-million event.  It must be one, the other, or an accurate observation.  I am still waiting for you to pick one of the three.  They are the only three options.

"one=in=many-million"?!?

Go back and reread basic survey research methodology.

One out of twenty times the MoE is likely to be exceeded for a sample.

As to subsamples, the MoE is generally so large, its not worth examing.

Go back and read this thread.  The chance that the sub-sample would be off by ~70 points is one in many million, and that is accounting for the high MoE.

I didn't say that the chance it would be out of MoE is one in several million; obviously, that's 1-in-20.  That's the definition of Margin of Error (at the 95th confidence rate, per survey standards.)

Alcon, first I asked the question if any non Homo species ever asked why things are as they are.  You promised a link, but never answered.  I actually was disappointed.  

This is a subsample and I'm looking at if it is sufficiently bad to move the total result out of the MOE.  No.

[J. J.]

Or that the subsample selection was bad, i.e. unrepresentative.  The question is, does a bad subsample knock the poll out of MOE?  I say, with TIPP so far, no.

As Verily posted (which you've ignored three times), it affects the result by more than a couple of points, from Obama +1 to Obama +8.  I haven't checked his math thoroughly, but it appears sound.

The point is not the subsample's effect on the topline.  It is the cause of the bad subsample.  It must either be a random flaw, or sampling error.  In this case,we've already demonstrated that a random flaw has a ridiculously low probability.  Yet, you appear to be totally unwilling to admit that it appears to be a sampling error.  Do you really think a sampling error is less likely than a one-in-many-million event?

If it is likely a sampling error, it impeaches the methodology of the poll.  The methodology was not unique for that one subsample.  The entire poll comes into question.

Sorry, considering the results TIPP has had in prior elections, I don't see a methodological problem.  Ah, what part of "and I suspect II" do you have a problem with?  I'm questioning if a sample that would produce a bad result in a subsample would produce a bad result in the whole poll (one out of the MOE).

[J. J.]

Sorry, considering the results TIPP has had in prior elections, I don't see a methodological problem.  Ah, what part of "and I suspect II" do you have a problem with?  I'm questioning if a sample that would produce a bad result in a subsample would produce a bad result in the whole poll (one out of the MOE).

OK.  You're assuming that a statistically infinitesimal chance is more likely than a pollster having a flawed methodology.  Is that an accurate summary?

In any case, if you'd accepted that the subsample is flawed, are you willing to adjust the overall result to match other polls' subsample (i.e., to Obama +8)?  Why not, if not?

Right now, this poll has the gap at 3.5 points.  Rasmussen has it at 7.0 points.  Gallup at 6/5 points.  All of these numbers have a 3.5 point range, a reasonable range (I think that is less than the MOE for TIPP).  As I said, it's in line with the others.  I will be willing to concede that it tends to over count the undecided vote. 

I won't try to play around with the R2K weighting.  So I'm playing around with this.

[J. J.]

I'll re-post my post, and highlight the parts you didn't reply to in bold.

You suspect the chances of flawed methodology are less than 1 in many millions?  Why?

Also, Verily already demonstrated that it would cause a seven-point swing.  It also impeaches the credibility of the poll if it's (III).

Now, convince me that (II) is vastly more likely, and I'll trust it.  I still will not trust the top-line, though, for obvious reasons -- because the "off" sample affects it by many points.

The MoE on a poll of this size is +/-3%.  Verily's math shows that this would cause a seven-point swing.  In other words...outside of MoE*!   So, I should bold that part of my post, too.

(* - Why are you convinced that it's OK if a sub-sample is ridiculously outside of MoE, as long as the topline is OK?  Each subsample is its own 1-in-20, yes.  That does not mean that a 1-in-many-million is OK, especially if it affects the result by 7 points.)

Alcon, did it occur to you that it might overpoll for Obama in other subsamples, but that the average might still be correct?  For some reason, this poll has an excellent track record.  I try not looking too strongly at the subsamples but at the whole poll.

[J. J.]

J.J. is saying that it all averages out Alcon.  

So, yes, this mistake was just because of randomization.

He believes in the  0.000000002%

Something made that poll the most accurate one in 2004. 

[J. J.]

Something made that poll the most accurate one in 2004. 

And its performance in 2004, whether it be by solid methodology then, or luck, does not make the statistics here any less damning.

It maybe decreases the probability of "methodology error," but to 0.000000002% (from now on an inexact placeholder for whatever confidence rate is involved here)?  Moreover, to the 1 in 250 quadrillion necessary for another sample to be equally off (ditto caveat)?

No.

Sorry, but I go by track record.  Now, I'm not seeing this poll, in general being out of line with Rasmussen or Gallup.

[J. J.]

Zogby did pretty good in 2000.

...And I bet they didn't have such a high error rate on their samples.

What say ye, J. J.?

It's been good over the last two cycles (third in 2000), and it's in line with the other polls, even when you look at these unusual subsamples.

[J. J.]

So, how many times has lightning struck you this year, J. J.?

This year?  That's not fair, he has a 5.6% chance of not being struck by lightning this year under his system (~18 strikes a year).

And if something that is 0.000000002% is an acceptable random occurrence in J.J.'s world, 5.6% is twenty-eight million times more acceptable.

I don't expect to get struck by lightening underground, nor do I expect a subsample to be particularly good.

[J. J.]

Subsamples have to abide by the same laws of statistics as the overall sample.  Subsamples are not immune from criticism.

If this just a pole of the 18-24 year olds, I'd probably agree.  Newsflash, it isn't.

You must be logged in to read this quote.

No, I "defend" the poll because of it's previous accuracy.  I'm not defending Battleground which is more pro McCain; I actually think that one suffers from the R2Kos problem, but in reverse.  I was "defending" TIPP when it showed 7 point lead for Obama.

I'm not a fan of looking at subsamples that are exceedingly small, and trying to extrapolate from there.

I'm also looking at this poll, and trying to see how much different it is from the rest.  Answer is, not a lot.

[J. J.]

To add to Lunar's point, let me re-emphasize:

Subsamples are smaller.  But math accounts for that in calculating their margin of error.  So, you cannot argue that this is fine because of the sample size.  We are taking into account the sample size when we tell you how infinitesimal the chances of it being non-methodological are.

Or we're taking a subsample that is one of those out of the MOE, when it is independent.

If you are looking at a poll with 5 subgroups, and treating each as an independent poll, which is what you're doing, each subgroup has a 1 in 20 chance of being "bad."  There is now a 5 in 20 chance that one of those subgroups is bad.  That one subgroup doesn't necessarily move the entire poll out of the MOE.  You have to look at the whole poll, not just the subgroups.

Now, I keep coming back to the range.  It's about 3.5 points between three tracking polls, with a good record; I think that's less than TIPP's overall MOE.  TIPP might be at the bottom and Rasmussen at the top, so maybe the gap is 5.25.  I only recall one person saying it was below that.

And Lunar, this isn't about defending me.  It's about if this poll is adequate in terms of methodology.  Certainly their track record says it is.

[J. J.]

Or we're taking a subsample that is one of those out of the MOE, when it is independent.

If you are looking at a poll with 5 subgroups, and treating each as an independent poll, which is what you're doing, each subgroup has a 1 in 20 chance of being "bad."  There is now a 5 in 20 chance that one of those subgroups is bad.  That one subgroup doesn't necessarily move the entire poll out of the MOE.  You have to look at the whole poll, not just the subgroups.

You're right...but the chance is still 1 in 2.5 quadrillion or so that two sub-samples would be off, that much, in the exact polar ways.

The thing is, we're not dealing with 1/20.  We're dealing with 1/many million.

They wouldn't have to be.  An MOE underpolling by McCain across the others might account for it.  That's one of the reasons I'm not looking at subsamples.

[J. J.]

Zogby last three individual nights:
Thursday Obama 49%- McCain 42%
Friday Obama 50%- McCain 45%
Saturday Obama 49%- McCain 46%

A.  It could be a bad sample.

B.  It's Zogby!

[J. J.]

Table updated.

Thank you.

[J. J.]

Daily Tracker Table - October 26, 2008

Poll Name	Obama	McCain	Margin	Change	Observation
Zogby	49.4%	44.1%	O+5.3%	M+4.3%	Who knows - it's Zogby!  Seriously though, a huge Obama sample did fall off today.
Rasmussen	52.46%	44.00%	O+8.46%	O+0.57%	Another good Obama sample.
Battleground	49%	46%	O+3%	NC	No polling done today.
Hotline	50%	42%	O+8%	O+1%	A good Obama sample should fall off tomorrow.
R2000/DKos	51%	40%	O+11%	M+1%	Ditto.
Gallup
Expanded	52%	43%	O+9%	O+1%	I have some real commentary on Gallup that I may post later.  It's now been over a week that the so-called "Expanded" model picks up less likely voters than the "Traditional" model.
Traditional	50%	45%	O+5%	M+2%
IBD/TIPP	46.5%	43.3%	O+3.2%	M+0.7%	Hard to tell.
NOT RELEASED
ABC/WP	52%	45%	O+7%	M+2%	I looked at the samples here yesterday, and it was pretty clear a strong Obama sample moved off today.
POLLS AVERAGE	50.17%	43.55%	O+6.62%	M+0.82%	Note: I do include Battleground in this average, even though it didn't poll yesterday or the day before.  If you wish not to, just remove and re-round.

In all honesty, but for Rasmussen, I'd be saying McCain is closing.

[J. J.]

Average of Rasmussen, Hotline, Gallup traditional, and IBD is McCain %+0.2825 today.  How are you viewing this, honestly?

Honestly, I look at LV on Gallup and I pay very close attention to the 'bots.  If they were to drop and Gallup, IBD and possible ABC would drop or hold, I'd say McCain was closing.

I do think Obama's lead in PA may be eroding, though not collapsing. 

[J. J.]

Average of Rasmussen, Hotline, Gallup traditional, and IBD is McCain %+0.2825 today.  How are you viewing this, honestly?

Honestly, I look at LV on Gallup and I pay very close attention to the 'bots.  If they were to drop and Gallup, IBD and possible ABC would drop or hold, I'd say McCain was closing.

I do think Obama's lead in PA may be eroding, though not collapsing. 

Lunar is excluding Gallup-Expanded, and only including those you indicated, including a M+2 on Gallup-Traditional that may simply be rounding.  Seems a bit leap to "I'd say McCain was closing" from McCain +0.28% in one day -- especially since we have no reason to exclude Rasmussen.  Heck, if you excluded Gallup-Traditional, the average would be moving toward Obama.

If it weren't for Gallup-Traditional, would you say that Obama is expanding his lead?

Alcon, if the polling was showing across the board increases for Obama, I'd say it.  

 I really give a great of weight to to Rasmussen, Gallup (traditional after reading Sam's comments), and TIPP.  After those ABC/WP and Hotline.  The lessers are GB, R2K, and Zogby.

Of the top three, two show gains for McCain; the third one unchanged.  The second tier is split Obama and no poll.  The two third tier two show a McCain gain, and one no poll.

Is the numbers were reversed, I could not Obama was gaining.  I cannot say McCain is gaining today.