Random Noise and why the polls might be wrong.

[The Vorlon]

The following is a graph that simulates a 3 day tracking poll of 500 "calls" per day.

The "true" result is an exact 50/50 spit between the two candidates, with a 7 day moving average tossed in to simulate a 7 day tracking poll.

All of these fairly wild variations are pure random noise.



The reason I post this is simply to demonstrate how massively weighted and massaged all polls are these days. - True "probability sampling" in the classic sense is utterly dead, and every pollster builds a huge number of weights and assumptions into their sampling.

These assumptions can and often are wrong.

There are, more or less, three basic ways pollsters weight and sample these days, and looking over the vast array of polls out there, when you break the pollsters into these three groups, these three groups are producing subtly different results.

Group 1 - The "quota callers"

The US census breaks the nation up into 48 census "cells" based upon race, income, age, geography, etc...  Quota callers (ABC news is likely the best executed of this category) essentially keep calling till they have enough completed interviews within each cell.. they then add the results up - essentially they are pre-weighting the sample via their calling strategy....


Group 2 - Post call weighting

In this strategy, the pollster calls who they call with their best attempt at a probability sample, and then the weight to census targets AFTER the data is collected. - Galup is the best known example of this.

Group 3 - Hard Party ID weighting

TIPP and Zogby are the biggest examples of this.

If you group the pollsters by these three types of sampling/weighting strategies, you actually get three different predicted outcomes for election 2012.

[King]

Yeah, tracking polls are pretty bad.  I'm also pretty sure they're wrong.

[Antonio the Sixth]

I have a hard time thinking the chart you posted is completely random. With so many iterations it's pretty weird that the red guy seems to be ahead 80% of the time.

[Antonio the Sixth]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

[Lief 🗽]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Also, racist.

[The Vorlon]

I have a hard time thinking the chart you posted is completely random. With so many iterations it's pretty weird that the red guy seems to be ahead 80% of the time.

And if I do it again, maybe the blue guys wins...

It's all +/- 2%

[The Vorlon]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Gee, thanks for elevating the debate, I appreciate it

[Mister Twister]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Gee, thanks for elevating the debate, I appreciate it

One of the basic rules of the internet is to judge a person based on their sig. Since your sig is stupid and wrong, that means you as a person are stupid and wrong.

[Antonio the Sixth]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Gee, thanks for elevating the debate, I appreciate it

One of the basic rules of the internet is to judge a person based on their sig. Since your sig is stupid and wrong, that means you as a person are stupid and wrong.

That's... not what I said.

And Vorlon, I'm sorry, but there's really little more to be said about your sig. I'm sure you conservatives can easily find a less blatantly fraudulent argument against Obama's policies.

[opebo]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Gee, thanks for elevating the debate, I appreciate it

One of the basic rules of the internet is to judge a person based on their sig. Since your sig is stupid and wrong, that means you as a person are stupid and wrong.

You've certainly hit the nail on the head in this case, Mister Twister.

[Torie]

I have been saying the same thing as Vorlon. One can massage the raw data with a partisan metric, or with demographic metrics, but massage all the pollsters must. And that gets outside the realm of true probability sampling theory.  I suspect the pollsters are unusually nervous this time. Many of them may end up looking bad. Just what will the electorate end up looking like? Will be it more like 2004, 2008 or 2010, or something in-between?  Nobody really knows for sure.

[Vern]

I too have said the polls are wrong and we really don't know what is going to happen. Of course I'm not as smart as other when it comes to all this stuff..

[Gustaf]

I have a hard time thinking the chart you posted is completely random. With so many iterations it's pretty weird that the red guy seems to be ahead 80% of the time.

I take it you haven't done much randomizing then.

[Brittain33]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Yeah, it's out of date and the major issue with the chart is that Obama's predictions were made on the basis of data that understated how severe the crash was at the end of 2008 and early 2009. GDP collapsed something like 8% when the number they had when they made the prediction was for a shallower dip.

[M]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Er... has the sig changed since this was posted?

If not, do you believe that the actual unemployment rate presented by the chart is "utterly stupid and fallacious," or that the administration's predictions as to the unemployment rate with and without the stimulus were "utterly stupid and fallacious?" Or do you believe that it is "utterly stupid and fallacious" to present this material in the same graph?

[Brittain33]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Er... has the sig changed since this was posted?

If not, do you believe that the actual unemployment rate presented by the chart is "utterly stupid and fallacious," or that the administration's predictions as to the unemployment rate with and without the stimulus were "utterly stupid and fallacious?" Or do you believe that it is "utterly stupid and fallacious" to present this material in the same graph?

The caption is misleading. It says the stimulus produced an economy worse than what was predicted. The implication is that the stimulus hurt the economy. The major reason for the discrepancy, which is a matter of data and not supposition, is that the predictions for unemployment drops in early 2009 were built on a foundation of statistics very much understating the collapse in the economy that was already underway. Unemployment with the stimulus was higher than Obama predicted, but unemployment without the stimulus would have unquestionably been much worse because the economy was in such free-fall in early 2009.

[angus]

Group 1 - The "quota callers"

The US census breaks the nation up into 48 census "cells" based upon race, income, age, geography, etc...  Quota callers (ABC news is likely the best executed of this category) essentially keep calling till they have enough completed interviews within each cell.. they then add the results up - essentially they are pre-weighting the sample via their calling strategy....


Group 2 - Post call weighting

In this strategy, the pollster calls who they call with their best attempt at a probability sample, and then the weight to census targets AFTER the data is collected. - Galup is the best known example of this.

Group 3 - Hard Party ID weighting

TIPP and Zogby are the biggest examples of this.

If you group the pollsters by these three types of sampling/weighting strategies, you actually get three different predicted outcomes for election 2012.

Interesting.  Which method do you think is best?

[Torie]

Group 1 - The "quota callers"

The US census breaks the nation up into 48 census "cells" based upon race, income, age, geography, etc...  Quota callers (ABC news is likely the best executed of this category) essentially keep calling till they have enough completed interviews within each cell.. they then add the results up - essentially they are pre-weighting the sample via their calling strategy....


Group 2 - Post call weighting

In this strategy, the pollster calls who they call with their best attempt at a probability sample, and then the weight to census targets AFTER the data is collected. - Galup is the best known example of this.

Group 3 - Hard Party ID weighting

TIPP and Zogby are the biggest examples of this.

If you group the pollsters by these three types of sampling/weighting strategies, you actually get three different predicted outcomes for election 2012.

Interesting.  Which method do you think is best?

Yes interesting, but there is another issue. You have to decide who you think will really vote, because almost all the responders say they will vote. And if the ensuing demographics arising from your screen seem way off from prior voting patterns, you begin to worry a bit, and maybe you change your filter screen. There is a lot alchemy going on out there. My intuition having said that, is that the keep calling model is best, but then, that is the most expensive way to do it, because you have to make more calls. I suspect the major reason the super secret internal polls of campaigns are best, is because they have the most sophisticated filters.

[SirMuxALot]

I think a case could be made the likely voter screens end up having a slant towards the winning party in the previous cycle.

Most pollster's LV screens will have some form of question about "did you vote 4 years ago".  Some pollsters use that question alone as their LV screen.

But think about two subsets of voters from 2008:

1) First time voters who turned out for Obama, because of the general excitement and historic nature of his candidacy. Many of these would otherwise be relatively unreliable voters (like the youth vote).  I think there's a pretty good case to be made this group will not turn out in 2008 numbers.  But the nature of the LV screen will pass them right through.

2) Generally reliable Republican voters who failed to turn out for a perceived moderate and unwhelming McCain candidate (not mention McCain campaign's pathetic GOTV).  There's a good case to be made this group will be well more motivated than 2008.  But the LV screen will bucket them out of the LV group since the answer to "did you vote in 2008" will be "no".

The interesting thing is, only Gallup and Rasmussen do anything significantly different than the rest of the industry regarding a likely voter model.  So the rest of the polling outfits would generally all be susceptible to this kind of methodology problem.

And this would explain the D+n numbers that are showing *better* than 2008 turnout for Ds, while worse than 2008 turnout for Rs.  The LV bar is a bit too low for Ds and a bit too high for Rs.

Won't know if this theory is right until election night.  But if Romney does win, I think the above is the clear explanation for so many polling failures.

[gsmiro]

I think the best model should be the "keep calling" model.  Each time, poll the same number of people, say about 1000 eligible adults 18 years and older, and repeat the process until the sample size is large enough for us to compile a distribution chart.  This way will be very expensive, but it will be quite accurate, because it will definitely create a very plausible confidence interval for the population.

2nd issue:  stimulus doesn't help the economy!  Why would debasing your currency, printing more money, giving them to banks to hold in their balance sheet help to create jobs?

Private sectors create jobs.  Private capital create jobs.  Those stimulus are not true capital in the sense.  It will only serve to create another bubble, just like the housing bubble, and causes more harm in the long run. 

[CountryRoads]

On an unrelated note, your sig is utterly stupid and fallacious in a way that even a 8-year-old kid could perceive.

Gee, thanks for elevating the debate, I appreciate it

One of the basic rules of the internet is to judge a person based on their sig. Since your sig is stupid and wrong, that means you as a person are stupid and wrong.

No, just no. Go away you hack.