Polls

[CARLHAYDEN]

Great Job Dave,

Suggest everyone take a look at the results of the National Council on Public Polls Review Board with respect to the 2002 polls.

Zogby not only had the highest margin of error of the five polling organizations but also had far and away the highest error rate in predicting the winner!

[CARLHAYDEN]

Several things I have seen in a number of polls (different organizations):

1. Bush support is firmer than Kerry's,

2. Bush support is primarily support FOR Bush while half or more of the Kerry support is merely ANTI-BUSH,

3. when the undecideds and leaners are asked to respond to a variety of issue questions they consistently score significantly closer to the core Bush supporters than they do to the core Kerry supporters (or more accurately anti=Bush people).

[CARLHAYDEN]

Several things I have seen in a number of polls (different organizations):

1. Bush support is firmer than Kerry's,

2. Bush support is primarily support FOR Bush while half or more of the Kerry support is merely ANTI-BUSH,

3. when the undecideds and leaners are asked to respond to a variety of issue questions they consistently score significantly closer to the core Bush supporters than they do to the core Kerry supporters (or more accurately anti=Bush people).

Correct on #1
Correct on #2

Please provide examples and links on #3

Please & Thank-you

Unfortunately, the polls which have explicity differeniated between the beliefs of Kerry voters, Bush voters and undecided voters on the issues are not available for publication at this time.

The sources which have commissioned the surveys are more interested in used the data than in publishing it at this time.

The Kerry campaign is well aware of this which explains the waffling by Kerry.  His core supporters (actually mostly anti-Bush voters) agree with the stances he took on the issues during his Senate career.  He is backing off on those issues (attempting to obfuscate the issues) because his campaign is well aware of the fact that the undecided voters differ in their preferences from those earlier Kerry positions and prefer the position held by Bush.

I hope on of the reputable public pollsters (i.e. news media funded) will provide the data (they can easily correlate this).

[CARLHAYDEN]

Too many people are too trusting of survey results.

First, there is a sample size error, which on national polls for total results, typically runs from more than two to four per cent.

That is to say that a 50-50 tie for the total sample could easily be 52-48, or 48-52 (just considering size alone).

Second, virtually all surveys are telephone polls.  Since federal law prohibits calling a person who might be charged for receiving the call (which is the case with some cell phones), this skews the results.

Third, with systems such as Call Intercept, many landline potential respondents are removed from consideration.

Fourth, a large number of persons answering the phone refuse to participate in the surveys, or do not complete the survey.

Fifth, there is an open question as to how many respondents lie in their responses.

Sixth, the screening of the sample is very important, and problematical.  The better surveys use past voting behavior as more creditable that self-proclaimed intent to vote.

Seventh, the phrasing of the question can change the result.  All to often what is reported is the analysis of the results, rather than the exact wording of the question.

Eighth, the horserace question answers can be affected by other questions preceding the horserace questions.

Ninth, the responses can also vary as to what candidates are included.  Is Nader included?

Tenth, the date(s) the survey questions were posed can affect the result.  Some of the surveys showing Bush doing badly were taken during the period when the liberal media was playing a 24 hours wall to wall coverage of the prison scandal in Iraq.

These are just a few of the matters which affect survey results.  

In short, view all 'poll' results with a jaundiced eye.

[CARLHAYDEN]

But he says its a Mason-Dixon poll.  You have a very high opinion of Mason-Dixon, I thought.

Mason-Dixon - absolutely a VERY good firm.  At the state by state level easily[/b] the best firm out there.  Frankly, compared to a lot of the other firms, it's not even close.

MD is the only firm in the public domain I actually trust from here to September that does state polls. (Maybe Ipsos too actually)

I was talking "generically" in this post.

Mason Dixon's record speaks for it's self.  In 2002 they publically polled 23 races and got 22 right.  Their average candidate error was 1.8%.  (If you limit it to polls published in the last 7 days of the campaign they did even better) The other 70 or so private polls they did in 2002 had a comparable success rate.

The only firm with lower average candidate error was Gallup, and there is certainly no shame in coming in 2nd to Gallup..

First, welcome back.  Hope you had a good vacation.

Second, thank you for your observations on the previous post to me.

Third, I agree on MD being a quality firm, I have my doubts about ISPOS (they seem to do ok in Canada but, IMHO they tend to overstate the liberal and Democrat support in this country).

Fourth, while the numbers aren't "hard," and aren't kept on official record, it appears that in the battleground states (which I define as being a state which either Bush or Gore carried with a margin of less than ten per cent) the "refuse" to participate in the surveys is highest among males whose age group is estimated to be 45-64 (without noticable accents).  

While you can (eventually) get enough 45-64 year old males to meet quota, I maintain that those responding do not necessarily reflect the views of those who refuse to participate.  

I believe that the exit polls should be funded by a consortium of existed reputable pollsters, who could use the data base to adjust their future polls.  This would probably be able to either confirm or deny my suspicion on this matter.

Fifth, also, please note that even when reading the questions verbatim to the respondents, the accent of the questioner appears to have some impact on the responses.  This is particularly important as firms do not pay particularly well for the questioners and tend to hire the less affluent (and more prone to regional accent) segment of the population.  

[CARLHAYDEN]

I think you got cut off before you could complete your post.

I'd be interested in what else you have to say.

In addition to exit polls to serve as a partial corrective to existing problems, some firms on the qt also compare their data to University of Michigan SRC data (when it eventually, long after the fact becomes available).

[CARLHAYDEN]

While waiting for Vorlon to reply, let me suggest two reasons.

First, sample size is not usually the biggest error built into polls.

For example, a poll with 900 respondents would have a sample size error of approximately 3.33% whereas a poll with 1600 respondents would have a sample size error rate of approximately 1.66%.  Even if you got up to a poll with 6400 respondets you would still have a sample size error of approximately 1.25%.  Increasing the sample size beyond a certain point yields verly little in reducing error.

Second, even thought there are a lot of people doing polls, the cost of doing a decent poll (one without a lot of inbuilt problems) is very high!

There is one advantage to doing a supersize poll , namely that (if correctly done) it would provide valid information on small subsets.

For example, the subset on a typical poll for Jewish voters is so small that the sample size error rate exceeds ten per cent.  This means that a poll that shows Jewish voters favoring Kerry over Bush  by say 65 to 35 could easily actually be 50-50, or 80-20, or anywhere in between.

The same holds true for other groups (farmers come to mind).

[CARLHAYDEN]

Lets take a case of an 800 respondent poll versus a 500,000 respondent poll.

If the 800 repondent poll said half the respondents prefer apple pie to cherry pie, and half prefer cherry pie to apple pie, its likely that (based on sample size) somewhere between 47-53 per cent prefer either.

While the 500,000 respondent survey would reduce this to 49.8-50.2% swing, the cost is not worth the small improvement.

The built in problems with polls (irrespective of sample size) would dwarf the sample size improvement resulting in negligible improvement for a vastly increased cost.

[CARLHAYDEN]

It was Liberty Digest and it went out of business shortly afterwards.

Thank-you.

There is an interesting paper by Brown & Chappell (University of South Carolina) called Forecasting Presidential Elections Using History ahd Polls.

One of the interesting things they found looking at the Gallup Polls was that there was "a tendency for polls to overstate support for Democratic candidates" (page .

P.S. It was the Literary Digest.

 I knew it was xxx Digest, could not quite recall the name.

[CARLHAYDEN]

Glad to see someone else caught this.

[CARLHAYDEN]

Hey, we all make mistakes.

Actually Vorlon was partially mistaken about the source for the Literary Digest poll.

They sent return mail ballots to subscribers, and to people who had telephones (even if they weren't subscribers).  

This was as much a gimmick to gain subscriptions as it is to being a genuine poll.

[CARLHAYDEN]

Hey, we all make mistakes.

Actually Vorlon was partially mistaken about the source for the Literary Digest poll.

They sent return mail ballots to subscribers, and to people who had telephones (even if they weren't subscribers).  

This was as much a gimmick to gain subscriptions as it is to being a genuine poll.

Yes that sounds right.  I knew the sample was huge but really flawed.

Good memory you have there!

I find that I can remember the substance of matters, but sometimes have trouble dredging up a citation of source.

[CARLHAYDEN]

I can give you one issue which can be fairly well checked, Kerry voters are pretty closely divided on the death penalty (which Kerry opposes), while both Bush and swing voters overwhelmingly favor it.

Notice how neither side has raised this issue yet?

Kerry is afraid of it (as well as others) while Bush is saving it.

BTW, Public Opinion Strategies is NOT, in MHP, one of the better survey research firms.

[CARLHAYDEN]

FOX poll out has a lot of interesting questions being asked:
http://www.foxnews.com/story/0,2933,120492,00.html

Wow.  Fox News has Kerry ahead +5 in battleground states the same week that the most popular republican president of the 20th century dies.  That is impressive.

Also check out http://www.newdem.org/newmajoritycoalition/ They have Kerry +8 points in battleground states.

Point 1:

Polls are bad for Bush right now.

Point 2:

Battleground sample size in Fox (and most other national polls) ranges from about 240 in Gallup, to about 360 in Fox.  So the margin of error for the quoted "lead" in these subsamples varies from 11% to 14%

Two weeks ago when Bush was up 4% in the Fox "Battleground" it was statistically meaningless, this week when Kerry is up 5% it's just as meaningless.

As I noted some time ago, private surveys which I am prohibted by confidentiality from citing indicate that the Kerry supporters are broken into two approximately evenly divided groups, one of which I call the Deaniacs and the other the traditional Democrats.  

While Bush supporters overwhelmingly agree with him on just about every issue (his support for immigration reform being a notable exception), the traditional Democrats are largely ignorant of and in disagreement with many of Kerry's historical positions (I say this because he appears to being trying to change his positions now).

I am a little surprised that none of the public polls I have seen to date have really probed this matter.

Are you familiar with any which you can cite?

[CARLHAYDEN]

Don't place too much credance in polls between now and labor day.

Summer polls typically give Democrats a few extra points for a number of technical reasons.

Polls which use registered voters typically also have a Democrat bias.

Polls which base their 'likely voter screen' based on following the election (prior to labor day) also have a Democrat bias.

National surveys with fewer than 800 respondents should also be discounted.

I could go on, but, Vorlon has made many of the same points, and could probably give you even more.

I would in conclusion suggest that you need to take polls between now and labor day with more than a pinch of salt.

[CARLHAYDEN]

I expect many of the polls to inch toward Bush because the pollsters don't want to be embarassed by the actual results.

[CARLHAYDEN]

Recently there has been a rather significant divergence between polls from different pollsters as to the present intent of the voters.

While there are a number of possible explanations for this divergence, one which seems to me to explain much of the variation is in the way the particularly polling firm handles the 'likely voter.'

Lets take a concrete example of two radically different approaches.

Potential voter A is 50 years old, and has voted in the 2000, 1996, 1992, 1988.1984. 1980, and 1976 presidential elections.  He says he's leaning to vote for Bush.

Potential voter B is 23, has never voted in any election before in his life (was just registered) and declares that he hates Bush and will vote for "the other guy."

Some firms will look at the history of the voter to determine whether they are likely to vote this year, others will look at the 'intensity' expressed by the respondent to determine whether they will vote this year, and some consider both factors.

Those firms that use (at least primarily) historical voting to project who is more likely to vote, are more likely to give weight to voter A. than voter B.

Contrawise, those firms that use (at least primarily) 'intensity" to project likely voters, are more likely to give weight to voter B.

[CARLHAYDEN]

Rasmussen generally does a good job for the money, but he does have two signficant problems.

First, he weights his polls with a four point Democrat edge.  My data suggests that a one point Democrat edge would be more accurate.

Second, his state polls are often just subsets of his national polls, such that the data for his state polls can be from a two week period (not too reliable).