Polls

[The Vorlon]

Can you assume any states which have not been polled go to their 2000 victor?

Almost certainly.

Some democrats think Virginia may be close, (A POS showed Bush +7 but that poll made almost Doomsday assumptions from a GOP perspective) but I don't think any of the other unpolled states have a ton of suspense left in them...

I think it is safe to put Wyoming in the GOP column and The District of Columbia to Kerry

[The Vorlon]

I knew that, sorry for the miscommunication.

I was asking Dave to start assuming that in his model.

I put all the 2000 results in already.. every state has a result now...

[The Vorlon]

Several things I have seen in a number of polls (different organizations):

1. Bush support is firmer than Kerry's,

2. Bush support is primarily support FOR Bush while half or more of the Kerry support is merely ANTI-BUSH,

3. when the undecideds and leaners are asked to respond to a variety of issue questions they consistently score significantly closer to the core Bush supporters than they do to the core Kerry supporters (or more accurately anti=Bush people).

Correct on #1
Correct on #2

Please provide examples and links on #3

Please & Thank-you

[The Vorlon]

The trick is to find undecided, likely voters rather than just undecided.  I would be surprised if many had very strong conservative or liberal beliefs.

I see Vorlon finally has a Nov 2nd prediction.  You have Kerry doing better in the Southwest than I do.

There is (number varies widely depending on what you look at) "about" 8% of the population called the "Good Citizen" block.

They almost always vote because they are, well, good citizens who think you should vote, but are weakly (if at all) associated with any party or cause.

They are... an adventure... to try to poll properly.

[The Vorlon]

Just saw a Mason - Dixon Poll taken May 20 - 25 which shows Bush up 47 - 41 in Ohio (registered voters).  I'm really confused now.  I look at the national polls and see Bush running slightly behind in some polls and wonder how this can be.  I guess I'm looking for these state polls to be totally reflective of the national data.  I guess it just doesn't work that way always.

link?

[The Vorlon]

Too many people are too trusting of survey results.

First, there is a sample size error, which on national polls for total results, typically runs from more than two to four per cent.

That is to say that a 50-50 tie for the total sample could easily be 52-48, or 48-52 (just considering size alone).

Second, virtually all surveys are telephone polls.  Since federal law prohibits calling a person who might be charged for receiving the call (which is the case with some cell phones), this skews the results.

Third, with systems such as Call Intercept, many landline potential respondents are removed from consideration.

Fourth, a large number of persons answering the phone refuse to participate in the surveys, or do not complete the survey.

Fifth, there is an open question as to how many respondents lie in their responses.

Sixth, the screening of the sample is very important, and problematical.  The better surveys use past voting behavior as more creditable that self-proclaimed intent to vote.

Seventh, the phrasing of the question can change the result.  All to often what is reported is the analysis of the results, rather than the exact wording of the question.

Eighth, the horserace question answers can be affected by other questions preceding the horserace questions.

Ninth, the responses can also vary as to what candidates are included.  Is Nader included?

Tenth, the date(s) the survey questions were posed can affect the result.  Some of the surveys showing Bush doing badly were taken during the period when the liberal media was playing a 24 hours wall to wall coverage of the prison scandal in Iraq.

These are just a few of the matters which affect survey results.  

In short, view all 'poll' results with a jaundiced eye.

If you see 2 or 3 or 4 saying more or less the same thing, there is a "decent" chance its true, but one poll by its self is always a bit suspect.

The quality of the firms doing polls also varies hugely.  They range from really excellent to pretty dodgy.  Take a look at a firms last 100 polls compared to actual results, and consider the source.

A few kinda scary polling stats:

Even assuming a "perfect" poll with no methodological errors of any kind (ya... right....) where the only source or error is random chance.....

A sample size of 500 has a 50% of being with 3% of the "true" value (ie if it says X leads y by 5%, there is only a 50% chance the "true" lead is between 2% and 8%

A sample size of 600 has a 53% of being within 3% of the "true" value

A sample size of 800 has a 60% of being within 3% of the "true" value

A sample size of 1000 has a 66% of being within 3% of the "true" value

A sample size of 1500 has a 75% of being within 3% of the "true" value

<<Sixth, the screening of the sample is very important, and problematical.  The better surveys use past voting behavior as more creditable that self-proclaimed intent to vote>>

Yes, very very true.

Gallup uses a total of 13 questions (some of them with more than 1 part) to sort out "likely" votersas an example.  Thes little 2 and 3 question screens a lot of 2nd teir firms use are better than nothing, but barely....

If you simply ask people if the are "almost certain" or "very likely" to vote, you get a projected turnout of typically 85% or so... as compared to reality which is 50 something...

Your point re cell phones is very important.  Currently about 3% of people have ONLY a cell phone, so they are systemically excluded for polls.  

They tend to be young and typically have low voter turnout so it is not a huge issue yet, but it is a growing and real problem for pollsters.  

It is also a issue I cannot see a solution for that does not involve some fairly aggresive assumptions that I would be pretty uncomfortable building into a poll.

[The Vorlon]

The situation with the cell phones may well be resolved soon. A national cell phone book is in the works listing all cell phone numbers nationwide.

The issue is not availability - it is legal.

It is ILLEGAL under federal law for a polling firm to make a call to a line that may be charged for the call (ie many cell phones) We just simply are no allowed to call cell phones under current law.

NAPOR (the professional association for pollsters) is trying to get the big cell companies to try to find a fix (ie have the polling firm pay for the call) but till then the issue is legal, not technical.

[The Vorlon]

Just saw a Mason - Dixon Poll taken May 20 - 25 which shows Bush up 47 - 41 in Ohio (registered voters).  I'm really confused now.  I look at the national polls and see Bush running slightly behind in some polls and wonder how this can be.  I guess I'm looking for these state polls to be totally reflective of the national data.  I guess it just doesn't work that way always.

A poll with a poorly sorted out screen for likely voters will show big, and not real, shifts in public opinion in reaction to events like the prison thing.

Remember that in a lot of these recent national registered voter polls, about 1/3rd of the people included will not be actual voters, and many of them when the "leaners" are pushed will just sway in the wind with the last news story they heard on the nightly news...

(The Mason Dixon is Likely voters BTW - MD's are NEVER registered)

The poll, conducted May 20-25 by Mason-Dixon Polling and Research, is based on interviews with 1,500 registered voters who plan to vote in November. The poll, which has a margin of error of 2.6 percentage points, is the largest presidential poll taken in Ohio this year. [/i]

The typical "bounce" from the conventions is "about" 10% - after the first convention candidate X goes up 10%, then after the other convention candidate Y goes up 10%.

Is this swing of 10% "real"...?   of course not...

A bunch of folks who are not voting anyway are just swaying in the wind.

I a, not picking on Survey USA here, (just have a link handy) they are a loooong way from the worse firm out there, but their last Iowa poll talked to 879 people 794 of whom were deemed "likely" - I have no idea what Iowa turnout will be in 2004, but I happily bet you a reasonably fresh bag of donuts it will not be 794/879 = 90% !

1/4 to 1/3 of the people in that poll simply will not vote.  This makes the validity of the poll suspect.  And a lot of that 1/4 to 1/3 just sway in the wind based upon the last news story they saw.

[The Vorlon]

But he says its a Mason-Dixon poll.  You have a very high opinion of Mason-Dixon, I thought.

Mason-Dixon - absolutely a VERY good firm.  At the state by state level easily[/b] the best firm out there.  Frankly, compared to a lot of the other firms, it's not even close.

MD is the only firm in the public domain I actually trust from here to September that does state polls. (Maybe Ipsos too actually)

I was talking "generically" in this post.

Mason Dixon's record speaks for it's self.  In 2002 they publically polled 23 races and got 22 right.  Their average candidate error was 1.8%.  (If you limit it to polls published in the last 7 days of the campaign they did even better) The other 70 or so private polls they did in 2002 had a comparable success rate.

The only firm with lower average candidate error was Gallup, and there is certainly no shame in coming in 2nd to Gallup..

[The Vorlon]

So does this particular Ohio poll seem to have any validity to you given the fact that it doesn't seem in sync with other Ohio polling?  Also given the fact that it seems to come at a time when Bush has hit bottom in job approval and given all the $$ Kerry has spent over the last 3 wks in television ads?

One poll is, well, one poll.  My "map" had Bush up a couple as of yesterday, so this is not a shock to me at least.

I was serious when I said if a summer poll doesn't say Mason Dixon you should burn it.

Most 2nd teir (or worse) firms will underpoll the GOP in the summer which explains a few of the odd results if late. (See a few posts back re likely voters and turnout)

That being said, it basically agrees with two othe private polls showing Bush +3 and +4. so I wouldn't take it a the gospel truth, but I would not toss it either.

A Mason-Dixon with a 1500 sample size simply cannot be ignored

The sample size of 1500 was the choice of the newspaper that sponsored (ie paid for) the poll, not M/D.

If somebody want to pay the bill, M/D and any other firm would love a 10,000 sample size

Let's give the Post Dispatch a brownie point for stepping up with the big bucks to pay for both a good firm and a big sample.

I hope the way they rather proudly proclaimed the sample size will prompt other papers to step up to the plate and bump up their samples too..

[The Vorlon]

But he says its a Mason-Dixon poll.  You have a very high opinion of Mason-Dixon, I thought.

Mason-Dixon - absolutely a VERY good firm.  At the state by state level easily[/b] the best firm out there.  Frankly, compared to a lot of the other firms, it's not even close.

MD is the only firm in the public domain I actually trust from here to September that does state polls. (Maybe Ipsos too actually)

I was talking "generically" in this post.

Mason Dixon's record speaks for it's self.  In 2002 they publically polled 23 races and got 22 right.  Their average candidate error was 1.8%.  (If you limit it to polls published in the last 7 days of the campaign they did even better) The other 70 or so private polls they did in 2002 had a comparable success rate.

The only firm with lower average candidate error was Gallup, and there is certainly no shame in coming in 2nd to Gallup..

First, welcome back.  Hope you had a good vacation.

Second, thank you for your observations on the previous post to me.

Third, I agree on MD being a quality firm, I have my doubts about ISPOS (they seem to do ok in Canada but, IMHO they tend to overstate the liberal and Democrat support in this country).

Fourth, while the numbers aren't "hard," and aren't kept on official record, it appears that in the battleground states (which I define as being a state which either Bush or Gore carried with a margin of less than ten per cent) the "refuse" to participate in the surveys is highest among males whose age group is estimated to be 45-64 (without noticable accents).  

While you can (eventually) get enough 45-64 year old males to meet quota, I maintain that those responding do not necessarily reflect the views of those who refuse to participate.  

I believe that the exit polls should be funded by a consortium of existed reputable pollsters, who could use the data base to adjust their future polls.  This would probably be able to either confirm or deny my suspicion on this matter.

Fifth, also, please note that even when reading the questions verbatim to the respondents, the accent of the questioner appears to have some impact on the responses.  This is particularly important as firms do not pay particularly well for the questioners and tend to hire the less affluent (and more prone to regional accent) segment of the population.  

You point regard respone rates is also valid, and touchs on a big question re sample stratification.

You eventually meet your quota, you have enough whites, blacks, young, old, rich poor, etc...

I looks like a ramdom sample.  Does that mean it is a random sample?
Regarding

[The Vorlon]

Vorlon. Why doesnt some rich financier come up with the funds to have a 500,000 person sample? With todays modern technology that would be more then possible. Put some of those telemarkters who lost their jobs because they have been outsourced to work!


Vorlon, that wasn't a joke. It's a serious question.

A poll has two types of error - pure random error due to statistical fluctuation, and methodological error.

Unless you are talking a VERY good firm, methodological errors are typically larger than statistical ones.

A big sample will reduce the random statistical error, but does nothing about methodological error.

If you toss a coin 100 times, it might come up 50/50 heads/tails, but usually 51/49, or 52/48 or there abouts.  Once in a blue moon it comes up 80/20.  This is statistical or random error.  Nothing you can do about it.

To extend my example, let's say I have a problem with my coins and 80 of the have a head and a tail but 20 of them are two headed.

Over time, the 80 normal coins will tend towards 40/40, but the 20 two headed coins will always be 20 heads.

I can have an infinite sample and flip the coins a zillion times and I will typically get 60/40 rather than 50/50.  This is a methodological error.

For example if I was to make some crazy assumption and say weight my poll so that Republicans made up 50% of the sample or that there were 7% more Democrats than Republicans for example, I could talk to 10 million people and still get a garbage result.

The reduction in statistical error is proportional to the square root (not exactly but damn close) of the sample size, so you get to the point of diminish returns pretty fast.

For example, a sample of 1000 has an error of 3.1%
A sample size of 2000 has an error of 2.2%

You have doubled your costs to pick up 0.9%

A sample size of 10,000 has an error of 1.0%

I have made my sample 5 times bigger to get another 1.2%

A sample of 100,000 has an error of 0.31

Compared to 1000 size, we had to make our poll 100 times bigger to get 10 times more accurate.

A huge sample also does NOTHING to get rid of methodological errors - problems in how you have worded your question, how you select people, etc....

To take an extreme example, lets say I polled 10,000 "likely" voters.  In this case I defined "likely" as owning a BMW. (it's a silly example but it makes my point)

My sampling error on 10,000 is only 1%, but my methodological error due to a silly sampling method would render the poll likely to be waaaay out because 10,000 BMW owners are not a representitive sample of actual voters.

In 1936 Readers Digest (I think?) Magazine did a poll of 3,000,000 (?) Americans - all of whom susbscribed to Readers Digest and had telephones.  Sampling error should have been pretty close to zero.  Unfortunately in 1936 both telephones and magazine subscriptions were luxury items so the 3,000,000 polled looked very little like America.  In this poll "President" Alf Landon beat Rossevelt by about 15% if I recall corectly.

To use a recent example, here is Mr. Zogby's results from his 2002 senate tracking polls.



Clearly sample size is NOT the issue, the errors are just too many and too big to be statistical bad luck.

Something in they way he weighted/sampled/worded was just plain wrong (and hopefully corrected for 2004)

A bigger sample would NOT have corrected these problems.

Bottom line - a bad poll with a big sample is still a bad poll.

Regarding the cost of a poll.

A sample of 1500 from a good firm doing it totally absolutely by the book costs in the neighbourhood of $30,000 to $40,000.

If you poll is all of America or a state or just a city, the cost varies fairly little actually.

There are firms that change a heck of a lot less.

You get what you pay for.

[The Vorlon]

It was Liberty Digest and it went out of business shortly afterwards.

Thank-you.

 I knew it was xxx Digest, could not quite recall the name.

[The Vorlon]

Hey, we all make mistakes.

Actually Vorlon was partially mistaken about the source for the Literary Digest poll.

They sent return mail ballots to subscribers, and to people who had telephones (even if they weren't subscribers).  

This was as much a gimmick to gain subscriptions as it is to being a genuine poll.

Yes that sounds right.  I knew the sample was huge but really flawed.

Good memory you have there!

[The Vorlon]

3. when the undecideds and leaners are asked to respond to a variety of issue questions they consistently score significantly closer to the core Bush supporters than they do to the core Kerry supporters (or more accurately anti=Bush people).

*************************

Unfortunately, the polls which have explicity differeniated between the beliefs of Kerry voters, Bush voters and undecided voters on the issues are not available for publication at this time.

The sources which have commissioned the surveys are more interested in using the data than in publishing it at this time.

Boy, theoretically speaking I suspect it's barely conceivable that a Public Opinion Strategies poll from May 17th-23rd of 1137 likely voters nationwide (which may have been oversampled in 7 key states to provide better breakouts) could have hypothetically shown something similar to that.  

But that's just a totally random guess of course.

[The Vorlon]

Hehe...Vorlon, once again I explained something in a much shorter post and without images...

A way bak on another thread, but I made teh same point on marginal returns.

Here we go:

It's not worth it. The increase in accuracy becomes less and less cost-effective, simply. So you'd pay a lot of money and get very little for it. You would always have the MoEs anyway.  

ok, I admit it.  you are the literary xxx Liberty Reader Digest version of things.

But I make prettier diagrams than you do...

Damn swedish punks... always making fun of the dyslexic, forgetful, long winded old guys with bad reading and typing skills....

[The Vorlon]

The 2000 results on the Polls page for Mississippi are not correct...Bush won by 16%, not 8%.

Corrected & Thank-you!

[The Vorlon]

The polls page map doesn't line up with the most recent polls.  I assume that the map is supposed to reflect the latest results, however a 3% lead in MN is a lean but a 9% lead in MI is a tossup according to the map.  What's up with that?

I "think" dave has something in there that averages a number of polls - I do not know the fomula he is using.

Dave...?

[The Vorlon]

Vorlon-

How do you rate Garin-Hart-Yang?

They have a new poll from 10 days ago showing Bush-Kerry 49-43.

http://www.drdan2004.com/site/DocServer/Poll_Memo_5.25.04.pdf?docID=221

Peter Hart's firm A++

That said, the sample of 504 makes me think is was a "fundraising" poll versus an actual poll.

Often a poll (especially a small sample) will be done with, um,  favorable assumptions re turnout/etc to rally the troops, drive fundraising, etc.

This one was also done for Senate candidate.

If Peter actually wanted to do a state, I can guarantee you it would not be 504 sample size.

[The Vorlon]

Just a side note - Zogby's final poll in the SD congressional race had the Dem up 11 points.  Hmmm.

http://www.uselectionatlas.org/FORUM/index.php?board=5;action=display;threadid=4306&start=0

Side Note - Mason-Dixon Poll May 18th of "100% Certain" voters - Dems by 3 versus actual of 1.2%.

[The Vorlon]

Vorlon,

Found one for you which you may or may not want to rip apart:

http://www.zogby.com/Soundbites/ReadClips.dbm?ID=8387

"49.7 per cent of respondents would vote for the prospective Democratic nominee, while 44.3 per cent would support Republican incumbent George W. Bush."

"Independent candidate Ralph Nader garners the support of 2.9 per cent of respondents. The presidential election is scheduled for Nov. 2."

Odd how just two weeks ago, the polls showed Bush ahead 44% to 39%.  Anyway, for your info.

It is an INTERNET poll.  Enough said

[The Vorlon]

FOX poll out has a lot of interesting questions being asked:
http://www.foxnews.com/story/0,2933,120492,00.html

Wow.  Fox News has Kerry ahead +5 in battleground states the same week that the most popular republican president of the 20th century dies.  That is impressive.

Also check out http://www.newdem.org/newmajoritycoalition/ They have Kerry +8 points in battleground states.

Point 1:

Polls are bad for Bush right now.

Point 2:

Battleground sample size in Fox (and most other national polls) ranges from about 240 in Gallup, to about 360 in Fox.  So the margin of error for the quoted "lead" in these subsamples varies from 11% to 14%

Two weeks ago when Bush was up 4% in the Fox "Battleground" it was statistically meaningless, this week when Kerry is up 5% it's just as meaningless.

[The Vorlon]

I just read this morning that pollster Robert Teeter died. He's been one of Vorlon's top rated pollsters, and works with Hart for the WSJ poll. Will his death have an impact on that poll's performance?

Yes, I noticed that also.  I worked with Mr. Teeter for 1 election cycle in 1994 and he was truly a wonderful man as well as a great pollster.

He had an absolute dedication to "getting it right" and making sure all the tiny details of how you did things were all done properly - it was annoying at the time, but in retrospect he was just about always correct in what he said.

Bob has taught so many people over his life that I don't think the accuracy of his firms polls will go down, he has many many proteges who will carry the torch.  

More than perhaps any other, he will be remembered as "teacher" as much as a pollster.  You can look at many of the top firms today on the GOP side (Ayres McHenry, POS etc) and they all have direct links back to Bob.

http://www.freep.com/news/latestnews/pm20244_20040614.htm

[The Vorlon]

When will the polling section be updated?

I would do it if tohers are to busy, but I'm not allowed too, and I'll, be away for a good portion of Asugust.  But starting in sept. I could update it almost daily.

Just e-mail Dave, he will give you a password

[The Vorlon]

It looks like state polls are getting into the polling section regularly, but national polls are way behind. There was some discussion awhile ago about putting in the weekly Rasmussen, it hasn't been updated since May.

I believe this link to National poll averages it right up to date.

Link to poll averages