538 On the Cell Effect (We keep hitting controversial polling topics, haha)

[Lunar]

Take it how you will.  I'm not endorsing this.

Mark Blumenthal has a rundown of the pollsters that are including cellphone numbers in their samples. Apparently, Pew, Gallup, USA Today/Gallup (which I consider a separate survey), CBS/NYT and Time/SRBI have been polling cellphones all year. NBC/WSJ, ABC/Washington Post and the AP/GfK poll have also recently initiated the practice. So too does the Field Poll in California and Ann Selzer. There may be some others too but those are the ones that I am aware of. (EDIT: The director of the PPIC survey in California has kindly written to let me know that, while they use a cellphone supplement for some of their public policy surveys, they have not done so thus far this year for their Presidential trial heats. The remainder of this article has been corrected accordingly.

Let's look at the house effects for these polls -- that is, how much the polls have tended to lean toward one candidate or another. These are fairly straightforward to calculate, via the process described here. Essentially, we take the average result from the poll and compare it to other polls of that state (treating the US as a 'state') after adjusting the result based on the national trendline.

Since ABC, NBC/WSJ and AP/GfK all just recently began using cellphones, we will ignore their data for now. We will also throw out the data from three Internet-based pollsters, Zogby Interactive, Economist/YouGov, and Harris Interactive. This leaves us with a control group of 36 37 pollsters that have conducted at least three general election polls this year, either at the state or national level.

Pollster                 n   Lean
=========         ====
Selzer                    5   D +7.8
CBS/NYT               14   D +3.7
Pew                        7   D +3.4
Field Poll                 4   D +2.8
Time/SRBI               3   D +2.4
USA Today/Gallup 11   D +0.4
Gallup                 184   R +0.6

AVERAGE                      D +2.8


CONTROL GROUP (37 Pollsters) D +0.0

Six of the seven cellphone-friendly pollsters have had a Democratic (Obama) lean, and in several cases it has been substantial. On average, they had a house effect of Obama +2.8 +2.3. By comparison, the control group had essentially zero house effect (**), so this would imply that including a cellphone sample improves Obama's numbers by 2.8 points. (Or, framed more properly, failing to include cellphones hurts Obama's numbers by 2-3 points).

The difference is statistically significant at the 95 percent confidence level. Perhaps not coincidentally, Gallup, Pew and ABC/WaPo have each found a cellphone effect of between 1-3 points when they have conducted experiments involving polling with and without a cellphone supplement.

A difference of 2-3 points may not be a big deal in certain survey applications such as market research, but in polling a tight presidential race it makes a big difference. If I re-run today's numbers but add 2.2 points to Obama's margin in each non-cellphone poll, his win percentage shoots up from 71.5 percent to 78.5 percent, and he goes from 303.1 electoral votes to 318.5 (EDIT: I have not changed this part of the analysis in reflection of the new numbers, as it should still get the general point across). The difference would be more pronounced still if Obama hadn't already moved ahead of McCain by a decent margin on our projections.



So this is my plea to pollsters: let's get it right. Perhaps the cellphone effect will prove to be a mirage after all, but that's something for the data to determine on its own, rather than the pollster.

(**) Keen observers will wonder why the average house effect is greater than zero. This is because in determining our house effect coefficients, we weight based on how many polls each pollster has conducted. A couple of pollsters that account for a large proportion of our data, like Rasmussen and ARG, have had slight (very slight, but enough to skew the numbers) GOP leans.

http://www.fivethirtyeight.com/2008/09/estimating-cellphone-effect-22-points.html

[Alcon]

I think there are other properties of a pollster that might correlate with their likelihood of contacting cell phone voters, and it's a small sample size.

But, still, that's a little chilling.

[Sam Spade]

You do realize that Pew, CBS/NYT and Time have always had a historical Democratic lean, even before the invention of the cell phone.

Meanwhile, Gallup shows nothing.

[Alcon]

You do realize that Pew, CBS/NYT and Time have always had a historical Democratic lean, even before the invention of the cell phone.

My essential problem.

However, I think we're kidding ourselves if we think that this phenomenon isn't going to show up eventually.  How could it not?

[Lunar]

You do realize that Pew, CBS/NYT and Time have always had a historical Democratic lean, even before the invention of the cell phone.

My essential problem.

However, I think we're kidding ourselves if we think that this phenomenon isn't going to show up eventually.  How could it not?

Indeed, and never before have young, educated voters been so enthusiastic.

But Sam's right, which is why I didn't endorse 538's analysis.

[Sam Spade]

You do realize that Pew, CBS/NYT and Time have always had a historical Democratic lean, even before the invention of the cell phone.

My essential problem.

However, I think we're kidding ourselves if we think that this phenomenon isn't going to show up eventually.  How could it not?

I never said it wouldn't show up, Alcon.  I believe I said so in the other thread on the matter with Lunar. 

But this analysis does nothing to prove to me at present that the *cellphone effect* is any different than the *weekend bounce* JJ talks about.  I think both get lost in the noise.

[Wiz in Wis]

You do realize that Pew, CBS/NYT and Time have always had a historical Democratic lean, even before the invention of the cell phone.

Meanwhile, Gallup shows nothing.

Citation? When presented with numbers, retort with numbers

[??????????]

This assumes that the majority of those who own cellphones are liberal leaning Democrats. Cellphone ownership and usage knows no political bounds.

[Alcon]

This assumes that the majority of those who own cellphones are liberal leaning Democrats. Cellphone ownership and usage knows no political bounds.

Exclusive cell phone ownership is drastically higher among the young.

[??????????]

This assumes that the majority of those who own cellphones are liberal leaning Democrats. Cellphone ownership and usage knows no political bounds.

Exclusive cell phone ownership is drastically higher among the young.

Really? Most of the people I know in their 30's and 40's own a cell phone. I know very very few who do NOT have one. I'm not saying what you say is wrong, I'm just stating an observation.

[Lunar]

We're talking about people who don't have landlines, States.

If you read the article, it claims that seven out of eight cell-only users have favorable opinions of Obama.

[they don't love you like i love you]

This assumes that the majority of those who own cellphones are liberal leaning Democrats. Cellphone ownership and usage knows no political bounds.

Exclusive cell phone ownership is drastically higher among the young.

Really? Most of the people I know in their 30's and 40's own a cell phone. I know very very few who do NOT have one. I'm not saying what you say is wrong, I'm just stating an observation.

Key word bolded.

[??????????]

We're talking about people who don't have landlines, States.

If you read the article, it claims that seven out of eight cell-only users have favorable opinions of Obama.

Yeah, but they are young voters and we know their history of reliability when election time rolls around. So it probably evens out.

[Lunar]

We're talking about people who don't have landlines, States.

If you read the article, it claims that seven out of eight cell-only users have favorable opinions of Obama.

Yeah, but they are young voters and we know their history of reliability when election time rolls around. So it probably evens out.

Well, yeah, that's the common sense counter-argument that comes out, hah.  The point of the article is that it might be finally having an effect.  As Sam said though, the analysis is suspect.

But, like I said, young educated voters are part of Obama's base vote, far more so than Kerry or Gore.  Take that how you will.

[Nym90]

We're talking about people who don't have landlines, States.

If you read the article, it claims that seven out of eight cell-only users have favorable opinions of Obama.

Yeah, but they are young voters and we know their history of reliability when election time rolls around. So it probably evens out.

In the past that's definitely been true. Turnout among young voters was far higher in the primaries this year than before; remains to be seen whether that will carry over into the general.

[Bacon King]

We're talking about people who don't have landlines, States.

If you read the article, it claims that seven out of eight cell-only users have favorable opinions of Obama.

Yeah, but they are young voters and we know their history of reliability when election time rolls around. So it probably evens out.

True, but pollsters already screen for likely voters and all.

[Gustaf]

You do realize that Pew, CBS/NYT and Time have always had a historical Democratic lean, even before the invention of the cell phone.

Meanwhile, Gallup shows nothing.

Citation? When presented with numbers, retort with numbers

Read the numbers we are presented with in the opening post for the second part. The bias of polls like CBS/NYT is very well known.

Is it just me, or does there seem to be a correlation between bad polls and polls that include cell-phone users?

[Sam Spade]

We're talking about people who don't have landlines, States.

If you read the article, it claims that seven out of eight cell-only users have favorable opinions of Obama.

Yeah, but they are young voters and we know their history of reliability when election time rolls around. So it probably evens out.

In the past that's definitely been true. Turnout among young voters was far higher in the primaries this year than before; remains to be seen whether that will carry over into the general.

Ya, but that's not the important figure.  In order to conclude that we're understimating young (18-29) voters who might be cellphone-only, and that this may skew things towards Obama, first, young voters must greatly increase their turnout as compared to any increase in older voter turnout.  If not, any cellphone dilemma will get lost in the noise.

Moreover, I really doubt that there is much distinction in voting patterns between young voters who are cellphone-only and young voters who are not cellphone-only.  For anecdotal evidence, keep in mind that I still qualify as a young voter who is cell-phone only. 

[JohnCA246]

Im really hoping this  cell phone effect turns out but here are some questions.  Don't poll samples already have a sample of that demographic?  So I don't really see cell phone only young people as being more likely to vote Obama than landline young people.  Or does not including cell phones throw off their sample size of young voters altogether?

Finally, how exactly does a pollster find cell phone numbers?   

[Gustaf]

Im really hoping this  cell phone effect turns out but here are some questions.  Don't poll samples already have a sample of that demographic?  So I don't really see cell phone only young people as being more likely to vote Obama than landline young people.  Or does not including cell phones throw off their sample size of young voters altogether?

Finally, how exactly does a pollster find cell phone numbers?   

You may a) believe that "cell phone only" young are more likely to vote Obama than "landline young" or b) that their likeliness to vote has increased but doesn't show because they're not included. Though I think this second part would also demand a difference between cell-phone young and landline young.

[Sbane]

The cell phone effect doesn't happen mainly because the difference in voting between landline youngs and cell phone youngs is minimal. It's not as if only conservative younguns have landlines and only liberals have cell phones. It just depends from person to person and I doubt it is political.

[they don't love you like i love you]

The cell phone effect doesn't happen mainly because the difference in voting between landline youngs and cell phone youngs is minimal. It's not as if only conservative younguns have landlines and only liberals have cell phones. It just depends from person to person and I doubt it is political.

But it results in the young vote being undersampled.

[The Vorlon]

It would seem to me the question is not if cell phone users vote differently, but rather do they differ from non-cell phone users of similar demographic charateristics.

There is no doubt that the 21 year old cell phone only demographic is substantially different than the 47 year old land land owning demographic, but that is not the question..

Pollsters already ensure that young people are appropriately represented in the sample, and that their income, education, etc matches the totality of their demorgraphic -the isolated variable here is just the cell phones, not age, income, education, etc...

IE is the totality of 23 year old males with just a cell phone intending to vote differently that the totality of 23 year old males who may also have a landline?

Gallup BTW is trying to isolate this by doing calls to cell phone exchanges and then askng as one of the screening questions if the cell phone user is cell phone only.. this way they can build up a database on cell phone only users versus cell phone + land line users.

The next issue is VOIP users... but that is another ball of wax

[Gustaf]

The cell phone effect doesn't happen mainly because the difference in voting between landline youngs and cell phone youngs is minimal. It's not as if only conservative younguns have landlines and only liberals have cell phones. It just depends from person to person and I doubt it is political.

But it results in the young vote being undersampled.

Only if young cell phone only users increase their turnout while no one else does. Or, I guess, if the pollster is completely clueless when it comes to weighting.

[Alcon]

Only if young cell phone only users increase their turnout while no one else does. Or, I guess, if the pollster is completely clueless when it comes to weighting.

It's not a relative increase in turnout, but a relative increase in the demographic size, probably combined with a more Democratic result among them than previously.