AAPOR hits YouGov/NYT/CBS for polling methodology
       |           

Welcome, Guest. Please login or register.
Did you miss your activation email?
March 28, 2024, 07:12:37 PM
News: Election Simulator 2.0 Released. Senate/Gubernatorial maps, proportional electoral votes, and more - Read more

  Talk Elections
  Election Archive
  Election Archive
  2014 Gubernatorial Election Polls
  2014 Senatorial Election Polls
  AAPOR hits YouGov/NYT/CBS for polling methodology
« previous next »
Pages: [1]
Author Topic: AAPOR hits YouGov/NYT/CBS for polling methodology  (Read 2351 times)
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« on: August 03, 2014, 09:15:05 AM »

The leading organization representing the nation’s pollsters criticized CBS News and the New York Times on Friday for releasing results of a nationwide poll the survey-researchers organization said was conducted using an unproven methodology.

Last Sunday, the two news organizations unveiled their “Battleground Tracker,” an online survey updated each month. The poll — conducted by Internet pollster YouGov — interviewed more than 100,000 people nationwide, the news organizations said, allowing them to project results for each Senate race in the country.

The results were featured on CBS’s “Face the Nation,” and the New York Times “Upshot” data-driven vertical published multiple entries about the survey. The first edition of the “Battleground Tracker” showed Republicans leading in 51 Senate races, which prompted The Upshot to project that Republicans had a 60 percent chance of winning control of the Senate.

The American Association for Public Opinion Research’s statement on Friday criticized CBS and the Times for using a survey method that has “little grounding in theory” and for a lack of transparency.

“[M]any of the details required to honestly assess the methodology remain undisclosed,” according to the AAPOR statement, issued under the organization’s letterhead and signed by president Michael Link. “This may be an isolated incident with the Times / CBS News providing more information on this effort in the coming weeks. If not, it is a disappointing precedent being set by two of our leading media institutions.”

CBS News elections director Anthony Salvanto disputed that charge, saying in a statement that CBS disclosed the methodology for the study “in great detail.”

http://www.politico.com/blogs/media/2014/08/cbs-new-york-times-hit-for-polling-standards-193303.html

...

It's better not to enter the YouGov polls, because of this.

I have deleted all the YouGov polls that were entered to the database, other than for states that have not been polled yet.
Logged
Senator Cris
Cris
Junior Chimp
*****
Posts: 5,613
Italy


Show only this user's posts in this thread
« Reply #1 on: August 03, 2014, 10:22:42 AM »

Why the YouGov polls were not included in our database?
Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #2 on: August 03, 2014, 11:58:42 AM »

Why the YouGov polls were not included in our database?

Because the AAPOR (the umbrella group of US pollsters) have big doubts about their methods.

Which means I don't enter them and it's up to Dave to decide if they should be added or not.

It's his website, not mine.

Wink

So, please ask Dave what to do.
Logged
Lief 🗽
Lief
Atlas Legend
*****
Posts: 44,876


Show only this user's posts in this thread
« Reply #3 on: August 03, 2014, 12:00:36 PM »

AAPOR has also criticized Rasmussen in the past. We're still adding their polls to the database.
Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #4 on: August 03, 2014, 12:16:00 PM »

AAPOR has also criticized Rasmussen in the past. We're still adding their polls to the database.

Really ? That's interesting. Link ?

...

In general, I don't really have much of a problem with adding the YouGov polls to the database - mostly because their 2008 and 2012 polls from all states have generally been pretty good (and I think they were already online-polls as well, so what's the big deal about them now ?).

But it's really for Dave to decide what to do ...

I'm just saying we shouldn't enter them right now, because of the AAPOR concern (but I guess they could only be afraid that their majority-phone pollster members are about to go bankrupt, if more companies in the future turn to internet-only polls and they see NYT/CBS as a threat who started all of this ...)
Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #5 on: August 03, 2014, 12:22:34 PM »

538's analysis of 2012 pollster performance found that YouGov was somewhere in the middle when it comes to accuracy, slightly better even than PPP in bias:



In general, Internet-pollsters were even the best in terms of accuracy - compared with live operator pollsters and IVR pollsters (bottom of the chart):

Logged
Attorney General, LGC Speaker, and Former PPT Dwarven Dragon
Dwarven Dragon
Atlas Politician
Atlas Superstar
*****
Posts: 31,578
United States


Political Matrix
E: -1.42, S: -0.52

P P P

Show only this user's posts in this thread
« Reply #6 on: August 03, 2014, 12:29:15 PM »

AAPOR has also criticized Rasmussen in the past. We're still adding their polls to the database.

Really ? That's interesting. Link ?

...

In general, I don't really have much of a problem with adding the YouGov polls to the database - mostly because their 2008 and 2012 polls from all states have generally been pretty good (and I think they were already online-polls as well, so what's the big deal about them now ?).

But it's really for Dave to decide what to do ...

I'm just saying we shouldn't enter them right now, because of the AAPOR concern (but I guess they could only be afraid that their majority-phone pollster members are about to go bankrupt, if more companies in the future turn to internet-only polls and they see NYT/CBS as a threat who started all of this ...)
The other big deal about the YouGov polls is that this year they're making tons of obvious mistakes, such as showing durbin and Tom udall in close races, showing Franken as safer than warner, showing Brownback ahead by double digits, deal ahead by 9, showing tillis ahead when everyone else has him behind, showing land ahead when ever one else has her behind, showing md-gov equally competitive as az-gov, and showing mead (WY gov.) barely over 50%. It also refused to include Cutler in me-gov. The yougov polling as a whole is so bad that none of it should be in the database.
Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #7 on: August 03, 2014, 12:33:46 PM »

AAPOR has also criticized Rasmussen in the past. We're still adding their polls to the database.

Really ? That's interesting. Link ?

...

In general, I don't really have much of a problem with adding the YouGov polls to the database - mostly because their 2008 and 2012 polls from all states have generally been pretty good (and I think they were already online-polls as well, so what's the big deal about them now ?).

But it's really for Dave to decide what to do ...

I'm just saying we shouldn't enter them right now, because of the AAPOR concern (but I guess they could only be afraid that their majority-phone pollster members are about to go bankrupt, if more companies in the future turn to internet-only polls and they see NYT/CBS as a threat who started all of this ...)
The other big deal about the YouGov polls is that this year they're making tons of obvious mistakes, such as showing durbin and Tom udall in close races, showing Franken as safer than warner, showing Brownback ahead by double digits, deal ahead by 9, showing tillis ahead when everyone else has him behind, showing land ahead when ever one else has her behind, showing md-gov equally competitive as az-gov, and showing mead (WY gov.) barely over 50%. It also refused to include Cutler in me-gov. The yougov polling as a whole is so bad that none of it should be in the database.

Some of their results look really off, yeah.

But I wouldn't necessarily say that say Deal+9 or Brownback by 11 is wrong (it really depends how many Republicans and Indies are backing them, if 90% of Republicans back them in the end, or 65% in a heavy GOP-state like KS - that could create a difference of 10-20 points between 2 pollsters).

Other results you posted like NC, are well within the MoE.

Some of their mistakes are pretty bad though, excluding Cutler in Maine for example (they actually included the 3rd party candidate in VT, so why not Maine if Maine is so pro-Indy ?)

I hope they figure out these mistakes in their next waves.
Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #8 on: August 03, 2014, 12:47:13 PM »

Also consider the "1-in-20 rule":

If YouGov polls all Senate races (36 this year), there's a chance that at least 2 of them will be really off, outside the MoE.

If we assume that the "off races" are MI and VA in the Senate wave and KS and GA in the Governor wave, the results for the other 34 races are more in line with other recent polls.

(ME of course was way off, because they failed to include Cutler).
Logged
Queen Mum Inks.LWC
Inks.LWC
Atlas Superstar
*****
Posts: 35,028
United States


Political Matrix
E: 4.65, S: -2.78

P P

Show only this user's posts in this thread
« Reply #9 on: August 03, 2014, 05:36:25 PM »

Why the YouGov polls were not included in our database?

Because the AAPOR (the umbrella group of US pollsters) have big doubts about their methods.

Which means I don't enter them and it's up to Dave to decide if they should be added or not.

It's his website, not mine.

Wink

So, please ask Dave what to do.

If you're going to defer to Dave (as I think you should), then it's probably best not to go around unilaterally deleting polls until he actually tells you to do so.
Logged
Mr. Morden
Atlas Legend
*****
Posts: 44,073
United States


Show only this user's posts in this thread
« Reply #10 on: August 03, 2014, 08:11:48 PM »

I don't see why a pollster should be excluded because their *results* are poor.  Otherwise, why didn't we ban ARG from the database years ago?  Rather, they should be banned if their methodology is questionable.  But is YouGov's methodology different from other internet-based polls?  Is it different from what it used to be?  Should all internet-based polls be banned?
Logged
Keystone Phil
Atlas Institution
*****
Posts: 52,611


Show only this user's posts in this thread
« Reply #11 on: August 03, 2014, 08:18:50 PM »

Give it a month or so and certain people here will start demanding that only polls showing the Dems in the lead should be entered.
Logged
IceSpear
Atlas Superstar
*****
Posts: 31,841
United States


Political Matrix
E: -6.19, S: -6.43

Show only this user's posts in this thread
« Reply #12 on: August 03, 2014, 08:28:24 PM »

I don't see why a pollster should be excluded because their *results* are poor.  Otherwise, why didn't we ban ARG from the database years ago?  Rather, they should be banned if their methodology is questionable.  But is YouGov's methodology different from other internet-based polls?  Is it different from what it used to be?  Should all internet-based polls be banned?

I'm surprised it wasn't. ARG is garbage.
Logged
Senator Cris
Cris
Junior Chimp
*****
Posts: 5,613
Italy


Show only this user's posts in this thread
« Reply #13 on: August 04, 2014, 03:50:40 AM »

Give it a month or so and certain people here will start demanding that only polls showing the Dems in the lead should be entered.
Logged
Senator Cris
Cris
Junior Chimp
*****
Posts: 5,613
Italy


Show only this user's posts in this thread
« Reply #14 on: August 04, 2014, 06:23:45 AM »

Also (for those who do not want that the YouGov polls are entered in the database because of their "weird numbers")... Why the Marist poll that give Udall leading by 7 was entered in the database, with all the Colorado polls that show a dead heat?
Logged
King
intermoderate
Atlas Star
*****
Posts: 29,357
United States


Show only this user's posts in this thread
« Reply #15 on: August 04, 2014, 10:22:49 AM »

I can't speak for the rest, but the NM poll has Udall up just 52-44 topline with 51% of whites, 72% of hispanics, and 96% of blacks in crosstabs. That's an impossible outcome for this state in any election. He should be up 62-34 with that kind of demo support.
Logged
Anti Democrat Democrat Club
SawxDem
Atlas Icon
*****
Posts: 14,094
United States


Show only this user's posts in this thread
« Reply #16 on: August 04, 2014, 11:17:24 AM »

Also (for those who do not want that the YouGov polls are entered in the database because of their "weird numbers")... Why the Marist poll that give Udall leading by 7 was entered in the database, with all the Colorado polls that show a dead heat?

Marist has a good track record. YouGov does not.
Logged
Senator Cris
Cris
Junior Chimp
*****
Posts: 5,613
Italy


Show only this user's posts in this thread
« Reply #17 on: August 04, 2014, 11:39:05 AM »

Also (for those who do not want that the YouGov polls are entered in the database because of their "weird numbers")... Why the Marist poll that give Udall leading by 7 was entered in the database, with all the Colorado polls that show a dead heat?

Marist has a good track record. YouGov does not.

538's analysis of 2012 pollster performance found that YouGov was somewhere in the middle when it comes to accuracy, slightly better even than PPP in bias:



In general, Internet-pollsters were even the best in terms of accuracy - compared with live operator pollsters and IVR pollsters (bottom of the chart):



Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #18 on: August 04, 2014, 12:16:44 PM »

Here you can see YouGov's final 2008 polls in all states:

http://voteforamerica.net/editorials/Comments.aspx?ArticleId=144&ArticleName=Poll+Update+%2811%2F4%29%3A+Finale

And here are all their 2012 polls:

http://cdn.yougov.com/r/1/2012%20Election%20results%20table%20YouGovLV%20ONLY.pdf

...

Generally, they were very good.
Logged
Senator Cris
Cris
Junior Chimp
*****
Posts: 5,613
Italy


Show only this user's posts in this thread
« Reply #19 on: August 21, 2014, 04:48:51 AM »

Dave said that he is OK with the YouGov polls in the database, but in the database there are only 3 YouGov polls.
Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #20 on: August 21, 2014, 05:06:54 AM »

Dave said that he is OK with the YouGov polls in the database, but in the database there are only 3 YouGov polls.

Enter them, if you like ... Wink

(I certainly won't enter 240 YouGov polls to the database, ca. 80 in each wave. Maybe I'll add the last wave in late October, yeah).
Logged
Senator Cris
Cris
Junior Chimp
*****
Posts: 5,613
Italy


Show only this user's posts in this thread
« Reply #21 on: August 21, 2014, 05:12:41 AM »

Dave said that he is OK with the YouGov polls in the database, but in the database there are only 3 YouGov polls.

Enter them, if you like ... Wink

(I certainly won't enter 240 YouGov polls to the database, ca. 80 in each wave. Maybe I'll add the last wave in late October, yeah).

I haven't permission. IMHO, It also goes well insert the late october polls Smiley
Logged
Tender Branson
Mark Warner 08
Atlas Institution
*****
Posts: 58,173
Austria


Political Matrix
E: -6.06, S: -4.84

Show only this user's posts in this thread
« Reply #22 on: August 21, 2014, 05:14:25 AM »

Dave said that he is OK with the YouGov polls in the database, but in the database there are only 3 YouGov polls.

Enter them, if you like ... Wink

(I certainly won't enter 240 YouGov polls to the database, ca. 80 in each wave. Maybe I'll add the last wave in late October, yeah).

I haven't permission. IMHO, It also goes well insert the late october polls Smiley

Just write Dave a PM or send him an email, by clicking on "email" on top of the page.

I'm sure you will get access.
Logged
Senator Cris
Cris
Junior Chimp
*****
Posts: 5,613
Italy


Show only this user's posts in this thread
« Reply #23 on: August 21, 2014, 05:27:14 AM »

Dave said that he is OK with the YouGov polls in the database, but in the database there are only 3 YouGov polls.

Enter them, if you like ... Wink

(I certainly won't enter 240 YouGov polls to the database, ca. 80 in each wave. Maybe I'll add the last wave in late October, yeah).

I haven't permission. IMHO, It also goes well insert the late october polls Smiley

Just write Dave a PM or send him an email, by clicking on "email" on top of the page.

I'm sure you will get access.
I wrote 2 messages for the Wiki page and I have also made a post in The Atlas section but I haven't received an answer.
Logged
Miles
MilesC56
Atlas Icon
*****
Posts: 19,324
United States


Show only this user's posts in this thread
« Reply #24 on: August 21, 2014, 09:56:14 AM »

^ Emailing Dave is the best way, in my experience.
Logged
Pages: [1]  
« previous next »
Jump to:  


Login with username, password and session length

Terms of Service - DMCA Agent and Policy - Privacy Policy and Cookies

Powered by SMF 1.1.21 | SMF © 2015, Simple Machines

Page created in 0.069 seconds with 14 queries.