The Poll Aggregators Got Wrecked This Cycle

[Seriously?]

Huffington Post: 98.1% Clinton chance of winning to 1.6% Trump
NYT/Upshot: 85% Clinton chance of winning to 15% Trump
538 Blog: 71.4% Clinton chance of winning to 28.6% Trump

Should much stock be put into these forecasts or polling for that matter anymore?

And what caused the polls to be off so much? People not discovered by pollsters or a blind adherence/assumption of a certain demographic/regional reweigh that didn't pan out in reality?

[○∙◄☻¥tπ[╪AV┼cVê└]

The Democrats underperformed polls 2 years ago, too. Maybe after a few more upset losses they'll take a look into what they're doing wrong.

[Hammy]

If any good has come out of Trump's election it's forcing the polling industry to finally take a good hard look at how far off their methodology has been for some time.

[Desroko]

The Democrats underperformed polls 2 years ago, too. Maybe after a few more upset losses they'll take a look into what they're doing wrong.

Over and underperformance comes and goes. This is more about how polling can miss.

National polling was pretty good - depending on the final margin, just a point or two of error. The battleground state polling was pretty bad though, averaging around 4 points.

This is the opposite pattern from 2012, btw, when state polling was pretty good and the national polling missed by nearly four points.

[Seriously?]

The Democrats underperformed polls 2 years ago, too. Maybe after a few more upset losses they'll take a look into what they're doing wrong.

Over and underperformance comes and goes. This is more about how polling can miss.

National polling was pretty good - depending on the final margin, just a point or two of error. The battleground state polling was pretty bad though, averaging around 4 points.

This is the opposite pattern from 2012, btw, when state polling was pretty good and the national polling missed by nearly four points.

The NYT estimate has the final margin at about Clinton +1.2 or so, although it could be a little closer than that. The average polling estimate of the herded polls was probably somewhere between Clinton +3 and Clinton +4, so there will be about a 2-3 point miss on most national polls.

I think some of this though has to be incorrect assumptions about turnout. Racial breakdown ended up 70/12/11/4/3. I think a lot of this polling had it at 68 or so with the white vote and the wilted pickle had it at 62% the last time I checked.

The exit poll suggested a D+4 electorate 37/33/31. I think the underlying national exit poll, unlike the state polls in FL and NC, is generally accurate.

[Desroko]

The Democrats underperformed polls 2 years ago, too. Maybe after a few more upset losses they'll take a look into what they're doing wrong.

Over and underperformance comes and goes. This is more about how polling can miss.

National polling was pretty good - depending on the final margin, just a point or two of error. The battleground state polling was pretty bad though, averaging around 4 points.

This is the opposite pattern from 2012, btw, when state polling was pretty good and the national polling missed by nearly four points.

The NYT estimate has the final margin at about Clinton +1.2 or so, although it could be a little closer than that. The average polling estimate of the herded polls was probably somewhere between Clinton +3 and Clinton +4, so there will be about a 2-3 point miss on most national polls.

I think some of this though has to be incorrect assumptions about turnout. Racial breakdown ended up 70/12/11/4/3. I think a lot of this polling had it at 68 or so with the white vote and the wilted pickle had it at 62% the last time I checked.

The exit poll suggested a D+4 electorate 37/33/31. I think the underlying national exit poll, unlike the state polls in FL and NC, is generally accurate.

Exit polls generally overstate younger (and thus nonwhite) voters, so I'm tentatively guessing the white non-Hispanic vote in the final ACS data will come in at about 72%. For 2012, the ACS estimates it at 73.7%. I'll have to look at some of the final natl/state polling and compare their demos, but in the rust belt, where the white population and polling error was high, that's a plausible explanation for why the natl and state diverged so much.

[Erich Maria Remarque]

Yeah, and the fun fact is that when some polls showed >72% white or Trump doing better than Romney among Black and Hispanic they got very big anount of critique, both from Atlas and Twitter/media pundits. "No way", said them. "Outlier/lack of methodology/no spanish interview, muh latino decision"

[Desroko]

Yeah, and the fun fact is that when some polls showed >72% white or Trump doing better than Romney among Black and Hispanic they got very big anount of critique, both from Atlas and Twitter/media pundits. "No way", said them. "Outlier/lack of methodology/no spanish interview, muh latino decision"

It will take a while for county-level data to be digested, but I doubt that it will show any significant increase for Trump v Romney, at least nationally. Very few of Edison's interviewers are Spanish-fluent, even in Hispanic-majority areas. Hispanics who speak English as a second-language or poorly are far more Democratic.

I'll say this as often as I have to - don't rely on exit polls.  

[Seriously?]

Yeah, and the fun fact is that when some polls showed >72% white or Trump doing better than Romney among Black and Hispanic they got very big anount of critique, both from Atlas and Twitter/media pundits. "No way", said them. "Outlier/lack of methodology/no spanish interview, muh latino decision"

To me it was an absolute no-brainer that the black vote would regress to the norm in a post-Obama election.

The Hispanic block didn't appear as monolithic to me as those that were critical on here.

At the end of the day, your skin color doesn't matter that much if you are experiencing economic hardship. Trump took 33% of the Hispanic male vote. 29% overall.

[Panda Express]

Yeah, the polls were a joke this election and  I'm done taking them seriously. I'm also not going to believe early vote data anymore.

[SirMuxALot]

Silver deserves some credit here.

Despite all the inputs that he had no control over (public polls), his model correctly identified a significant chance for systematic error that few others correctly quantified.

Stated another way, his model thought there was a 1-in-4 chance the polls were off enough to tip the winning call.  That's once every 16 years.  Not exactly a super-rare occurrence.

Silver's model was at the mercy of the polls, so it's way too much to ask for his model to independently declare *all the polls* wrong and somehow have Trump > 51% chance to win.

Given the state of the race (close, within margin of error) and the one-sidedness of the polling error, Nate's model really did about the best it possibly could.

[cinyc]

Exit polls generally overstate younger (and thus nonwhite) voters, so I'm tentatively guessing the white non-Hispanic vote in the final ACS data will come in at about 72%. For 2012, the ACS estimates it at 73.7%. I'll have to look at some of the final natl/state polling and compare their demos, but in the rust belt, where the white population and polling error was high, that's a plausible explanation for why the natl and state diverged so much.

I suspect a lot of the pollsters modeled their electorates based on the 2012 ACS estimates.  I know we did for the Atlas Google Survey polls.  The real issue might be that the 2012 electorate didn't show up to the polls, and turnout was more like 2014.  African-American turnout appears to be down by enough in places like Milwaukee, Detroit and Philadelphia to, along with higher working class white turnout in small town and rural areas, narrowly give those states to Trump.  The swing toward Clinton in the suburbs was not enough to counterbalance the other two effects. 

Trump also appears to have done slightly better among minorities than Romney, which virtually no pollster picked up (and any pollster that did was laughed at on this forum, anyway).  In particular, the Hispanic-only pollsters like Latino (D)ecisions, which, according to Wikileaks, was hired to do Clinton's own Hispanic polling and had a huge conflict of interest when producing public polls, were terrible.

[Desroko]

Exit polls generally overstate younger (and thus nonwhite) voters, so I'm tentatively guessing the white non-Hispanic vote in the final ACS data will come in at about 72%. For 2012, the ACS estimates it at 73.7%. I'll have to look at some of the final natl/state polling and compare their demos, but in the rust belt, where the white population and polling error was high, that's a plausible explanation for why the natl and state diverged so much.

I suspect a lot of the pollsters modeled their electorates based on the 2012 ACS estimates.  I know we did for the Atlas Google Survey polls.  The real issue might be that the 2012 electorate didn't show up to the polls, and turnout was more like 2014.  African-American turnout appears to be down by enough in places like Milwaukee, Detroit and Philadelphia to, along with higher working class white turnout in small town and rural areas, narrowly give those states to Trump.  The swing toward Clinton in the suburbs was not enough to counterbalance the other two effects. 

Trump also appears to have done slightly better among minorities than Romney, which virtually no pollster picked up (and any pollster that did was laughed at on this forum, anyway).  In particular, the Hispanic-only pollsters like Latino (D)ecisions, which, according to Wikileaks, was hired to do Clinton's own Hispanic polling and had a huge conflict of interest when producing public polls, were terrible.

I think Trump doing as well as a "normal" Republican among black voters with Obama off the ballot is very likely.

I'm suspicious of the Hispanic margins, as mentioned above, but generally the exits overstate Hispanic turnout slightly, and understate their Democratic margins slightly, so it would be a wash. There are probably states where Trump improved (FL?), and some where Clinton improved (CA?)

[Sbane]

Exit polls generally overstate younger (and thus nonwhite) voters, so I'm tentatively guessing the white non-Hispanic vote in the final ACS data will come in at about 72%. For 2012, the ACS estimates it at 73.7%. I'll have to look at some of the final natl/state polling and compare their demos, but in the rust belt, where the white population and polling error was high, that's a plausible explanation for why the natl and state diverged so much.

I suspect a lot of the pollsters modeled their electorates based on the 2012 ACS estimates.  I know we did for the Atlas Google Survey polls.  The real issue might be that the 2012 electorate didn't show up to the polls, and turnout was more like 2014.  African-American turnout appears to be down by enough in places like Milwaukee, Detroit and Philadelphia to, along with higher working class white turnout in small town and rural areas, narrowly give those states to Trump.  The swing toward Clinton in the suburbs was not enough to counterbalance the other two effects. 

Trump also appears to have done slightly better among minorities than Romney, which virtually no pollster picked up (and any pollster that did was laughed at on this forum, anyway).  In particular, the Hispanic-only pollsters like Latino (D)ecisions, which, according to Wikileaks, was hired to do Clinton's own Hispanic polling and had a huge conflict of interest when producing public polls, were terrible.

I doubt Trump did better among Hispanics vs Romney. Although it should be noted Romney likely did better among Hispanics than the exit poll showed. He also only won the white vote by 17%, not 20. There was likely a 3-4 point swing in the white vote this year vs 2012, and a 7-8 point swing among Blacks along with lower turnout. Hispanics probably did swing to Clinton by at least 2-3 points and turnout was definitely higher as well. Look at results from Texas, Arizona and California.

[Erich Maria Remarque]

Yeah, and the fun fact is that when some polls showed >72% white or Trump doing better than Romney among Black and Hispanic they got very big anount of critique, both from Atlas and Twitter/media pundits. "No way", said them. "Outlier/lack of methodology/no spanish interview, muh latino decision"

It will take a while for county-level data to be digested, but I doubt that it will show any significant increase for Trump v Romney, at least nationally. Very few of Edison's interviewers are Spanish-fluent, even in Hispanic-majority areas. Hispanics who speak English as a second-language or poorly are far more Democratic.

I'll say this as often as I have to - don't rely on exit polls.  

Nate Cohn also mentioned that a quick look at early data at least indicate that Trump didn't do significantly worse than Romney. And mostly polls that didn't show ridiculously low numbers for Trump, were criticised.

[Seriously?]

Yeah, the polls were a joke this election and  I'm done taking them seriously. I'm also not going to believe early vote data anymore.

What was fascinating to me last night was the Early Vote vs. Day Of vote in NC.  It looked like the Dems did such a good job with early voting, come election day, there was no one left to vote for Hillary. Trump cleaned up and took the state.

You can look at the county-level data here: http://er.ncsbe.gov/contest_details.html?election_dt=11/08/2016&county_id=0&contest_id=1001

[Desroko]

Yeah, and the fun fact is that when some polls showed >72% white or Trump doing better than Romney among Black and Hispanic they got very big anount of critique, both from Atlas and Twitter/media pundits. "No way", said them. "Outlier/lack of methodology/no spanish interview, muh latino decision"

It will take a while for county-level data to be digested, but I doubt that it will show any significant increase for Trump v Romney, at least nationally. Very few of Edison's interviewers are Spanish-fluent, even in Hispanic-majority areas. Hispanics who speak English as a second-language or poorly are far more Democratic.

I'll say this as often as I have to - don't rely on exit polls.  

Nate Cohn also mentioned that a quick look at early data at least indicate that Trump didn't do significantly worse than Romney. And mostly polls that didn't show ridiculously low numbers for Trump, were criticised.

I saw that too, and I hope he does a deep dive into this stuff over the next few weeks.

[Erich Maria Remarque]

Exit polls generally overstate younger (and thus nonwhite) voters, so I'm tentatively guessing the white non-Hispanic vote in the final ACS data will come in at about 72%. For 2012, the ACS estimates it at 73.7%. I'll have to look at some of the final natl/state polling and compare their demos, but in the rust belt, where the white population and polling error was high, that's a plausible explanation for why the natl and state diverged so much.

I suspect a lot of the pollsters modeled their electorates based on the 2012 ACS estimates.  I know we did for the Atlas Google Survey polls.  The real issue might be that the 2012 electorate didn't show up to the polls, and turnout was more like 2014.  African-American turnout appears to be down by enough in places like Milwaukee, Detroit and Philadelphia to, along with higher working class white turnout in small town and rural areas, narrowly give those states to Trump.  The swing toward Clinton in the suburbs was not enough to counterbalance the other two effects. 

Trump also appears to have done slightly better among minorities than Romney, which virtually no pollster picked up (and any pollster that did was laughed at on this forum, anyway).  In particular, the Hispanic-only pollsters like Latino (D)ecisions, which, according to Wikileaks, was hired to do Clinton's own Hispanic polling and had a huge conflict of interest when producing public polls, were terrible.

I think that almost every pollster reweighed data to census for RV, and use voter intention/ historic data if there is for LV. So they might simply "miss" some Trump's voters.

[cinyc]

Exit polls generally overstate younger (and thus nonwhite) voters, so I'm tentatively guessing the white non-Hispanic vote in the final ACS data will come in at about 72%. For 2012, the ACS estimates it at 73.7%. I'll have to look at some of the final natl/state polling and compare their demos, but in the rust belt, where the white population and polling error was high, that's a plausible explanation for why the natl and state diverged so much.

I suspect a lot of the pollsters modeled their electorates based on the 2012 ACS estimates.  I know we did for the Atlas Google Survey polls.  The real issue might be that the 2012 electorate didn't show up to the polls, and turnout was more like 2014.  African-American turnout appears to be down by enough in places like Milwaukee, Detroit and Philadelphia to, along with higher working class white turnout in small town and rural areas, narrowly give those states to Trump.  The swing toward Clinton in the suburbs was not enough to counterbalance the other two effects. 

Trump also appears to have done slightly better among minorities than Romney, which virtually no pollster picked up (and any pollster that did was laughed at on this forum, anyway).  In particular, the Hispanic-only pollsters like Latino (D)ecisions, which, according to Wikileaks, was hired to do Clinton's own Hispanic polling and had a huge conflict of interest when producing public polls, were terrible.

I doubt Trump did better among Hispanics vs Romney. Although it should be noted Romney likely did better among Hispanics than the exit poll showed. He also only won the white vote by 17%, not 20. There was likely a 3-4 point swing in the white vote this year vs 2012, and a 7-8 point swing among Blacks along with lower turnout. Hispanics probably did swing to Clinton by at least 2-3 points and turnout was definitely higher as well. Look at results from Texas, Arizona and California.

Clinton did worse than Obama 2012 among Hispanics.  This is clear from both the exit polls (Clinton's 65% versus Obama's 71%), and looking at heavily Hispanic counties in the Rio Grande Valley, like
Hidalgo County, Texas (McAllen).  While Trump and Romney both received about 28% of the vote there, Obama received 70.3% to Clinton's 68.6%.  Similarly, in Webb County (Laredo), Obama got 76.4% to Clinton's 74.3%.  Some of this was due to a higher vote for third parties in 2016.  But what's clear to me is that the Hispanic vote broke nothing like Latino Decision's alleged Clinton 76, Trump 14 split.  They lowballed the Hispanic vote.

[Meclazine for Israel]

Silver deserves some credit here.

Despite all the inputs that he had no control over (public polls), his model correctly identified a significant chance for systematic error that few others correctly quantified.

Stated another way, his model thought there was a 1-in-4 chance the polls were off enough to tip the winning call.  That's once every 16 years.  Not exactly a super-rare occurrence.

Silver's model was at the mercy of the polls, so it's way too much to ask for his model to independently declare *all the polls* wrong and somehow have Trump > 51% chance to win.

Given the state of the race (close, within margin of error) and the one-sidedness of the polling error, Nate's model really did about the best it possibly could.

Yeah,

Silver had Trump at 36% and rising with three days to go.

That is a significant chance when entering an election.

[jaichind]

I would argue that 538 is quite credible. The national poll averages were around Clinton +3 so it is reasonable to put her chances of winning at above 50%.  But 538 as along in modeling for potential error  at the national vote level as well as how that error might translate into state level results.  In that sense its projection of Trump chances around the low 30s were very accurate in retrospect.

[rafta_rafta]

In 2012, national polls were too close to call but states polls were bang on, This time around the national polls were closer to the mark but states poll were totally wrong.

Turnout didn't increase, but the demographics totally changed.

[AGA]

Not all the polls were off. There were plenty of polls showing tight races, sometimes leads, in Wisconsin,  Michigan, Pennsylvania, and New Hampshire, but people just laughed them off as if they were Republican hackery.

[jaichind]

Not all the polls were off. There were plenty of polls showing tight races, sometimes leads, in Wisconsin,  Michigan, Pennsylvania, and New Hampshire, but people just laughed them off as if they were Republican hackery.

Yeah.  Trafalgar polls which is rumored to be the internal Trump pollster and mocked days ago now looks like genious.

[muon2]

Silver deserves some credit here.

Despite all the inputs that he had no control over (public polls), his model correctly identified a significant chance for systematic error that few others correctly quantified.

Stated another way, his model thought there was a 1-in-4 chance the polls were off enough to tip the winning call.  That's once every 16 years.  Not exactly a super-rare occurrence.

Silver's model was at the mercy of the polls, so it's way too much to ask for his model to independently declare *all the polls* wrong and somehow have Trump > 51% chance to win.

Given the state of the race (close, within margin of error) and the one-sidedness of the polling error, Nate's model really did about the best it possibly could.

This is an important point. A prediction that gives a probability should go the wrong way sometimes. What matters is how often does it actually go against the odds. For example a bet against a 7 on two dice is 83% likely, but no one would think the dice were rigged if it actually came up 7. However if it came up 7 on 33 tries out of 100 one would be right to ask questions about the dice.

In this case if a prediction claimed 90% likely for Clinton then we should see are they getting it wrong about 1 time out of 10. For 538 would we expect this level of polling to be wrong more like 1 time out of 4. Unfortunately we don't have a lot of data for this type of prediction for presidential races. However if I consider races where the media miscalled the race based on polls as part of the data I can include 1948 and 2000 as well as 2016. That's 3 out of 18 races since WW2. So a pollster claiming an 80% chance for Hillary this year would be making a fair statement, but someone saying 95% likely for Hillary is being overconfident.