Nate Silver: Dear Media, Stop Freaking Out About Donald Trump’s Polls

[Bull Moose Base]

http://www.realclearpolitics.com/articles/2015/11/24/early-state_polls_arent_predictive_--_yet__128828.html

Here's another analysis based on the last 3 open primaries- both parties in 08, GOP in 12- that suggests IA and NH polls become much more predictive after Thanksgiving. But as it points out, since the elections are a month later this time, maybe after Christmas would align more with those past examples.

As far as analogies to last time, I'd go roughly with Trump=Gingrich, Cruz=Santorum, Carson=Cain, Rubio=Romney? (though if Bush, Kasich or even Fiorina or Christie beat Rubio in New Hampshire, they could snatch that role away from him).

Anyway, I don't think it makes sense to assume Trump would lose a one-on-one primary race. The last PPP for example showed him beating Rubio one-on-one. It remains an open question how much of his own money Trump will spend, but if he chooses to, he could compete with Rubio (or other establishment candidate) in a way that Gingrich could not.

[Slander and/or Libel]

Silver is desperately grasping to retain his reputation. He's tried to position himself as something different than just a pundit, but he went out on the same limb as everyone else early in the process, saying that Trump had to collapse. He can't admit that he was just shooting from the hip, spouting his opinion, because that's off-brand. But he also can't sit back and not address how wrong he seems to be, so he has to come out with this, telling everybody to believe him and not their lying eyes.

[IceSpear]

Silver is desperately grasping to retain his reputation. He's tried to position himself as something different than just a pundit, but he went out on the same limb as everyone else early in the process, saying that Trump had to collapse. He can't admit that he was just shooting from the hip, spouting his opinion, because that's off-brand. But he also can't sit back and not address how wrong he seems to be, so he has to come out with this, telling everybody to believe him and not their lying eyes.

To be fair to Silver, he was vindicated in calling out the Biden/Sanders stuff as mostly media hype.

[Xing]

Silver is a very good statician, to his credit (at least when there are actual numbers to work with.) He's not as consistent with his analysis, though. Even in 2012, when he was "right on the mark," he insisted that Oregon was a swing state, and significantly underestimated a lot of Democrats in senate races. This year, he insisted that Walker was going to win the nomination, and then claimed that Walker lost because of "bad luck." (I guess Bronn Buckley was also just "unlucky" last year in Iowa.) Silver knows how to work with percentages very well, but he's not great at making predictions without actually crunching any real numbers.

[Asian Nazi]

I think Atlas is just intimidated by and jealous of Nate Silver.  It's the only reason I can think of why you people seem to have such an irrational hate boner for him.

[The Other Castro]

I think Atlas is just intimidated by and jealous of Nate Silver.  It's the only reason I can think of why you people seem to have such an irrational hate boner for him.

It is understandably frustrating to a politics-based forum that someone with a spotty record is able to regurgitate the currently held understandings of pundits, even if they are later shown to be incorrect, and have his word treated as gospel every time.

[Lief 🗽]

I think Atlas is just intimidated by and jealous of Nate Silver.  It's the only reason I can think of why you people seem to have such an irrational hate boner for him.

Certainly not intimidated; he's a fraud. Jealous is fair; this guy's analysis is mostly garbage at this point and if he posted on the Atlas Forum he'd probably be regarded as a dumb, consistently wrong hack. But instead he's paid seven figures by ESPN to average polls.

[ProudModerate2]

... even if they are later shown to be incorrect, and have his word treated as gospel every time.

From your quote above, it seems that you are describing Trump.
Trump has consistently been proven wrong, on various topics and exaggerations he tells, but yet his yes-men supporters treat Trump's word as gospel every time.

[Mister Mets]

Silver is desperately grasping to retain his reputation. He's tried to position himself as something different than just a pundit, but he went out on the same limb as everyone else early in the process, saying that Trump had to collapse. He can't admit that he was just shooting from the hip, spouting his opinion, because that's off-brand. But he also can't sit back and not address how wrong he seems to be, so he has to come out with this, telling everybody to believe him and not their lying eyes.

His argument over the last few months has been that Trump has a 50/50 shot of surviving various stages of scrutiny.

http://fivethirtyeight.com/features/donald-trumps-six-stages-of-doom/

So far Trump seems to have survived the first stage.

Silver is a very good statician, to his credit (at least when there are actual numbers to work with.) He's not as consistent with his analysis, though. Even in 2012, when he was "right on the mark," he insisted that Oregon was a swing state, and significantly underestimated a lot of Democrats in senate races. This year, he insisted that Walker was going to win the nomination, and then claimed that Walker lost because of "bad luck." (I guess Bronn Buckley was also just "unlucky" last year in Iowa.) Silver knows how to work with percentages very well, but he's not great at making predictions without actually crunching any real numbers.

He gave Walker a 26% chance of winning the nomination.

http://fivethirtyeight.com/datalab/listen-to-our-totally-subjective-presidential-odds-for-march-2015/

That's hardly insisting that the guy will win.

[Slander and/or Libel]

What value is Silver providing? If he's not more right than other pundits, what's the point of him?

[Mr. Morden]

He fails to distinguish betweens true "undecideds" and the fact that some people change their minds during a campaign. The fact that of the 30% that currently support Trump, a fair bunch of them might very well end up supporting someone else does not mean that they are "undecided". They have decided for now, but they might chance their mind.

I'm not sure I agree with that.  When the voting is still months away, I think the polls capture a large number of people who still view their choice of candidate in a highly hypothetical way.  Sure, they give the pollster a name on the phone when they're polled, but it's just a nominal "choice" at the moment, as opposed to a die hard partisan who's pretty solid about their selection.  I think that's the phenomenon that Silver's referring to.

This is why I think it's not exactly clear what early polls are measuring.  "If the election was held today..."  Well, it isn't held today.  And people don't necessarily think about it in the same terms the day before the vote as they do months before the vote.

[emailking]

What value is Silver providing? If he's not more right than other pundits, what's the point of him?

Not everything he posts amounts to a quantitative prediction of anything. Some of it is just discussion of what's going on, and what might happen, and how it could happen. It's usually synthesized in a way that's enjoyable to read, even if most of it isn't something you couldn't find somewhere else.

In terms of actual quantitative predictions, he can't be right or wrong because all he ever does is provide probabilities. If he predicts Walker has a 25% chance of winning the nomination, with others trailing behind, that's not a prediction that Walker will do well, but most people who read his column interpret it as such. You can take the probabilities however you want: seriously, for fun, or not at all. It doesn't really matter in the end since they can't be right or wrong. That's not a copout, it's just math.

He's not the only one making quantitative predictions using poll aggregation, but he was one of the first to do it rigorously.

Sometimes he does make an actual prediction on a result but labels it as subjective, so there's no attempt to be rigorous, he's just acquiescing to the fact that he's a human being with subjective ideas.

Ok, so his columns and language suggested that he thought Trump would probably flame out. So what? Almost everybody else did too. He never made a guarantee. He said we should be cautious about assuming his current lead means he will be the nominee, but he's also quick to point out there isn't enough data to say anything definitively. Primary modeling is a lot harder and less accurate than the general election. He's said this many times before.

I think a lot the hate stems from a general misunderstanding amongst the populace of how statistics works and how it should be interpreted.

[Iosif]

Silver should stop commentating. He knows nothing about politics, weighting polls aside.

Stick with the numbers.

[Attorney General, Senator-Elect, & Former PPT Dwarven Dragon]

What value is Silver providing? If he's not more right than other pundits, what's the point of him?

Not everything he posts amounts to a quantitative prediction of anything. Some of it is just discussion of what's going on, and what might happen, and how it could happen. It's usually synthesized in a way that's enjoyable to read, even if most of it isn't something you couldn't find somewhere else.

In terms of actual quantitative predictions, he can't be right or wrong because all he ever does is provide probabilities. If he predicts Walker has a 25% chance of winning the nomination, with others trailing behind, that's not a prediction that Walker will do well, but most people who read his column interpret it as such. You can take the probabilities however you want: seriously, for fun, or not at all. It doesn't really matter in the end since they can't be right or wrong. That's not a copout, it's just math.

He's not the only one making quantitative predictions using poll aggregation, but he was one of the first to do it rigorously.

Sometimes he does make an actual prediction on a result but labels it as subjective, so there's no attempt to be rigorous, he's just acquiescing to the fact that he's a human being with subjective ideas.

Actually, this whole year, his predictions for the nomination have been labeled as subjective odds. He's just trying to make educated guesses, he's not using his model to do it.

And the whole Nate Gospelism thing is that he is better than the media - he can tell when they're going to be wrong and goes against them when they're going to be. This year, all he's done is run with the failed media consensus - he said that Bush and Walker would be top contenders for the nomination, they aren't. He continues to say that Trump will fade shortly, even though all evidence points to Trump making it to at least Super Tuesday - Sure, these are things that Atlas ran with for a time, but we're just commoners - Nate, according to his fans, is supposed to be better than that, and he's not.

Finally, while what you say about his statistics is technically correct, that's not how Nate takes it, really. When he gave Brazil the best chance of winning the world cup and then they didn't, he outright called it an error. He didn't say "Hey, well this is supposed to happen 45% of the time, and we're living in the 45%.", he instead admitted that he, or rather, his model, had made an incorrect prediction and acknowledged it as such. In 2012, when he made his famous all 50 states correct prediction, he took in all the praise for it. He didn't say "Well, I can't really be right, I'm just throwing out percentages and we just happen to be in the right universe out of the 1000 I simulated", No, he took in all the praise and worship he could. The only time he's used the "It's just percentages" excuse is when it benefits him greatly - best example is when he gave Heitkamp a >90% chance of losing and she won, his claim was "Hey, this shows how good I am - these low probability things are supposed to happen sometimes - if they never did, I wouldn't be good." - it was a clear desire to win some support out of a prediction he knew was terribly wrong through pure spin.

[Mister Mets]

What value is Silver providing? If he's not more right than other pundits, what's the point of him?

Silver's main value is in pointing out the bullsh**t about standard political coverage where minor developments are exaggerated and fundamentals often ignored.

[emailking]

Actually, this whole year, his predictions for the nomination have been labeled as subjective odds. He's just trying to make educated guesses, he's not using his model to do it.

And the whole Nate Gospelism thing is that he is better than the media - he can tell when they're going to be wrong and goes against them when they're going to be. This year, all he's done is run with the failed media consensus - he said that Bush and Walker would be top contenders for the nomination, they aren't. He continues to say that Trump will fade shortly, even though all evidence points to Trump making it to at least Super Tuesday - Sure, these are things that Atlas ran with for a time, but we're just commoners - Nate, according to his fans, is supposed to be better than that, and he's not.

Has he ever built a model for the primary before though? I honestly can't remember. I know he's done one off regression analyses here and there, but I thought his heavy duty models were always for general elections. I think he's giving the subjective odds because what else can he do, close up shop until the nominees are decided? He knows that polling has very limited predictive value for the primary even this far out. I don't take his subjective odds as anything more than some guy's opinion, but I don't think he means for them to be taken as anything more than that either.

Finally, while what you say about his statistics is technically correct, that's not how Nate takes it, really. When he gave Brazil the best chance of winning the world cup and then they didn't, he outright called it an error. He didn't say "Hey, well this is supposed to happen 45% of the time, and we're living in the 45%.", he instead admitted that he, or rather, his model, had made an incorrect prediction and acknowledged it as such. In 2012, when he made his famous all 50 states correct prediction, he took in all the praise for it. He didn't say "Well, I can't really be right, I'm just throwing out percentages and we just happen to be in the right universe out of the 1000 I simulated", No, he took in all the praise and worship he could. The only time he's used the "It's just percentages" excuse is when it benefits him greatly - best example is when he gave Heitkamp a >90% chance of losing and she won, his claim was "Hey, this shows how good I am - these low probability things are supposed to happen sometimes - if they never did, I wouldn't be good." - it was a clear desire to win some support out of a prediction he knew was terribly wrong through pure spin.

I definitely think it would be better if he didn't do that, but I think it's probably just comes from having to deal with non-statisticians all the time. I remember columns that would come to something like "well it could be x, but y could also happen, but x could be more likely because...." and he would get excoriated in the comments about hedging his bets.

Also, for the Heitkamp thing, that is just right. If he never had (apparent) errors like that, his models would be flawed. But I get your point that he has inconsistent sometimes in describing how to interpret the results.

[Mr. Morden]

And the whole Nate Gospelism thing is that he is better than the media - he can tell when they're going to be wrong and goes against them when they're going to be. This year, all he's done is run with the failed media consensus - he said that Bush and Walker would be top contenders for the nomination, they aren't. He continues to say that Trump will fade shortly, even though all evidence points to Trump making it to at least Super Tuesday - Sure, these are things that Atlas ran with for a time, but we're just commoners - Nate, according to his fans, is supposed to be better than that, and he's not.

As I just said in this thread:

https://uselectionatlas.org/FORUM/index.php?topic=224006.msg4817473#msg4817473

show me where Nate is saying that Trump will fade shortly.  He says he doesn't think that he's going to win the nomination, but explicitly says he's not sure about what stage of the game that's going to happen in, conceding that he could hold on to most/all of his current support into the primaries, and lose the nomination.

As for Bush/Walker...I think you're reading that wrong.  Saying A, B, and C are the strongest contenders for the nomination isn't contradicted when A and B flame out and drop out / fail to win any primaries.  You're not saying that those will be the top three in the end.  You're saying that you think one of them will be the winner, but you don't know which one yet.  The second, third, and fourth place winners in delegates could be candidates X, Y, and Z who you give little chance of winning, and you wouldn't be wrong.  I mean, does anyone dispute that Phil Gramm had a better chance of winning the 1996 GOP nomination than Pat Buchanan?  That's not contradicted by the fact that Buchanan actually won some primaries, while Gramm dropped out after Iowa.  Buchanan never had a real chance at winning the nomination, while Gramm did.

[○∙◄☻¥tπ[╪AV┼cVê└]

And the whole Nate Gospelism thing is that he is better than the media - he can tell when they're going to be wrong and goes against them when they're going to be. This year, all he's done is run with the failed media consensus - he said that Bush and Walker would be top contenders for the nomination, they aren't. He continues to say that Trump will fade shortly, even though all evidence points to Trump making it to at least Super Tuesday - Sure, these are things that Atlas ran with for a time, but we're just commoners - Nate, according to his fans, is supposed to be better than that, and he's not.

As I just said in this thread:

https://uselectionatlas.org/FORUM/index.php?topic=224006.msg4817473#msg4817473

show me where Nate is saying that Trump will fade shortly.  He says he doesn't think that he's going to win the nomination, but explicitly says he's not sure about what stage of the game that's going to happen in, conceding that he could hold on to most/all of his current support into the primaries, and lose the nomination.

As for Bush/Walker...I think you're reading that wrong.  Saying A, B, and C are the strongest contenders for the nomination isn't contradicted when A and B flame out and drop out / fail to win any primaries.  You're not saying that those will be the top three in the end.  You're saying that you think one of them will be the winner, but you don't know which one yet.  The second, third, and fourth place winners in delegates could be candidates X, Y, and Z who you give little chance of winning, and you wouldn't be wrong.  I mean, does anyone dispute that Phil Gramm had a better chance of winning the 1996 GOP nomination than Pat Buchanan?  That's not contradicted by the fact that Buchanan actually won some primaries, while Gramm dropped out after Iowa.  Buchanan never had a real chance at winning the nomination, while Gramm did.

He's been really off. Jindal was supposed to be doing well in Iowa now. Webb was supposed to be the anti-Hillary. And Trump was supposed to immediately crash and burn. H

[mencken]

And the whole Nate Gospelism thing is that he is better than the media - he can tell when they're going to be wrong and goes against them when they're going to be. This year, all he's done is run with the failed media consensus - he said that Bush and Walker would be top contenders for the nomination, they aren't. He continues to say that Trump will fade shortly, even though all evidence points to Trump making it to at least Super Tuesday - Sure, these are things that Atlas ran with for a time, but we're just commoners - Nate, according to his fans, is supposed to be better than that, and he's not.

As I just said in this thread:

https://uselectionatlas.org/FORUM/index.php?topic=224006.msg4817473#msg4817473

show me where Nate is saying that Trump will fade shortly.  He says he doesn't think that he's going to win the nomination, but explicitly says he's not sure about what stage of the game that's going to happen in, conceding that he could hold on to most/all of his current support into the primaries, and lose the nomination.

As for Bush/Walker...I think you're reading that wrong.  Saying A, B, and C are the strongest contenders for the nomination isn't contradicted when A and B flame out and drop out / fail to win any primaries.  You're not saying that those will be the top three in the end.  You're saying that you think one of them will be the winner, but you don't know which one yet.  The second, third, and fourth place winners in delegates could be candidates X, Y, and Z who you give little chance of winning, and you wouldn't be wrong.  I mean, does anyone dispute that Phil Gramm had a better chance of winning the 1996 GOP nomination than Pat Buchanan?  That's not contradicted by the fact that Buchanan actually won some primaries, while Gramm dropped out after Iowa.  Buchanan never had a real chance at winning the nomination, while Gramm did.

What falsifiable metric do you propose to measure probabilistic predictions then? I initially proposed looking at whether they would have made money on the betting markets, which initially looked good for Silver & Co. until smilio pointed out that they likely would not have held those shares indefinitely.

They issued odds again in late September. If you had bought or sold one share of each candidate as Nate Silver recommended then, when you reevaluated your portfolio in early November, you would have made $22 off buying Rubio, $14 off shorting Bush to 25 (it seems to be implied that their reevaluation of Bush occurred after Bush crossed the 25 mark on October 21), $2 from shorting Huckabee, and $1 from buying Christie to 2. You also would have lost most of your investment in Fiorina (-$6), all of it on Walker (-$4), and lost some more money on Trump (-$6) and Kasich (-$1), for a net profit of $22. But again, from a frequentist perspective, his predictions were no better than a coin toss, and this analysis just gets lucky since he correctly realized that Bush was overpriced.

His most recent subjective odds did not substantively disagree with the betting markets except being more bullish on Rubio and Kasich and more bearish on (of course) Trump. He was 1 for 3 in those.

[Mr. Morden]

What falsifiable metric do you propose to measure probabilistic predictions then? I initially proposed looking at whether they would have made money on the betting markets, which initially looked good for Silver & Co. until smilio pointed out that they likely would not have held those shares indefinitely.

I'm suggesting that on this particular topic of Trump's chances at the nomination, it's not falsifiable at all yet because no one has voted yet.  Three or four months from now, it will be.  At that point, he'll be proven right or wrong on Trump.  But since all he's said (as far as I can tell) is that he thinks Trump isn't going to be the nominee, without suggesting either that there's going to be some kind of "collapse" of his existing support or that he knows the moment when another candidate will surpass him, it's premature for people to say he's already been proved wrong.  That's not to say that people shouldn't laugh at his prediction for being ridiculous based on currently available information, if they think it is ridiculous.  But I don't understand people saying that Silver predicted Trump's "collapse was imminent".  Did Silver say that?  Maybe he did, and I missed it.  If I did, where's the quote?

But let's broaden this out from Silver.  Others, like Jonathan Bernstein, are even more "the party decides" fundamentalists than Silver is.  As a group, I think all these people are saying is that the party will ultimately stop candidates it deems unacceptable (such as, presumably, Trump, Carson, Cruz).  That's all.  Not that none of those candidates will ever win any primaries, or get more than 30% of the vote in a state.  Just that they won't win the nomination.  But people here distort that, and suggest that "this person is doomed to lose the nomination" translates into "this person is going to see all of their support erode tomorrow, and will come in 12th place".  When, again, we have many examples of candidates in the past who stood no realistic chance of winning the nomination, yet won multiple primaries.

So sure, maybe they're right and maybe they're wrong.  But saying "the polls have already proved these people wrong" is incorrect.

[Gustaf]

In fact, there is a correlation where lost cause candidates are more likely to stay in the race long than serious contenders. See Huckabee, Santorum and especially Paul in the last 2 GOP primaries.

[Slander and/or Libel]

Silver's appeal has rested on his basing things on numbers. He's supposed to have done something that allows him to supersede punditry. I very much agree that there is a lot of misunderstanding of statistics out there, and that doesn't help. But if all he's saying is, "I can never be wrong because I never put 100% or 0% odds on anything," then there's not really much value to his brand of analysis. It really comes down to poll averaging, plus a little bit of incorporation of fundamentals.

I've personally wondered if somebody could go back and mark the point at which his predictions shift to correct (defined in this case as greater than 50% chance of what will ultimately occur occurring) and never shift back. If he "called" all 50 states correctly by this metric for, say, the last month running up to the election, that would be impressive. But if his model just follows the shifts in the polls, and winds up being an average of them, then what's the point?

[Mr. Morden]

Silver's appeal has rested on his basing things on numbers. He's supposed to have done something that allows him to supersede punditry. I very much agree that there is a lot of misunderstanding of statistics out there, and that doesn't help. But if all he's saying is, "I can never be wrong because I never put 100% or 0% odds on anything," then there's not really much value to his brand of analysis. It really comes down to poll averaging, plus a little bit of incorporation of fundamentals.

You can check how good someone is at assigning probabilities by looking at their overall track record.  Do events that they predict to be 80% likely happen 80% of the time?  Do the 90% events happen 90% of the time?, etc.  But you can't pick out a single one of those events and "prove" that the probability assigned was wrong after the fact, because there's no objective check for such a thing.  That doesn't mean that you can't laugh at the assigned probability for being too optimistic or pessimistic, based on the information at hand.  But those assessments are subjective.  That's not a dodge by Silver.  That's just a matter of definition.

I don't have a problem with people laughing at Silver's assessment of the race.  I'm just confused about people making things up about what Silver is predicting (suggesting that "this guy isn't going to win the nomination in the end" means "this guy's support is going to evaporate tomorrow"), and then saying that he's already been "proved wrong" on this because of polls with Trump at 30%.  And again, this isn't just about Silver, but all of the "party decides" people.  They are largely saying that Trump isn't going to win.  Not that he's not going to get a lot of votes.  Let's wait until the votes are counted before declaring them as having been "proved wrong".

[Slander and/or Libel]

Silver's appeal has rested on his basing things on numbers. He's supposed to have done something that allows him to supersede punditry. I very much agree that there is a lot of misunderstanding of statistics out there, and that doesn't help. But if all he's saying is, "I can never be wrong because I never put 100% or 0% odds on anything," then there's not really much value to his brand of analysis. It really comes down to poll averaging, plus a little bit of incorporation of fundamentals.

You can check how good someone is at assigning probabilities by looking at their overall track record.  Do events that they predict to be 80% likely happen 80% of the time?  Do the 90% events happen 90% of the time?, etc.  But you can't pick out a single one of those events and "prove" that the probability assigned was wrong after the fact, because there's no objective check for such a thing.  That doesn't mean that you can't laugh at the assigned probability for being too optimistic or pessimistic, based on the information at hand.  Those assessments are subjective, by definition.  That's not a dodge by Silver.  That's just a matter of definition.

I don't have a problem with people laughing at Silver's assessment of the race.  I'm just confused about people making things up about what Silver is predicting (suggesting that "this guy isn't going to win the nomination in the end" means "this guy's support is going to evaporate tomorrow"), and then saying that he's already been "proved wrong" on this because of polls with Trump at 30%.  And again, this isn't just about Silver, but all of the "party decides" people.  They are largely saying that Trump isn't going to win.  Not that he's not going to get a lot of votes.  Let's wait until the votes are counted before declaring them as having been "proved wrong".

I'm not at all saying it's a dodge. I understand the math, and I understand how probability works. I'm just wondering whether there's value in it to justify the reputation he got as some sort of wunderkind.

[Mr. Morden]

I'm not at all saying it's a dodge. I understand the math, and I understand how probability works. I'm just wondering whether there's value in it to justify the reputation he got as some sort of wunderkind.

Yeah, sorry, my comment in that regard was more a response to other comments in the thread than to you.