Nate Silver eats crow

[mencken]

Surprised nobody has posted this yet:

http://fivethirtyeight.com/features/how-i-acted-like-a-pundit-and-screwed-up-on-donald-trump/

You must be logged in to read this quote.

It makes an interesting read, regardless of your opinion of Nate Silver.

[Boston Bread]

Is he still defending using "endorsement points"? What a joke.

[Nym90]

He deserves credit for at least acknowledging his mistakes and making an effort to improve, as opposed to getting defensive and more stubborn as so many do in this situation.

[beaver2.0]

Nate Silver might not always be correct, and is possibly over-hyped, but he is still a pretty reliable pollster in a field that is anything but.

[Xing]

About time he admitted how wrong he was. Well, most people never admit that they're wrong, so at least this is a step in the right direction. If he ignores "conventional wisdom" in his predictions of the general election, he'll probably do better than he did in the primaries.

[mencken]

He deserves credit for at least acknowledging his mistakes and making an effort to improve, as opposed to getting defensive and more stubborn as so many do in this situation.

I concur entirely.

[Hatman 🍁]

The New York elite bubble will do that to you

[Mr. Morden]

Is he still defending using "endorsement points"? What a joke.

What's the problem with endorsement points?

[Penelope]

Is he still defending using "endorsement points"? What a joke.

What's the problem with endorsement points?

The idea that 'endorsement points' were at all influential on the results of the primaries has surely been discredited by now.

[LLR]

Nate Silver might not always be correct, and is possibly over-hyped, but he is still a pretty reliable pollster in a field that is anything but.

This ^^

The people at 538 also put out some of the best interactive graphics, etc. of any politics sites, so for that, they're pretty great. They do not deserve the s*** they've been getting

[publicunofficial]

Good point in that his denial of Trump wasn't based of any sort of model, but rather pure steadfast disbelief.

[Mr. Morden]

Is he still defending using "endorsement points"? What a joke.

What's the problem with endorsement points?

The idea that 'endorsement points' were at all influential on the results of the primaries has surely been discredited by now.

No, that’s not the point.  The endorsement points themselves don’t influence the results of primaries.  Endorsement points are just a proxy for “party support”, translated into a quantity that’s measurable.  The point is that, when you look back at the recent history of presidential primaries, if you look at the polling for a given state say, three or four weeks out from primary day, what combination of variables gives you the most accurate prediction for what the primary result it going to be?  Obviously, public opinion polling is by far the most important factor, but it’s not the only one.  As 538 pointed out, in past election cycles, other variables, like the number of endorsements, also had predictive power.  That is, candidates with more endorsements tended to outperform their polls.

So 538 came out with two different models.  One which was the vanilla “polls only” version, and another which incorporated additional variables, like endorsements.  They even acknowledged “Hey, some people are saying `this time is different’, and party support doesn’t matter anymore.  That might be true.  And if you believe that, go with the polls only model.”  Obviously, Trump was nominated with very few endorsements, so the polls only model worked better on the Republican side at least.  That doesn’t mean that project to also create a model with additional variables was misguided in principle.

[Beet]

The problem with endorsement points is twofold. One, their value seems to be justified primarily in the last three or four elections. No academic will tell you that a valid sample size can be three or four, or even seventeen, as Silver himself acknowledges.

Secondly, what is the theoretical mechanism of transmission? In the Washington caucus, Hillary had the endorsement of both party Senators, the Governor, the mayor of Seattle, and dozens of state legislators. If these people had some magical powers of mind control, it wouldn't have been a 48 point loss. In 2008, Obama lost Massachusetts by 16 points even though both Kerry and the Kennedy family rallied around him. So if endorsements don't work at the state level, why would it work at the national level? It's not explained now it's supposed to work.

[Mr. Morden]

Secondly, what is the theoretical mechanism of transmission? In the Washington caucus, Hillary had the endorsement of both party Senators, the Governor, the mayor of Seattle, and dozens of state legislators. If these people had some magical powers of mind control, it wouldn't have been a 48 point loss. In 2008, Obama lost Massachusetts by 16 points even though both Kerry and the Kennedy family rallied around him. So if endorsements don't work at the state level, why would it work at the national level? It's not explained now it's supposed to work.

To clarify, 538 never did any kind of national forecast, whether with endorsement points or not.  Unless you're counting the "subjective odds", where they just act as pundits.  But that never had the pretense of being based on a quantitative model.  The polling and endorsement points, etc. were always just for the state-level forecasts (since there were no national forecasts).

There is no "magical power of mind control".  The endorsements themselves might not even be what's driving the effect.  Endorsement points are just a proxy for "support from party actors", where party actors can be defined broadly.  The point is simply that candidates without support from party actors are more likely to fizzle out than those who do have party support.  This doesn't happen every single time, obviously.  You can name many counterexamples.  But looking at the existing dataset of presidential primaries from the past few decades, there is a correlation between support from party actors and primary day performance, when you hold polling #s equal.  You don't have to know the exact mechanism to observe that.

(Heck, the explanation could simply be that party actors are quicker than voters to figure out that a candidate is an unelectable buffoon, and between one month out from primary day and primary day itself, more voters figure this out as well, and that explains why the candidate's poll #s tend to drop.)

So endorsement points are just a way to quantify that.  Maybe there was a better way to do it than the specific method used by 538, but some here seem to think that accounting for such things is misguided in principle, and I don't understand why.

[Boston Bread]

The problem was using endorsements to predict election results when polls did not back it up at all. The polls plus was worse than polls only overall, and though they came up with a demographic model (that would of allowed them to predict states with few polls decently well) they never treated seriously, never incorporating it into their model. They just avoided forecasting a third of the contests because there were no polls, even though they had same kind of demographic model that performed well for Nate in 08.

[I spent the winter writing songs about getting better]

He has a bit less of an excuse than most analysts because had he stuck to just what he's known for and averaged the polls, he would've seen Trump winning.

[Lurker]

So Silver's mistake was, in his own words, that he acted like a pundit. Apparently he isn't really one?
 
"If you take the life lie from an average man, you take away his happiness as well"


But I thought that this was a good read. Other than the self-criticism (impressive, even if a bit limited), I enjoyed the discussion of "fundamentals"-based electoral models, which I've always been very skeptical of in general.

[Mr. Morden]

The problem was using endorsements to predict election results when polls did not back it up at all. The polls plus was worse than polls only overall,

I’m not sure what you mean about “polls did not back it up at all”.  Polls are one observable.  Endorsements are another.  In past presidential primary races, they both had predictive power.  Sure, the polls have much more predictive power, but for two candidates who are equal in the polls, the candidate with more endorsements was more likely to do well on election day.  That was the point of “polls plus”.  To incorporate additional variables that aren’t captured in polling.

Yes, the polls plus model did worse than polls only this time, but would have done better in previous races.  You can’t know for sure which model is going to work better in advance.  That said, as Nate said in his mea culpa, the use of fundamentals in “polls plus” was probably an example of “overfitting”.  So, again, we can knock him for that, but I don’t see a problem per se with incorporating additional information besides polls alone.

You must be logged in to read this quote.

OK, there are two different things here which we shouldn’t confuse.  You can make a model that you develop based on previous years’ presidential primaries.  You look at how predictive a poll or any other variable is X number of days before primary day.  Then you just wait for the data points to roll in over the course of the campaign.  Your model doesn’t actually change over the course of the campaign.  Just the data inputs.  Both “polls only” and “polls plus” worked this way.

That’s one way to do it.  But another way is to wait for election results from the early primary states, then build a demographic model for the rest of the primary campaign based on how each candidate does with each demographic.  That’s more of a dynamic model, where you need election returns from this election in order to even get started.  NYT’s Upshot did that, and I had a thread on it.  538 also had a quickie version of this, which they used for their “delegate targets”.

But the problem with this approach is that you’re assuming that there is no movement in candidate support over the course of the campaign.  You’re assuming that all of the differences from state to state are demographic differences only, rather than an indication that the race is moving.  Maybe that works for the Democratic race this year, but it definitely doesn’t work for the Republican race.  I mean, it can give you crude benchmarks, but it’s complicated by the fact that candidates kept dropping out every couple of weeks, and you also saw some real movement, like after New York, when Trump started blowing through demographic targets in subsequent states.  It’s a fun thing to look at, but it would be tough to use that in a serious way when the number of candidates in the race changes so much over the course of the campaign.

[I spent the winter writing songs about getting better]

The Endorsement Points thing was stupidly set up for even what it was alleged to do. For example Kasich had a lead over Trump because he got ten points from the Governor of Alabama, a guy facing possible impeachment the party wants to resign. Meanwhile Paul Ryan is just counted as a Representative Worth only one point, aka 1/10 of that guy. The 1/5/10 scoring made no sense. It also didn't count Mitt Romney or Jeb Bush once he dropped out at all for example

[IceSpear]

In all fairness to Nate, he did nail the Democratic side at least. While every other media pundit was bloviating about how Elizabeth Warren, Martin O'Malley, Joe Biden, and Bernie Sanders were going to clean Hillary's clock and cause a #2008redux, he never bought the hype.

[Nichlemn]

Too many people see Nate Silver as if he is some kind of aspiring clairvoyant, so he gets too much credit for being right and too much for being wrong.

[ag]

He deserves credit for at least acknowledging his mistakes and making an effort to improve, as opposed to getting defensive and more stubborn as so many do in this situation.

Exactly. Eating crow in public is the honest and honorable thing to do.