Pretty Damning Indictment of Nate Silver

[emailking]

I think the 538 model has already had an embarrassing run in the last couple of months.  Even if Obama wins a narrow re-election, the fact that the model had him at 85% just a few weeks ago demonstrates just how deeply flawed it is.

No it doesn't. Scenarios in which Romney comes back a bit (and even ones in which he gets ahead for awhile) but Obama still wins are included in the 85%.

The only real metric here is what Mr. Morden said. Now maybe nobody's done that analysis or there isn't enough data to make a conclusion. Ok, fine. But pointing out that the model once had Obama ahead by a lot and now Romney is closer, that's just baseless criticism. The reason it was at 85% instead of 100% was due to the evaluation that Romney could still come back.

[Wonkish1]

I think the 538 model has already had an embarrassing run in the last couple of months.  Even if Obama wins a narrow re-election, the fact that the model had him at 85% just a few weeks ago demonstrates just how deeply flawed it is.

No it doesn't. Scenarios in which Romney comes back a bit (and even ones in which he gets ahead for awhile) but Obama still wins are included in the 85%.

The only real metric here is what Mr. Morden said. Now maybe nobody's done that analysis or there isn't enough data to make a conclusion. Ok, fine. But pointing out that the model once had Obama ahead by a lot and now Romney is closer, that's just baseless criticism. The reason it was at 85% instead of 100% was due to the evaluation that Romney could still come back.

Hmm, probably should think about that once because there is a very, very high overlap to what SirMux just pointed out and what Morden suggested we judge Nate by.

Basically, with a sharp drop in probabilities like we've seen Mr. Morden's method of measurement would in aggregate come to the same conclusion as what SirMux just did. The reason being is that Mr. Morden's system basically would calculate mistakes in probabilities in advance. Large swings over a short period of time in aggregate would cause either the pre swing probability or the post swing probability to be way off or both.


You just poo pooed another posters method of looking at results and then suggested that a very, very similar one is the best way to do it and you didn't even realize it. Oops!

[Bacon King]

I think the 538 model has already had an embarrassing run in the last couple of months.  Even if Obama wins a narrow re-election, the fact that the model had him at 85% just a few weeks ago demonstrates just how deeply flawed it is. 

Obama's Election Day odds were artificially inflated throughout September mostly because of Nate Silver's attempt to control for Convention bounces, so the polling spikes wouldn't affect the model's results. Unfortunately, it didn't really work as intended.

If I recall correctly, Silver analyzed polling from past cycles and concluded that the average convention bounce topped out at 3% after however many days, slowly fading away until it the bounce fully ended however many weeks later. The problem was that Romney's national polling only improved by about one percentage point after the RNC, while Obama's immediate bounce was something like four points. So, when the model compared those numbers to the expectations based on "average" post-convention polls, it essentially behaved like Obama actually gained a margin of two points over Romney after the RNC, with his own convention boost only adding to that.

I might be a bit fuzzy on the details, but that's basically why Obama's odds were so high before they snapped downwards after the first debate. It's definitely a good idea, on paper, to control for post-convention bounces when you're using polls to predict election results- but there was probably a better way to implement something like that (though I have no idea how).

[emailking]

Hmm, probably should think about that once because there is a very, very high overlap to what SirMux just pointed out and what Morden suggested we judge Nate by.

Basically, with a sharp drop in probabilities like we've seen Mr. Morden's method of measurement would in aggregate come to the same conclusion as what SirMux just did.

Mr. Morden pointed out that to evaluate the model you have to check how accurate the probabilities are. This involves looking at many predictions from the model and comparing them with results. In particular, it would mean ensuring that 15% of the time a candiate is giving an 85% of winning, s/he in fact loses.

SirMux thinks the model has a problem because we can point to a single instance in which a candiate was giving an 85% of winning, and maybe he doesn't win. That's not a problem with the model at all. In fact, whether Obama wins or loses, that on its own says nothing whatsoever about the accuracy of the 85% figure.

They're talking about 2 completely different things.

The reason being is that Mr. Morden's system basically would calculate mistakes in probabilities in advance. Large swings over a short period of time in aggregate would cause either the pre swing probability or the post swing probability to be way off or both.


You just poo pooed another posters method of looking at results and then suggested that a very, very similar one is the best way to do it and you didn't even realize it. Oops!

I don't think you understand in the slightest what Mr. Morden suggested.

[SirMuxALot]

The reason it was at 85% instead of 100% was due to the evaluation that Romney could still come back.

I'm sorry, but then it is a worthless model.  By this definition, it could have been at 99.99% for Obama, and it's "still not wrong" since it allowed for a Romney comeback.  C'mon...that 85% was garbage.  If it wasn't garbage, then it would still be close to that.  I thought the 85% then was garbage.  And I think his current 72% is equally garbage.

I know how Nate would defend this.  He would say, well, if the election were held on that day, then there was 85% chance of Obama winning.  But that is also worthless.  Election day is not some unknown date.  I can create a model that says the Dow or the S&P will do such and such on this coming Saturday and Sunday...but of course...you can't prove my model wrong because the market wasn't open those days.

The 538 model has a one Presidential election track record.  I know people like to point to 49/50 states, but how many of those were tough calls?  Three states?  IN, MO, NC?  Maybe MT?  And IN was a miss.  2008 was not a difficult election to call.  This one is.

[SPQR]

The reason it was at 85% instead of 100% was due to the evaluation that Romney could still come back.

I'm sorry, but then it is a worthless model.  By this definition, it could have been at 99.99% for Obama, and it's "still not wrong" since it allowed for a Romney comeback.  C'mon...that 85% was garbage.  If it wasn't garbage, then it would still be close to that.  I thought the 85% then was garbage.  And I think his current 72% is equally garbage.

I know how Nate would defend this.  He would say, well, if the election were held on that day, then there was 85% chance of Obama winning.  But that is also worthless.  Election day is not some unknown date.  I can create a model that says the Dow or the S&P will do such and such on this coming Saturday and Sunday...but of course...you can't prove my model wrong because the market wasn't open those days.

The 538 model has a one Presidential election track record.  I know people like to point to 49/50 states, but how many of those were tough calls?  Three states?  IN, MO, NC?  Maybe MT?  And IN was a miss.  2008 was not a difficult election to call.  This one is.

Then what the hell do you want from that model?
To predict that Obama will have a sh*t debate and that there will be a huge reaction?

You don't want a model,you want some sort of magician.

[SirMuxALot]

Then what the hell do you want from that model?
To predict that Obama will have a sh*t debate and that there will be a huge reaction?

You don't want a model,you want some sort of magician.

I want it to correctly account for the softness in an incumbent's support, which clearly the model missed.  If Nate can't get that from current polling methods, then he's going to need to find another way to account for that.  Perhaps the Bickers and Berry model isn't so nutty after all. And maybe Nate should keep an open mind and learn from that.

Romney wasn't *that* good in the 1st debate; and Obama wasn't *that* bad.  What happened there is that Romney peeled away a big chuck of soft Obama supporters.  A good model could and should account for that.  Nate missed that...badly.  And I think he's still missing the minefield of undecideds hanging out there.

This doesn't require magic.  But it does require some knowledge or historical perspective on how polling can be flawed.  And in my opinion, Nate has missed a big chunk of what can be misleading in polls in certain election scenarios.

Was it magic that moved Romney on Intrade from the 20s to mid 30s before the first debate even started?

[Wonkish1]

Hmm, probably should think about that once because there is a very, very high overlap to what SirMux just pointed out and what Morden suggested we judge Nate by.

Basically, with a sharp drop in probabilities like we've seen Mr. Morden's method of measurement would in aggregate come to the same conclusion as what SirMux just did.

Mr. Morden pointed out that to evaluate the model you have to check how accurate the probabilities are. This involves looking at many predictions from the model and comparing them with results. In particular, it would mean ensuring that 15% of the time a candiate is giving an 85% of winning, s/he in fact loses.

SirMux thinks the model has a problem because we can point to a single instance in which a candiate was giving an 85% of winning, and maybe he doesn't win. That's not a problem with the model at all. In fact, whether Obama wins or loses, that on its own says nothing whatsoever about the accuracy of the 85% figure.

They're talking about 2 completely different things.

The reason being is that Mr. Morden's system basically would calculate mistakes in probabilities in advance. Large swings over a short period of time in aggregate would cause either the pre swing probability or the post swing probability to be way off or both.


You just poo pooed another posters method of looking at results and then suggested that a very, very similar one is the best way to do it and you didn't even realize it. Oops!

I don't think you understand in the slightest what Mr. Morden suggested.

Actually I did understand exactly what Mr. Morden suggested. I would also say that your characterization of what Mr. Morden said is 100% spot on and exactly the way I took it.

The problem is that you didn't understand how Mr. Morden's method of measurement would produce a negative outcome for Nate Silver when you see that large of change in short period of time.

You don't understand that Mr. Mordem's method of looking at x, y, z, etc. periods before election day and comparing them with results is extremely correlated with a focus on volatility in probabilities that SirMux referred to.

Here's why:

What Mr. Morden is essentially suggesting is that we basket the 85% predictions, the 70% predictions, the 60%, predictions, etc. at different stages of the race and see if they in fact happened at that level.

The math behind SirMux's thought process is that let's say I have an assumption of an 85% probability of X event happening. With that I can create a normal distribution of other possible probabilities. 85% is in the center and farther away from 85% are in the tails of the curve. Now if a move happens quickly it's either an extreme unlikelihood(lower than the 15% you're thinking) or the original assumption was incorrect.

Now since Mr. Morden's suggested methodology involves taking snapshots at different periods of a race than a major swing between one date and another prior to the election would in aggregate produce at least one of those 2 to be grossly incorrect.

For that reason, whether or not you can understand this Mr. Morden has volatility as a key piece of his suggested methodology. Essentially Sir Mux's thought process is apart of Mr. Morden's. Mr. Morden's just takes into account additional items as well, but if volatility in probabilities rise than Nate Silver would also fall 'in success' in Mr. Morden's system. It has to:

Example:

One day it's candidate A 80% likely, the next month its candidate B 80% likely(very volatile outcome). Once Period A and period B get put in a basket of 80% probabilities(which was Mr. Morden's analysis would have to do) and then compared them to the results Nate Silver would take a hit regardless of the actual outcome. This is more of an extreme example, but if the level of volatility mentioned by SirMux happened many times then Mr. Morden's methodology would show Nate Silver's probabilities to not be that accurate.

[Wonkish1]

The reason it was at 85% instead of 100% was due to the evaluation that Romney could still come back.

I'm sorry, but then it is a worthless model.  By this definition, it could have been at 99.99% for Obama, and it's "still not wrong" since it allowed for a Romney comeback.  C'mon...that 85% was garbage.  If it wasn't garbage, then it would still be close to that.  I thought the 85% then was garbage.  And I think his current 72% is equally garbage.

I know how Nate would defend this.  He would say, well, if the election were held on that day, then there was 85% chance of Obama winning.  But that is also worthless.  Election day is not some unknown date.  I can create a model that says the Dow or the S&P will do such and such on this coming Saturday and Sunday...but of course...you can't prove my model wrong because the market wasn't open those days.

The 538 model has a one Presidential election track record.  I know people like to point to 49/50 states, but how many of those were tough calls?  Three states?  IN, MO, NC?  Maybe MT?  And IN was a miss.  2008 was not a difficult election to call.  This one is.

Then what the hell do you want from that model?
To predict that Obama will have a sh*t debate and that there will be a huge reaction?

You don't want a model,you want some sort of magician.

In theory the model would account for a large range of possible changes in the race and keep the probabilities closer together when you're farther away from the election and really only allow for one candidate to get a large spread in their probability as the election approached(with less time available to produce a material change in the race).

Really think about what I just said and look back to how Mr. Morden would analyze Nate Silver and you'll understand intimately why his 'probabilities' are bound to be way off.


This simple post is all I need to say in order to claim game, set, and match if we're operating off of Mr. Morden's method of analysis.

[Wonkish1]

The reason it was at 85% instead of 100% was due to the evaluation that Romney could still come back.

I'm sorry, but then it is a worthless model.  By this definition, it could have been at 99.99% for Obama, and it's "still not wrong" since it allowed for a Romney comeback.  C'mon...that 85% was garbage.  If it wasn't garbage, then it would still be close to that.  I thought the 85% then was garbage.  And I think his current 72% is equally garbage.

I know how Nate would defend this.  He would say, well, if the election were held on that day, then there was 85% chance of Obama winning.  But that is also worthless.  Election day is not some unknown date.  I can create a model that says the Dow or the S&P will do such and such on this coming Saturday and Sunday...but of course...you can't prove my model wrong because the market wasn't open those days.

The 538 model has a one Presidential election track record.  I know people like to point to 49/50 states, but how many of those were tough calls?  Three states?  IN, MO, NC?  Maybe MT?  And IN was a miss.  2008 was not a difficult election to call.  This one is.

Guys truly read this once because this guy is very smart.

If you don't understand how he's blowing apart your argument it's you not him.

[7,052,770]

A model that isn't wrong about approximately 1/6 of the things it gives an 85% chance to is worthless. Also his number is always the final probability of winning, not what it would be that day. He calculates the today number in the Nowcast.

[cavalcade]

He calculates the today number in the Nowcast.

He is showing Obama +1 for the national popular vote in the "nowcast".  If today's update still has Obama winning the national popular vote that would be pretty damning.

The fact is, a lot of Silver's appeal is that he takes the conventional wisdom- "Obama is the favorite right now because he leads in the electoral college"- and "calculates" a 68.3% chance of him winning or something like that.  He really doesn't work off much more than we do, and in the past he hasn't been much more accurate than RCP averages in races with lots of polling.

Where he can really shine is where there isn't a lot of polling, like in the 2008 D primaries.  I bet he'll come a lot closer in places like Kansas or Washington than the polls will.

[emailking]

The reason it was at 85% instead of 100% was due to the evaluation that Romney could still come back.

I'm sorry, but then it is a worthless model.  By this definition, it could have been at 99.99% for Obama, and it's "still not wrong" since it allowed for a Romney comeback.

You're right. It wouldn't be wrong in that case either. The same way you can calculate that the probability of winning the lottery is perhaps 1 in a million and then notice that somebody won it. The only way you would be able to prove it wrong with one data point would be if it gave a 100% or 0% evaluation, and the opposite occurred. If it had been 99.99% and then Romney won, that doesn't impugn the model. However, if the model really were as worthless as you imply it would be in that case, it would not be hard to show that rather easily with a few more data points. The model is only "worthless" if it doesn't stand up scrutiny upon multiple evaluations, enough to be statistically valid. If you don't like that answer, well you can ignore the model if you want. But your problem isn't with anything Nate has done, your issue is with the reality of probability and statistics.

C'mon...that 85% was garbage.  If it wasn't garbage, then it would still be close to that.

No it wouldn't. Because in most of the 15% of scenarios underdog wins, s/he does it by coming back steadily as the election approaches, rather than having a huge surge at the last moment (which is less likely).

I know how Nate would defend this.  He would say, well, if the election were held on that day, then there was 85% chance of Obama winning.  But that is also worthless.  Election day is not some unknown date.  I can create a model that says the Dow or the S&P will do such and such on this coming Saturday and Sunday...but of course...you can't prove my model wrong because the market wasn't open those days.

Sure. But you understand then every single poll is also worthless. Because they ask the respondent how s/he were vote if the election were at hand, but it isn't.

The 538 model has a one Presidential election track record.  I know people like to point to 49/50 states, but how many of those were tough calls?  Three states?  IN, MO, NC?  Maybe MT?  And IN was a miss.  2008 was not a difficult election to call.  This one is.

We'll see. I'm not defending the model. I am saying this line of criticism is nonsense.

[emailking]

Example:

One day it's candidate A 80% likely, the next month its candidate B 80% likely(very volatile outcome). Once Period A and period B get put in a basket of 80% probabilities(which was Mr. Morden's analysis would have to do) and then compared them to the results Nate Silver would take a hit regardless of the actual outcome. This is more of an extreme example, but if the level of volatility mentioned by SirMux happened many times then Mr. Morden's methodology would show Nate Silver's probabilities to not be that accurate.

No it wouldn't! The chance of candidate A winning may really be 80% one month and 20% the next. You're only looking at a single data point (who wins this single hypothetical election) and claiming it has anything to say about the accuracies that were determined in the leadup to the result.

It could go 99%, 1%, 99%, 1%, etc. It still doesn't matter.

If the model is really that volatile because it's bad (and not capturing something reflected in reality) then it would only take a few more elections with such volatility in the model to show that it is highly likely to wrong. (You would need a lot more before you could demonstrate that it's right about such volatility.) But you can't do it from one!

[SirMuxALot]

Sure. But you understand then every single poll is also worthless.

Nonsense.  The polls add to our knowledge in the sense that they give us information that is otherwise not discoverable by an average person.  And they (mostly) don't claim to give us a predictive metric.

But what is Nate's 70% "Chance of Winning"?  We don't need to guess if Nate intends for it to be a prediction or simply a "present state of the race".  He himself calls it a "Forecast".  He's claiming to add value to the poll data.  He is claiming to add to our knowledge about what *will* happen, not what has happened.

The value of Nate's model must be judged by its predictive ability since he promotes it as a forecast.  If any outcome on election day can still be called a success by Nate's model, then it is adding absolutely no value as a "forecast".

I think the fact it was clinging onto 85% at the moment the 1st debate started while Intrade had already moved Obama down to 65% at that same moment tells a little something about its predictive value.

Clearly we will know more about its predictive value after the election.  But let me ask this...let's say on the morning of the election and Nate pulls a Zogby and flips his prediction to 50.1% Romney win, 49.9% Obama win.  And Romney wins.  Does that prove his model's value?

Or let's try this...maybe Nate goes Dick Morris on us and makes Obama 99.9% that morning, Romney 0.01%.  And Obama wins the EC 270-268, with a popular vote margin less than half a percent.  Did his model prove it's value in that case?  In both cases, it's a "win" and he's 2-for-2 in Presidential election forecasts.

[SirMuxALot]

And I'll be curious to read Nate's "thought dump" on Intrade tomorrow.

I'll lose a lot of respect for him if all he does is crap on Intrade and doesn't mention that Intrade correctly predicted the shift to Romney *before* the 1st debate; while his model was still running out that 85% number for Obama for days afterwards.

[Mr. Morden]

Nate Silver's model is indeed attempting to predict the future, however it's doing so using only existing published information.  That is, it uses polls and economic data, which take time to produce.  So obviously if a candidate has really strong debate performance for example, it's going to take time for that to be priced in, because it takes time for polling firms to conduct post-debate polls.  Whereas with Intrade, bettors are able to price in information from a debate performance instantaneously, because the bettors can try to predict what the polls are going to look like in advance.

It's not problematic per se to have a race where the odds are at 85%/15%, and then start shifting closer to 50/50.  That sort of thing should happen once in a while.  But there are more elections in which one is fairly uncertain of what's going to happen six months in advance, but then by the day before the election, the polls give you a pretty good idea of who's going to win, so the probabilities should be closer to 100%/0%.  As I recall, we had this discussion after the 2010 midterms, when there were a couple of Senate races that 538 gave one candidate a 95% chance of winning many months in advance, but then saw the odds move closer to 50/50 when the polls tightened.  That sort of thing should happen only very rarely.

[nhmagic]

Another indictment is the updates over the last two days.  Today, there were four national polls released today, 3 of which showed Romney leads, and one which showed a bare lead for Obama.  His forecast for today - Romney still losing the popular vote, Romney actually losing ground in the electoral vote even when state polls continued to show movement towards Romney if you look at the averages. 

Even more curious, Nate waits till late in the evening to do updates when Romney is doing better.  On days where national polls come in with leads for Obama, he'll give a mid-day update.  I think he's tweaking his model to be beneficial to Obama when favorable Romney polls come in.  Also, look at the weight he gives that crap UNH poll for NH - a 9pt lead for Obama (with the governor's race an 8 pt lead for Hassan).  It's ridiculous.  I can't wait for this election to prove his model wrong; however, I guarantee that he's gonna start moving things toward Romney at the last minute possible to save credibility and give a long explanation as to why the model worked.  Then, the next election democrats will be using it again to say: See look at what Nate says. 

[SirMuxALot]

Nate Silver's model is indeed attempting to predict the future, however it's doing so using only existing published information.

(That is, polls.)  Yes - this is precisely my point.  The model has serious flaws because it's missing entire categories of inputs it should account for.  (This is exactly what the Univ. of Colorado guys are trying to find with their non-poll based model.)  I hope Nate doesn't just settle on "it's the pollsters fault" after election day and run the same flawed model out there again in 2016.  It needs some big holes filled.

Intrade did not react to the 1st debate.  It predicted.  Romney went from mid-20s to about 36 the moment the 1st debate started.  Romney actually traded slightly down during the debate, ending at 34 when it was over.  Intrade added some value there where 538 did not.

Somewhere out there is additional (probably counter-intuitive) information that aggregate Intrade traders picked up on that the 538 model missed.  It might be a tough task to figure out how to account for that, but Nate should make an effort.  I think finding a way to measure the softness of support in Aug/Sept polls is one of those big holes he needs to fill.

[emailking]

Sure. But you understand then every single poll is also worthless.

Nonsense.  The polls add to our knowledge in the sense that they give us information that is otherwise not discoverable by an average person.  And they (mostly) don't claim to give us a predictive metric.

Well of course. It's not actually worthless. Just like Nate's model is not worthless for predicting the outcome of a future event if it somehow happened today.

But what is Nate's 70% "Chance of Winning"?  We don't need to guess if Nate intends for it to be a prediction or simply a "present state of the race".  He himself calls it a "Forecast".

He has two results. One is the "present state of the race" which he calls the now-cast. The other is the "forecast." The 85% number was in the forecast.

The value of Nate's model must be judged by its predictive ability since he promotes it as a forecast.  If any outcome on election day can still be called a success by Nate's model, then it is adding absolutely no value as a "forecast".

Either you don't understand what I am saying or you are twisting my words. I'll give you the benefit of the doubt. Of course it must be judged by its predictive ability. But you need MANY elections to do this. You cannot do it from one.

A result on election day is neither a success nor a failure for the model. On its own, it's nothing.

You have reasoned several times now that his model is bunk because it once said 85% and now it says something else. That's  nonsense.

Clearly we will know more about its predictive value after the election.  But let me ask this...let's say on the morning of the election and Nate pulls a Zogby and flips his prediction to 50.1% Romney win, 49.9% Obama win.  And Romney wins.  Does that prove his model's value?

No.

Or let's try this...maybe Nate goes Dick Morris on us and makes Obama 99.9% that morning, Romney 0.01%.  And Obama wins the EC 270-268, with a popular vote margin less than half a percent.  Did his model prove it's value in that case?  In both cases, it's a "win" and he's 2-for-2 in Presidential election forecasts.

It's only a "win" according to people who don't understand statistics.

[The Constable]

I can't link yet but I highly recommend Sam Wang (do not know how many here know of him). His blog can be found at election[dot]princeton[dot]edu. His model is somewhat similar to Nate's but less arbitrary, and he has consistently outpredicted Silver since Silver has started (Sam started in '04).

[tokar]

The 538 model has a one Presidential election track record.  I know people like to point to 49/50 states, but how many of those were tough calls?  Three states?  IN, MO, NC?  Maybe MT?  And IN was a miss.  2008 was not a difficult election to call.  This one is.

You have a good point. He only has a 1-POTUS-election track record. 1 data point is not enough to make a trend. OK, point taken. But, honestly, do you think his current predictions are really that far off?

As an earlier poster pointed out, Nate has only been wrong 4 times (margins aside):
2008 - Indiana...he had R+1.5
2008 - NE-02...he had R+6.0
2010 - Nevada...he had R+5.5
2010 - Colorado...he had R+1.0

That is what, 4 out of 128 (50 states in 2008, 5 split EV's in 2008, DC in 2008, 35 senate seats 2008, 37 senate seats in 2010), for a rate of 96.88%?

You could go even further and match his predicted margins (I can't find his Senate 2010 predicted margins outside of the 2 noted above).

In 2008, in all the states that Obama won, he UNDER-PREDICTED democratic performance in all instances except for 4 (out of 32):
ME-2 - over-predicted by 2.44%
NH - 0.19%
IA - 2.16%
NC - 0.68%
Of the states Obama lost, he OVER-PREDICTED democratic performance in all instances except for 5 (out of 24):
MO - under-predicted by 0.07%
MT - 0.32%
SD - 0.29%
SC - 0.73%
AL - 0.62%

So what does this say? It says democrats did BETTER than predicted in the states he predicted democrats to win, while democrats did WORSE in states he predicted them to lose.

Not to say I am using the above generalized statement as a basis to view the 2012 election, but if one WAS to do that, it would mean than Democrats probably shouldn't be worried based on his election prediction.

The notable swing states:
NV: predicted D+4.9, was D+12.5, under-predicted democratic performance by 7.6.
NM: predicted D+9.7, was D+15.13, under-predicted by 3.84.
IN: predicted R+1.5, was D+1.03, under-predicted by 2.53.
WI: predicted D+11.5, was D+13.91, under-predicted by 2.41.
CO: predicted D+6.6, was D+8.95, under-predicted by 2.35.
PA: predicted D+8.1, was D+10.32, under-predicted by 2.22.
OH: predicted D+3.4, was D+4.58, under-predicted by 1.18.
FL: predicted D+1.7, was D+2.81, under-predicted by 1.11.
VA: predicted D+11.5, was D+13.91, under-predicted by 2.41.
MO: predicted R+0.2, was R+0.13, over-predicted democratic performance by 0.07.
NH: predicted D+9.8, was D+9.61, over-predicted by 0.19.
NC: predicted D+1.0, was D+0.32, over-predicted by 0.68.
IA: predicted D+11.7, was D+9.54, over-predicted by 2.16.