538 Model Megathread

[Antonio the Sixth]

Because I was very close to giving Florida to Trump in my final map, which would have meant getting one less state wrong. My trust in his model was the reason I kept it in Hillary's column.

His model was saying Florida was a tossup. A reasonable person can take this information and still go with their gut feeling.

[Figueira]

He was never a psychic, but he's still the best poll aggregator.

[Mr. Morden]

That makes a lot of sense. I think 538 might use past election results as a baseline for how nationwide margins translate into statewide ones - thus, it tends to underestimate the extent to which States are trending away from their prior results.

Do they actually do that in the polls only model on states that actually get polled though?

Even if they do, what about if you compare how straight up polling averages in the more heavily polled states (taken from somewhere like HuffPo or RCP, which doesn't include the 538 secret sauce) did relative to their actual vote results?  Do you see the same correlation with the trend map?  If so, then maybe it means that the pollsters themselves were weighting their results in a way that would match up with the 2012 results, and so were blind to real shifts?

[Attorney General, Senator-Elect, & Former PPT Dwarven Dragon]

In the polls-only they used national polls to estimate the outcome in each state, but gave individual state polls more weight than national polls. Polls-plus was exactly the same but was set to be less reactive to sudden polling shifts and modify the numbers based on economic conditions, which predicted that the race should be around 50-50.

[Mr. Morden]

In the polls-only they used national polls to estimate the outcome in each state, but gave individual state polls more weight than national polls. Polls-plus was exactly the same but was set to be less reactive to sudden polling shifts and modify the numbers based on economic conditions, which predicted that the race should be around 50-50.

But were they using 2012 vote results as part of the prediction in any of the models?  Or maybe they were doing that, but only in states that never got polled?

[Attorney General, Senator-Elect, & Former PPT Dwarven Dragon]

In the polls-only they used national polls to estimate the outcome in each state, but gave individual state polls more weight than national polls. Polls-plus was exactly the same but was set to be less reactive to sudden polling shifts and modify the numbers based on economic conditions, which predicted that the race should be around 50-50.

But were they using 2012 vote results as part of the prediction in any of the models?  Or maybe they were doing that, but only in states that never got polled?

Not entirely sure. They explained the usage of national polls to predict state polls as (basically) "Well, if the polls were showing the republican pulling away nationally, you'd expect them to be doing well in all or almost all of the battleground states".

[Mr. Morden]

In the polls-only they used national polls to estimate the outcome in each state, but gave individual state polls more weight than national polls. Polls-plus was exactly the same but was set to be less reactive to sudden polling shifts and modify the numbers based on economic conditions, which predicted that the race should be around 50-50.

But were they using 2012 vote results as part of the prediction in any of the models?  Or maybe they were doing that, but only in states that never got polled?

Not entirely sure. They explained the usage of national polls to predict state polls as (basically) "Well, if the polls were showing the republican pulling away nationally, you'd expect them to be doing well in all or almost all of the battleground states".

Well, in any case, like I said, it would be interesting to see if you take just a straight up polling average, and ignore all the extra things that 538 does....if you take that polling average for each state and compare to the actual results, does the map of deviations correlate with the trend map, as posted upthread?  If so, then I think it probably hints at the fact that the pollsters themselves were weighting things to look like 2012, which caused them to miss the real changes that were taking place.

[emailking]

I feel like Silver came closer to the others not because he detected something about the electorate that they didn't, but because his assumption of huge error bars made everything go closer to 50-50. He was more right, but only trivially so.

But he had much smaller error bars in 2012, which also looked like a much closer election going in. So yeah, he detected there was a lot more uncertainty this time.

[Slander and/or Libel]

I feel like Silver came closer to the others not because he detected something about the electorate that they didn't, but because his assumption of huge error bars made everything go closer to 50-50. He was more right, but only trivially so.

But he had much smaller error bars in 2012, which also looked like a much closer election going in. So yeah, he detected there was a lot more uncertainty this time.

But was there more uncertainty this time (lower precision), or were the estimates just centered at the wrong place (lower accuracy)? These differences matter. Putting everything down to greater uncertainty implies that things were still centered at the right place.

[emailking]

There was definitely lower accuracy. His model predicted that there was also lower precision in contrast to 2012, whereas the other prominent models did not.

[Antonio the Sixth]

I feel like Silver came closer to the others not because he detected something about the electorate that they didn't, but because his assumption of huge error bars made everything go closer to 50-50. He was more right, but only trivially so.

But he had much smaller error bars in 2012, which also looked like a much closer election going in. So yeah, he detected there was a lot more uncertainty this time.

But was there more uncertainty this time (lower precision), or were the estimates just centered at the wrong place (lower accuracy)? These differences matter. Putting everything down to greater uncertainty implies that things were still centered at the right place.

There was definitely more uncertainty. Polling numbers bounced back and forth a lot more than in 2012 when they were almost perfectly stable all through, and there was a much higher number of undecideds and 3rd party voters. Those are the two main cause of polling errors.

[Slander and/or Libel]

I feel like Silver came closer to the others not because he detected something about the electorate that they didn't, but because his assumption of huge error bars made everything go closer to 50-50. He was more right, but only trivially so.

But he had much smaller error bars in 2012, which also looked like a much closer election going in. So yeah, he detected there was a lot more uncertainty this time.

But was there more uncertainty this time (lower precision), or were the estimates just centered at the wrong place (lower accuracy)? These differences matter. Putting everything down to greater uncertainty implies that things were still centered at the right place.

There was definitely more uncertainty. Polling numbers bounced back and forth a lot more than in 2012 when they were almost perfectly stable all through, and there was a much higher number of undecideds and 3rd party voters. Those are the two main cause of polling errors.

Other aggregators disagreed about the level of uncertainty. And anyway, the uncertainty that mattered in Nate's model was the amount that he let today's results drift to get to Election Day. If you're centered off target, decreasing precision is definitely going to make you hit the target more often, but not because your model is better calibrated at hitting that target. Only because making things more scattershot means that if you're aiming at the wrong place, a bigger error means you'll more often be accidentally right.

It may be the case that this is the proper way to set things, and no election prediction should really be above 70%. But it's worth understanding that Nate's model was more right about Trump not because it saw something specific, but because he calibrated it to be way less specific.

[emailking]

It may be the case that this is the proper way to set things, and no election prediction should really be above 70%. But it's worth understanding that Nate's model was more right about Trump not because it saw something specific, but because he calibrated it to be way less specific.

As long as the odds are calibrated correctly, that's all I (and I think many others) are looking for. If his model says Trump's odds are 30%, I want to be confident that's what his odds are.

[Antonio the Sixth]

Other aggregators disagreed about the level of uncertainty.

Because they didn't take any account of the factors I just mentioned. Clearly they have something to learn from 538 in that respect at least.