How did the 2012 predictions compare?
       |           

Welcome, Guest. Please login or register.
Did you miss your activation email?
March 18, 2024, 11:45:18 PM
News: Election Simulator 2.0 Released. Senate/Gubernatorial maps, proportional electoral votes, and more - Read more

  Talk Elections
  Other Elections - Analysis and Discussion
  Election Predictions (Moderator: muon2)
  How did the 2012 predictions compare?
« previous next »
Pages: [1]
Author Topic: How did the 2012 predictions compare?  (Read 3051 times)
Princess Nyan Cat
nyancat
Rookie
**
Posts: 107
United States


Political Matrix
E: 4.52, S: 4.35

Show only this user's posts in this thread
« on: February 04, 2014, 03:04:02 AM »
« edited: February 04, 2014, 03:14:34 AM by nyancat »

During the 2012 election, my (now) husband and I compared the accuracy of the various predictions for races at the Presidential, Senate, House and Governor levels.  I thought you'd find the results interesting.

The "score" is a simple one based on whether the prediction picked the right party to win and does not attempt to assess strength of victory.  It is often difficult to quantify terms like Tilt, Lean, Likely, Moderate, Strong, Solid, and so on when many of the sites do not clearly define what those qualifications mean.  We also automatically classify a prediction of "tossup" as wrong since the prognosticator was unwilling to make a stand one way or the other.

-----

Presidential race - grades on at least 51 races (50 states + DC) and up to 56 (50 states + DC + congressional districts in Maine and Nebraska)

98.2% - 538 (Nate Silver) - 55/56 correct
98.0% - Election Projection (Scott Elliott) - 50/51 correct
98.0% - Intrade - 50/51 correct
96.3% - Larry Sabato's Crystal Ball - 52/54 correct
96.2% - Race-4-2012 - 51/53 correct
96.1% - Electoral Vote Predictor (Andrew Tanenbaum) - 49/51 correct
94.1% - Talking Points Memo - 48/51 correct
92.2% - Huffington Post - 47/51 correct
88.2% - Karl Rove - 45/51 correct
87.5% - Cook Political Report - 49/56 correct
86.8% - New York Times - 46/53 correct
86.3% - Washington Post - 44/51 correct
84.3% - Rothenberg Political Report - 43/51 correct
84.3% - ABC News - 43/51 correct
84.3% - CNN - 43/51 correct
79.2% - Real Clear Politics - 42/53 correct
78.4% - Wall Street Journal - 40/51 correct

The one state that every single prediction got wrong was Florida.
Logged
Princess Nyan Cat
nyancat
Rookie
**
Posts: 107
United States


Political Matrix
E: 4.52, S: 4.35

Show only this user's posts in this thread
« Reply #1 on: February 04, 2014, 03:06:34 AM »
« Edited: February 04, 2014, 03:14:19 AM by nyancat »

Senate races - graded on 33 races

93.9% - 538 (Nate Silver) - 31 correct
93.9% - Election Projection (Scott Elliott) - 31 correct
93.9% - Larry Sabato's Crystal Ball - 31 correct
87.9% - Rothenberg Political Report - 29 correct
87.9% - Electoral Vote Predictor (Andrew Tanenbaum) - 29 correct
84.8% - Talking Points Memo - 28 correct
84.8% - Washington Post - 28 correct
84.8% - Huffington Post - 28 correct
78.8% - Real Clear Politics - 26 correct
72.7% - New York Times - 24 correct
72.7% - Roll Call - 24 correct
69.7% - Cook Political Report - 23 correct
66.7% - Wall Street Journal - 22 correct

Every single prediction was wrong for Montana and North Dakota.
Logged
Princess Nyan Cat
nyancat
Rookie
**
Posts: 107
United States


Political Matrix
E: 4.52, S: 4.35

Show only this user's posts in this thread
« Reply #2 on: February 04, 2014, 03:08:25 AM »
« Edited: February 04, 2014, 03:14:08 AM by nyancat »

House races - graded on 435 races (number wrong listed instead of correct)

97.2% - Election Projection (Scott Elliott) - 12 wrong
97.0% - Larry Sabato's Crystal Ball - 13 wrong
94.5% - Rothenberg Political Report - 24 wrong
94.5% - Huffington Post - 24 wrong
94.0% - Washington Post - 26 wrong
93.3% - New York Times - 29 wrong
92.9% - Cook Political Report - 31 wrong
92.0% - Real Clear Politics - 35 wrong
91.7% - Roll Call - 36 wrong

Every prediction was wrong for these 8 districts: Florida-18, Kentucky-6, Massachusetts-6, Michigan-1, New Hampshire-1, New York-18, Texas-23, Utah-4

Only one prediction was right for these 3 districts: California-36 (EVP-Scott Elliot), Illinois-10 (New York Times), Pennsylvania-12 (Washington Post)
Logged
Princess Nyan Cat
nyancat
Rookie
**
Posts: 107
United States


Political Matrix
E: 4.52, S: 4.35

Show only this user's posts in this thread
« Reply #3 on: February 04, 2014, 03:10:33 AM »
« Edited: February 04, 2014, 03:13:56 AM by nyancat »

Governor races - graded on 11 races

90.9% - Election Projection (Scott Elliott) - 10 correct
90.9% - Larry Sabato's Crystal Ball - 10 correct
81.8% - Rothenberg Political Report - 9 correct
81.8% - Real Clear Politics - 9 correct
72.7% - Cook Political Report - 8 correct
72.7% - Wall Street Journal - 8 correct
72.7% - Washington Post - 8 correct

Every prediction was wrong for Montana.
Logged
ElectionAtlas
Atlas Proginator
YaBB God
*****
Posts: 3,628
United States


P P P
WWW Show only this user's posts in this thread
« Reply #4 on: February 04, 2014, 03:52:38 PM »

Nice lists - you might want to add the Atlas community aggregate predictions.
Logged
Princess Nyan Cat
nyancat
Rookie
**
Posts: 107
United States


Political Matrix
E: 4.52, S: 4.35

Show only this user's posts in this thread
« Reply #5 on: February 05, 2014, 12:00:20 AM »

We didn't know about this site in 2012, but have added it to the 2014 matrix.  The above predictions fed into the method we developed (which I'll call J&M for our initials).  For comparison, here is how we did and how the Leip Atlas would have affected the results.

President:

98.2% - J&M - 55/56 correct
98.2% - Leip Atlas - 55/56 correct

Including the Atlas would not have changed the outcome, and both match the best of the other predictions. The one incorrect state remains Florida.

Senate:

93.9% - J&M - 31/33 correct
93.9% - Leip Atlas - 31/33 correct

Again, no change in outcome and both match the best of the rest. Both Montana and North Dakota were incorrect.

House:

97.5% - J&M - 11 incorrect

The one area where our method slightly outperformed the others was in the House, where we did one better than Election Projection.

Governor:

100% - Leip Atlas -11/11 correct
90.9% - J&M - 10/11 correct

The Atlas was the only site to predict the Montana race correctly. Had we included it in our calculations, it would have made the race a complete tossup. In such a case, we look at the Real Clear Politics composite poll results and tilt the race to whoever leads it. This would have still resulted in the wrong prediction.
Logged
Pages: [1]  
« previous next »
Jump to:  


Login with username, password and session length

Terms of Service - DMCA Agent and Policy - Privacy Policy and Cookies

Powered by SMF 1.1.21 | SMF © 2015, Simple Machines

Page created in 0.027 seconds with 11 queries.