The polls getting it wrong - post notable examples

[Phony Moderate]

Can be national elections, local, state, referenda etc.

UK 1970 - Wilson's Labour seems on course for a clear, perhaps large, majority. In fact Heath's Tories win a solid majority of their own. Explanations/excuses include Gordon Banks being ill and bad balance of payment figures on the eve of the poll.

UK 1992 - Kinnock-led Labour seems sure to be in office, either as a minority, in a coalition with the Lib Dems or even with an outright majority. Major's Tories hold on with a smallish majority and easily win the PV by around 7%. Down, apparently, to Shy Tories and the Sheffield Rally.

UK 2015 - Few can predict who will be PM but the consensus is that there will be a gridlock with the SNP holding the balance of power. Cameron's Tories hold off Labour and destroy the Lib Dems in England to win their first majority since 1992. Polling errors blamed on failure to predict turnout accurately, the Royal Baby and the EdStone.

[DC Al Fine]

British Columbia 2013 - The pollsters missed the NDP-Liberal margin by a whopping 12% the day before the election; the difference between an NDP majority and a Liberal one.

Alberta 2012 - Danielle Smith defends anti-gay remarks by a candidate. There is about a 10% swing from Wildrose to the PC's over the weekend immediately before the election.

Canada 2011 - A recent example of the Shy Tory effect. Pollsters underestimate the Tories by 3.5-5.5%, completely failing to predict the Tory majority that occurred.

[○∙◄☻¥tπ[╪AV┼cVê└]

https://en.wikipedia.org/wiki/New_York_gubernatorial_election,_1994#Polling

[Filuwaúrdjan]

The polls weren't that far out in the recent GE here, it just happens that one feature of FPTP is that if you are somewhat wrong in terms of votes then you can be a long way out in terms of seats. One reason why it felt to many as if they were miles out was because of the absurd faith the media had placed throughout the campaign in dubious seat projection models.

[jaichind]

India 2004 general election.  Polls had NDA winning 290-300 seats out of 543.  Exit polls had NDA winning around 250-270 seats.  NDA ended up winning 187 seats and behind UPA.

[afleitch]

The polls weren't that far out in the recent GE here..

Err. They were. The models were the cherry on top of a cake of sh!t but were marginally more accurate than UNS. Every election model failed because the polls were simply wrong.

[🦀🎂🦀🎂]

How about that Literary Digest poll?

[Tender Branson]

Just a few weeks ago at the Styria state election:

The polls had the FPÖ at 17-22%, but they got 27%.

[Filuwaúrdjan]

Err. They were. The models were the cherry on top of a cake of sh!t but were marginally more accurate than UNS. Every election model failed because the polls were simply wrong.

For the two main parties the polls were typically out by between 2pts and 4pts and this for parties polling in the 30s: that's not good (and as it was consistent indicates industry wide errors), but was a degree of magnitude less severe than 1992 where the range was between 3pts and 8pts out...

[Antonio the Sixth]

The polls predicted a virtual tie, and it ended up a 7-point Conservative victory. This is not mere statistical variance. I don't know about Britain, but in the USA, France, and even friggin' Italy, it is extremely rare for polls to miss by so much. There was a large systematic bias in polls, and it is time British pollsters do something to correct it.

[Phony Moderate]

Prop 8 and the 1995 Quebec referendum are two milder examples.

[Filuwaúrdjan]

This is not mere statistical variance

I don't think I suggested otherwise?

But yes, British pollsters are indeed pretty bad by international standards; this has been noted for some time.

[Gustaf]

In the Swedish euro referendum a couple of polls showed a tie right before it happened. The actual result was a 14% landslide for no. I suspect those polls were basically bought though.

[YL]

The polls weren't that far out in the recent GE here..

Err. They were. The models were the cherry on top of a cake of sh!t but were marginally more accurate than UNS. Every election model failed because the polls were simply wrong.

Yes, with the seat models it was garbage in, so it was garbage out.

The polls weren't that great for the 2010 UK election either, but then it was the Lib Dems who were overestimated and the Con/Lab margin wasn't too far out.  I think the pollsters ought to look at that election too.

[Phony Moderate]

2005 was probably the best election for the pollsters; they were all within 1-2% or so. The Labour majority was overestimated in 1997 and 2001, 1987 was alright (though slightly overestimating the Labour recovery), 1983 overestimated the Tory landslide, 1979 was pretty good, October 1974 looked set to be a repeat of 1966, February 1974 looked set to be a Tory victory with the Liberals winning dozens of seats.

So yeah, not great.

[palandio]

The 2005 German federal election:
All pollsters predicted a CDU/CSU result of at least 41%, an SPD result of 34% at most and a clear CDU/CSU+FDP majority.
In the end the CDU/CSU got 35.2%, the SPD 34.2% and CDU/CSU+FDP were nowhere close to a majority.

From what I remember the 2002 German federal election had a similar phenomenon.

Also Italian elections in general, including exit-polls.

[Gunnar Larsson]

In the Swedish euro referendum a couple of polls showed a tie right before it happened. The actual result was a 14% landslide for no. I suspect those polls were basically bought though.

The EU referendum was quite similar in that it overestimated the pro-European vote quite a lot. At least the exit poll showed a yes landslide (about 15 % margin, if I remember right), whilst the actual result was quite close (~5 %).

I think with the Euro referendum it was also the exit poll that was particularly off. My memory is that No tended to have a sizeable lead in the usual polls. I know that I contemplated voting yes just to keep the numbers down as I didn't want the referendum to lead to calls for a new EU-referendum etc.

[buritobr]

What I wrote in the other thread
https://uselectionatlas.org/FORUM/index.php?topic=212502.0

"It is a worldwide phenomenum. Polls made in the election eve and even the exit polls underestimate the vote for the right. Sometimes, these errors are bigger than the margin of the error of the poll. We can see many examples

UK
Conservative victories in 1970 and 1992 were unexpected. Conservative majority in 2015 was unexpected.

USA
Reagan 10 point margin in 1980 was unexpected. Polls in the eve were showing 3 point margin.
Clinton margins in 1992 and 1996 were smaller than it was expected in the eve.
Polls in the first day of November were showing Bush and Kerry tied. Bush had a 2.5 point margin.
Some polls in the week of the 2008 election were showing 10 point margin for Obama.

Israel
Polls were predicting Netanyahu defeat in 2015

Germany
No poll predicted 41.5% for the CDU/CSU in 2013. The sum of the vote for CDU/CSU + FDP + AfD was bigger than it was expected. The pure right wing coalition failed only because of the rule of 5%.
SPD had 23% in 2009. During most of the days of the campaign, polls were predicting this percentage for the SPD. But in the last week, polls predicted a SPD surge that didn't happen. In the last week, many expected that the great coalition would remain in the power. But the CDU/CSU + FDP had enough votes to reach the majority.

France
Jospin's failure to reach the runoff in 2002 was a surprise

Brazil
PT was the winner of the presidential elections of 2002, 2006, 2010, 2014. But in these four elections, PSDB had more votes than the quantity predicted both by the eve polls and the exit polls

There are some examples of the opposite, like the US presidential election of 1948, but these examples are very rare."

[Filuwaúrdjan]

You know if you mention 1970 you should also mention February 1974, which was about as big a shock.

[Hatman 🍁]

I don't think you can possibly beat Alberta 2012 or BC 2013.

[Paleobrazilian]

1st round in 2010 and 2014 had some atrocious mistakes here in Brazil - both times Dilma Rousseff was overestimated by about 4%. IBOPE has been specially bad at national polls.

On Governor elections and Senator elections we had some even bigger misses. Some states like Bahia are notoriously hard to poll due to the large rural population and the remoteness of some areas. Also, the fact that many Brazilians only decide who they'll vote for Governor and Senator on the verge of the election makes it even hard to poll those races. Here are some examples:

Bahia, 2006 and 2014, Governor: pollsters failed to realize the surge of the PT candidate, who was believed to be some 10% behind on the 1st round and ended up winning on the 1st round.

São Paulo, 2010, Senator: pollsters failed to predict Aloysio Nunes (PSDB) would be elected. In the end not only he was elected, but nearly 60% of the voters voted for him.

Rio de Janeiro, 2014, Governor: pollsters didn't notice the collapse of Anthony Garotinho (PR), who ended up not even making the runoff against Governor Pezão (PMDB).

São Paulo, 1998, Governor: one of the most infamous cases here in Brazil. A poll conducted by Datafolha showed Marta Suplicy (PT) lagging way behind a dead heat between Mario Covas (PSDB) and Francisco Rossi (PDT), who were battling for a runoff spot against Paulo Maluf (PPB). Covas started asking Suplicy's voters to vote tactically for him, so that a center-left candidate could make the runoff. It was also believed that Covas would be the strongest candidate against Maluf, who was already a very controversial name by then. A few days later, though, it was realized that Mrs. Suplicy was badly underpolled, as she ended up only 50000 votes behind Covas, who barely made made it to the runoff. This ended up having far reaching consequences, as Covas would go on to famously mount one of the most amazing comeback campaigns ever, eventually defeating Maluf and getting reelected. After this, the PSDB cemented their hold to São Paulo, a state that they now hold for 21 years. Many on the left still loathe Datafolha for this poll, even though Datafolha was by far the most accurate national pollster in Brazil on the last 2 national elections

[Boston Bread]

I remember that poll showing Huelskamp losing in the most R district in Kansas. Off by, what, a 40 point margin?

[Phony Moderate]

What I wrote in the other thread
https://uselectionatlas.org/FORUM/index.php?topic=212502.0

"It is a worldwide phenomenum. Polls made in the election eve and even the exit polls underestimate the vote for the right. Sometimes, these errors are bigger than the margin of the error of the poll. We can see many examples

UK
Conservative victories in 1970 and 1992 were unexpected. Conservative majority in 2015 was unexpected.

USA
Reagan 10 point margin in 1980 was unexpected. Polls in the eve were showing 3 point margin.
Clinton margins in 1992 and 1996 were smaller than it was expected in the eve.
Polls in the first day of November were showing Bush and Kerry tied. Bush had a 2.5 point margin.
Some polls in the week of the 2008 election were showing 10 point margin for Obama.

Israel
Polls were predicting Netanyahu defeat in 2015

Germany
No poll predicted 41.5% for the CDU/CSU in 2013. The sum of the vote for CDU/CSU + FDP + AfD was bigger than it was expected. The pure right wing coalition failed only because of the rule of 5%.
SPD had 23% in 2009. During most of the days of the campaign, polls were predicting this percentage for the SPD. But in the last week, polls predicted a SPD surge that didn't happen. In the last week, many expected that the great coalition would remain in the power. But the CDU/CSU + FDP had enough votes to reach the majority.

France
Jospin's failure to reach the runoff in 2002 was a surprise

Brazil
PT was the winner of the presidential elections of 2002, 2006, 2010, 2014. But in these four elections, PSDB had more votes than the quantity predicted both by the eve polls and the exit polls

There are some examples of the opposite, like the US presidential election of 1948, but these examples are very rare."

'Very rare' is a stretch; Gallup had Romney in the lead in its final poll in 2012, most polls had Bush ahead in 2000, FDR was underestimated in 1936, 1940 and 1944. The Canadian Liberals did better than expected in 2004 and 2006 (though not in 2008). Boris Johnson was overestimated in 2012. Harry Reid was expected to lose in 2010, as was Michael Bennet.

[buritobr]

1st round in 2010 and 2014 had some atrocious mistakes here in Brazil - both times Dilma Rousseff was overestimated by about 4%. IBOPE has been specially bad at national polls.

On Governor elections and Senator elections we had some even bigger misses. Some states like Bahia are notoriously hard to poll due to the large rural population and the remoteness of some areas. Also, the fact that many Brazilians only decide who they'll vote for Governor and Senator on the verge of the election makes it even hard to poll those races. Here are some examples:

Bahia, 2006 and 2014, Governor: pollsters failed to realize the surge of the PT candidate, who was believed to be some 10% behind on the 1st round and ended up winning on the 1st round.

São Paulo, 2010, Senator: pollsters failed to predict Aloysio Nunes (PSDB) would be elected. In the end not only he was elected, but nearly 60% of the voters voted for him.

Rio de Janeiro, 2014, Governor: pollsters didn't notice the collapse of Anthony Garotinho (PR), who ended up not even making the runoff against Governor Pezão (PMDB).

São Paulo, 1998, Governor: one of the most infamous cases here in Brazil. A poll conducted by Datafolha showed Marta Suplicy (PT) lagging way behind a dead heat between Mario Covas (PSDB) and Francisco Rossi (PDT), who were battling for a runoff spot against Paulo Maluf (PPB). Covas started asking Suplicy's voters to vote tactically for him, so that a center-left candidate could make the runoff. It was also believed that Covas would be the strongest candidate against Maluf, who was already a very controversial name by then. A few days later, though, it was realized that Mrs. Suplicy was badly underpolled, as she ended up only 50000 votes behind Covas, who barely made made it to the runoff. This ended up having far reaching consequences, as Covas would go on to famously mount one of the most amazing comeback campaigns ever, eventually defeating Maluf and getting reelected. After this, the PSDB cemented their hold to São Paulo, a state that they now hold for 21 years. Many on the left still loathe Datafolha for this poll, even though Datafolha was by far the most accurate national pollster in Brazil on the last 2 national elections

There are many mistakes concerning polls about gubernatorial elections in Brazilian states, because since these elections take place in the same day of the presidential election, and people pay attenction only to the presidential election during the campaign, many people choose their candidate for state governor only on the election day.

[○∙◄☻¥tπ[╪AV┼cVê└]

I remember that poll showing Huelskamp losing in the most R district in Kansas. Off by, what, a 40 point margin?

You mean the Rasmussen 2010 HI-Sen might not be the only off by 40 points of margin poll?