Step aside Brexit, move out of the way Donald Trump, for the plucky country is once again punching above its weight. Yes, Australia has had its very own spectacular opinion poll flop.
We’ll never know for sure what went wrong, but here are a few ways of looking at the great 2019 polling disaster.
A big two-party-preferred miss: The headline fail was the fact that all public pollsters published Labor two-party-preferred support that rounded to 51 to 52 per cent in the final days of the campaign. When the counting is over it will be close to 48 per cent. So, 3 to 4 per cent off.
That’s about how wrong they were in Victoria last November, but the difference between a big win and a landslide doesn’t generate the same headlines.
A big primary-vote miss: Leaving aside idiosyncratic Ipsos, which always overstates the Greens (mostly at the expense of Labor), the polls exaggerated Labor support by two or three percentage points and understated the Coalition by three or four. See them all at Wikipedia.
Aggregated, that’s a slightly smaller fail in the primary votes than in two-party-preferreds.
Then there’s Queensland: The northern state has a long tradition of excess enthusiasm for Labor in the polls, which it then corrects as election day approaches. In 2019 the trajectory was extra steep. Just six months ago the polls were suggesting a two-party-preferred swing to Labor in that fifth of the electorate of around 7 or 8 per cent. The current estimate is 4.2 per cent — in the other direction.
The most recent pre-election state-by-state data comes from Ipsos and Newspoll, both sets published in the final week of the campaign but taken over the previous several. Allowing again for Ipsos’s bad Labor–Greens habit, it seems that Queensland was the site of the biggest exaggeration of Labor support.
Overstating minor parties, understating the Coalition: There was also a totally predictable overstating of surveyed support for the biggest right-wing minor parties, One Nation and the United Australia Party. One reason is the decision to include them in the initial list of “which of these will you vote for?” All the pollsters included One Nation in the question, while only Galaxy and Newspoll (which happens to be run by Galaxy) included the UAP.
Complicating the picture is the fact that while the UAP ran in all 151 electorates, One Nation only contested fifty-nine. Galaxy, Newspoll and Ipsos seem to have adjusted for this reality, but perhaps not Essential, which had One Nation on a great big 6.6 per cent in its final poll. At last count, One Nation received only 3.1 per cent. So how did Essential’s other 3.5 per cent vote?
It seems likely (though there’s not enough evidence to say, and never will be) that the vast bulk of the people who told pollsters they would vote for these two parties, but didn’t, put a “1” next to the Coalition instead.
Preference flows: This is part of the same minor-party problem. Not only did pollsters overstate them, but by using preference flows from 2016 (One Nation) and 2013 (PUP, Clive Palmer’s previous party) they understated preferences for the Coalition.
Only Galaxy and Newspoll seem to have assumed a relatively high 60 per cent flow to the Coalition from both minor parties, but even that looks likely to be too low. (The Australian Electoral Commission will publish two-party-preferred flows from all parties in a few weeks.)
(I went into this preference issue in a bit more detail earlier in the month.)
The ingredients came together in particularly combustible fashion in Queensland, where support for those minor parties was highest (both in actuality and polled). Still, even allowing for these minor party and preference errors, we’re left with a big polling miss.
Late swing? The statistically skilled commentators in particular tend to be dismissive of the concept of a late swing, seeing it as an excuse from the pollsters. I’m not so sure.
Voting-intention polls aim to measure, within the margin of error and so on, how the voting population would answer the “how will you vote?” question, not how the respondent actually would vote. As election day approaches, the voting scenario becomes less hypothetical and the two concepts should converge, making the final polls the most accurate.
To an extent, all elections involve scare campaigns with potential effects close to polling day. The desire to kick the government out is tempered by misgivings about the opposition and a fear of the unknown. At this election this dynamic was dialled up to the red zone.
A late swing, not captured by the pollsters, could happen. The fact that Labor was widely expected to win could concentrate voters’ minds. Newspoll stopped surveying at noon on Friday. All it takes is for undecideds to break predominantly one way.
As Essential’s Peter Lewis wrote after the election:
Our final poll was still at 8 per cent of voters undecided and thus, removed from the sample, nearly double the number from previous elections. A further 18 per cent had told us they hadn’t been paying a lot of attention to the campaign. That’s more than one-quarter of the electorate.
A counter-argument is that the YouGov/Galaxy exit poll had Labor on 52 per cent after preferences. But polls like these, conducted across a selection of unidentified electorates, have always been rough.
Internal polling: What about the internal party polls? From journalists’ reporting, Labor was delivering mixed messages. One, based on the tracking polls (conducted for the first time by YouGov/Galaxy), had them easily winning. (Just like that exit poll.)
But there was also handwringing in the final week, seemingly based on Labor’s individual seat polling. Expectations had dipped into minority government territory. Then, by Friday evening, Labor insiders evidently had a spring back in their step and were briefing journalists on a seat haul in the high seventies to low eighties. (Labor has ended up with sixty-seven or sixty-eight.)
The Coalition, meanwhile, was happy to confide doom and gloom. The polling was generally pessimistic, with the odd ray of light.
Overall it’s consistent with a belief on the part of Labor that the mere appearance of being headed for victory will generate something called “momentum” and become self-fulfilling. Perhaps the Coalition, by contrast, reckoned that being seen as the underdog was no bad thing.
Herding? Many brainy people, including Nobel laureate Brian Schmidt, detect something fishy in the pollsters’ two-party-preferred numbers across the campaign: they’re all so similar, sitting around 51 and 52 per cent. They calculate, to varying degrees, the chances of the campaign polls being so close together as astronomically remote.
See, for example, Kevin Bonham.
I humbly have several issues with this, and questions for the stats folks, which I’ll leave to another post. (Mostly it’s to do with the fact the pollsters measure primary support and estimate two-party-preferred, and the primary numbers didn’t obviously herd.)
My two cents: To overcome the minor-party problem, the pollsters might like to try surveying twice as many people, reading out only the major parties to half the sample, and also naming the minor parties they deem worthy of inclusion for the other half. The differences between the two sets of results, at least over time, could be illuminating, at least giving an indication of the softness of major-party support vis-a-vis minor parties.
And they should keep a very, very close eye on Queensland. But they’ll no doubt be doing that anyway. •
