Thursday, March 11, 2021

When Federal Polls Are 50-50, Oppositions Rarely Win

For nearly a year of federal polling, an unusual situation has existed.  The Prime Minister, Scott Morrison, has polled very high personal ratings, and large leads on Better Prime Minister scores, but the Coalition has struggled to build any meaningful two-party voting intention lead.  If polling is to be believed, an election held right now would be a close thing, though the government would probably survive.  

However, many observers just don't believe Labor is really at 50-50.  This is not surprising given that Newspoll was 3% out on the two-party preferred vote at the last election, so perhaps that's still the case and the government is 53-47 ahead and cruising.  (The history of polling failures elsewhere suggests probably not - every election cycle is different.)  But what I've noticed a fair bit is that people who don't believe the voting intention polling cite the PM's personal polling (or in some cases the primary votes) as evidence that the 2PP voting intention polling is wrong.   This doesn't make sense - why should voters give misleading answers on their current voting intention but not on what they think of the PM?  Or if sampling issues are causing the voting intention polling to be wrong, surely they would also drag the PM's personal ratings down (in which case he would actually be amazingly popular, and there would be a new issue of why he had only a modest 2PP lead instead of a massive one.)  


The other possibility (or it can be a bit of both) is that the voting intention polling might be telling us something about the personal approval ratings in the age of COVID, and that they may not mean what they normally do.   Labor voters might think the PM is doing a good job of handling the pandemic and therefore give him a tick without it affecting their vote.  It might be seen as unpatriotic to disapprove.  They might even be reluctant to suggest that at this particular time the Opposition Leader might do better.  These might just be times in which the normal dynamic involving Prime Ministerial approvals running hand in hand with voting intention breaks down.  It has done so before sometimes, especially in 2007.

Instead of rejecting the voting intention polling just because of a strong assumption that Labor cannot be competitive, here's another way of looking at it: even if the voting intention polling is accurate, 50-50 polling still may not be all that competitive.  The reason for this is that oppositions that have won have generally held large polling leads at some point in their term.  Almost every opposition has polled 50-50s or better at some stage of their term, but most federal oppositions lose.  

Polling history suggests that Labor probably won't win the next election unless it somehow takes a big polling lead at some stage.  Even if it does, that may or may not be enough.  There are, however, ways in which this pattern might not hold, which I discuss in the bottom.

A note that this article is rather number- and graph- heavy, but it doesn't contain a lot of complex maths, so it is probably only about a 2/5 on the Wonk Factor scale.  

Data Table

The following table shows the history of Opposition peak 2PP polling in each term since 1946-9.  The following fields are used:

High: The opposition's highest 2PP polling in the term.  

High (FL): The opposition's highest 2PP polling once both parties had their final leaders in place.

Diff: The opposition's performance on election day compared to their highest polling in the term.

FPE: The final poll error for the opposition on 2PP in the series used.  The reason for this is to check whether oppositions tend to overperform in leadup polling more when the polls turn out to be wrong at election time.

Adj Loss: The points lost by the opposition from its peak to the final result assuming that polling error was the same all term (which it probably wasn't).

Adj High: How high the opposition got if one assumes all the polls had the same error as the final reading in the series.

The data sources I've used are varied: my own 2PP calculations off old single Morgan Gallups prior to 1975, weighted average of consecutive Morgans for 1975-1985, a range of Newspoll rolling averages for 1986-2010 (last-election 2PPs substituted for 2004) except for my own cross-poll aggregations for 1990-3 and 1993-6, and my own aggregates since 2010.  It doesn't matter much what datasets are used as the broad picture is likely to be fairly similar.  The usual asterisk appears for 1975 on account of the Opposition Leader being appointed as Prime Minister before the election.  



What can be said from all this?

1. Nearly all oppositions hit the lead at some stage of the term

The last twelve completed terms in a row, and 23 of the last 28 completed terms, have seen an Opposition polling lead at some stage (as has the present term, with Labor reaching around 51.6% in my aggregate during early 2020.)  This is debatable for the 2007-10 term where not all aggregates show the Coalition reaching the lead.  Of the terms where an Opposition did not lead, 1983-4 was greatly shortened by an early election.  All the rest were in the early days of sparse polling, making it more likely that if an Opposition was ahead, the polls would miss it.  That said, some of the Opposition leads in polling (especially in the 1975-7 term) were products of polling error.

2. Oppositions that win have generally led big

The following graphs show the relationship between the peak polling 2PP for Oppositions and their final 2PP result:




The red dots are the Opposition victories.  Five of the seven Oppositions that won were at some stage at least 56-44 ahead.  The other two were more than fifty years ago, and sparse polling from those days means that they may have had bigger polling leads had polls been more regular.  Polling was so sparse in the 1946-9 term (the only one where an Opposition won without ever having a commanding lead) that there's a strong argument for not going that far back, but I've done it anyway for completeness.

3. Even Oppositions that have big leads often don't win

Six oppositions have at some stage had leads exceeding 54-46 yet failed to win.  These were:

* In the 1977-80 term Bill Hayden's ALP was widely expected to beat the Fraser Government, including in final polling, but fell short.  Based on the final polling errors it is questionable whether Labor were so far in front in the first place.

* In 1990-93 John Hewson's Coalition had massive leads over Labor as Bob Hawke's prime ministership approached its end, and large leads after Paul Keating replaced him for a while, but eventually lost.  This is the only case of the six that involved a Labor government.

* In 1998-2001 Kim Beazley's ALP had large leads over the Howard Government in early 2001, but the rest (a subsequent narrowing of the polls capped off by a surge from the Tampa incident and a much bigger surge from the September 11 attacks) is history.

* In 2013-6 Bill Shorten's ALP had massive leads at times over Tony Abbott's Coalition, but the Liberal Party replaced Abbott with Malcolm Turnbull who led them to victory.

* In 2016-9 Shorten's ALP again had massive leads over the Coalition immediately after the Liberals replaced Turnbull with Scott Morrison, but Morrison ended up winning.  This was another case where the final polls were wrong and it is questionable whether Labor's lead was ever as large as it seemed. As with 1990-3 this was a case where Opposition policy overreach was a factor.  

* And the biggest case of all - in the 1951-4 term the second-term Menzies government was at times as much as 39-61 behind the Evatt Labor opposition after the original "horror Budget" but still won (the Petroff affair was the icing on the cake but Menzies was probably winning anyway).  

4. Big government recoveries have been common in recent terms

Six of the last seven elections, and seven of the last ten, have exceeded the long term average government recovery from its worst polling position to the final result.


The big exception here was 2010, where the Opposition's polling peak occurred late in the term after a term of mostly huge government leads.  Kevin Rudd was removed as Prime Minister although his government was already back in the lead.  The history suggests Rudd would have been re-elected.  

1969 (Gorton) is notable as the only case where a government's final result was worse than its polling at any stage of the term, including final polls.

The increase over time in governments recovering isn't statistically significant, but the scarcity of polling in the early terms would be one explanation for it if it was.  The important point is that polling has been generally much more accurate since the mid-1980s bit this has not stopped governments from outperforming the worst points of their term by large margins.  If governments continue doing that, then oppositions will generally not win if their biggest lead in the term is narrow.

5. Polling error contributes to oppositions underperforming

This one is unsurprising.  If polling is wrong at election time, then it is more likely to have been wrong at any given time during the term, so if governments overperform their polling on election day, they may well have not been so far behind to start with.

The most dramatic example here is 1977 which appeared to be a cliffhanger in final Morgan polling but was actually a thumping win for Malcolm Fraser's government, so the appearance that the Whitlam opposition had closed the gap greatly from the 1975 thrashing as the election approached was actually total nonsense.  Many of the largest government comebacks, including 1993, 2019 and 1980, have been years when the polls underestimated the government at election time.  

Individual Newspolls

I also thought an interesting question to ask about all this was as follows: Suppose all that we knew about an Opposition was that it had polled a given 2PP in an unspecified Newspoll, how likely should we consider its chances of winning the next election?


Not too much should be read into this one - only three Oppositions have won in Newspoll history, so we don't know much about all possible ways that they might win.  But I think it's interesting all the same.

About a quarter of the 50-50 2PPs that have been polled in Newspoll by Oppositions since the mid-80s were by Oppositions that have gone on to win.  However, if the sample is limited to the Newspolls where the final leaders for both parties were in place, nearly all 50-50s or similar figures were polled by Oppositions that later lost.  (Those polled by eventual winners, for instance the sole case of Howard trailing Keating, were often rogue polls in the days when Newspoll still now and then threw those.) The cases of oppositions being around level in Newspoll and later winning mostly include Hewson and Downer (replaced by Howard) against Keating, Beazley (replaced by Rudd) against Howard, and early Gillard-Abbott polls post the 2010 election (Gillard was eventually replaced by Rudd).  It is not until the Opposition leads get up to 54-46 or 55-45 that they have won as often as lost, and not until they get up to the very high 50s that they've always won (a tiny sample).

How Does This Relate To Now?

The current parliamentary term is between 60%-80% over, depending on the timing of the next federal election.  There is still plenty of time for the Opposition to take the sort of lead taken by past recent winning Oppositions, so it's too early to use this historical pattern to say that the Opposition probably won't win.  What can be said is that a box generally ticked by winning Oppositions has not yet been ticked, and time is running down.  Given the historic pattern seen here, how might Labor win?  Here are some ways:

1. Labor flies under the radar

Usually an Opposition that wins will take a big lead at some point, but the only poll that counts is the one on the proverbial election day, and there's no reason why an Opposition can't poll its best result or something like it then, as a small minority of Oppositions have done.  However, historically three-quarters of Oppositions have shed at least two points from their polling peak to election day, so the chances of this one shouldn't be considered high.

2. The pandemic messes with the pattern

In this scenario the government is actually one of those governments that won an election too many a la Labor in 1993, and it would actually be on the way out by now but for a lift in voting intention provided by the COVID-19 pandemic and Australia's success in containing it.  Perhaps as COVID-19 assistance packages run out and people get immunised, politics returns to normal in early 2022 and the government slides to defeat.  I don't think this is a very convincing scenario, mainly because nothing in issue or attitude polling seems to provide much hint of it at this stage.

3. Labor gets a bigger lead

At some stage in the rest of this term something, or some combination of somethings, causes Labor to take a big lead in voting intention and to be perhaps taken more seriously as an alternative government as a result of that (not that that did Labor much good in the previous term).  If that happens then this article no longer applies, beyond that even a large lead does not guarantee victory.  

(Topical NB I have tried to avoid speculating about any possible current polling impact of the government's handling of the historic rape claims against Christian Porter, or any possible combined impact of that and the government's handling of the recent separate rape claims by Brittany Higgins and other former staffers.  The latter alone had no immediate impact on government polling, but there has since been some weakening in government attribute polling that may or may not be connected, as well as a dip for PM Morrison in the mysterious and under-documented Morning Consult approval tracker. The overall situation lacks any obvious parallel, let alone one that coincided with a pandemic.)

4. Labor finds an out-of-the-box leader

In the 2004-7 term Labor under Kim Beazley often led, but the leads were modest (up to 53-47).  This movie had been seen before in the 1996-8 and 1998-2001 terms without Labor actually winning.  The switch to Kevin Rudd in late 2006 resulted in a massive improvement in Labor's standing from which the Coalition could only partly recover.  However, Rudd was an exception.  The other cases of Opposition leader changes involve:

* Three cases where the Opposition had held a large lead under its initial leader, who was replaced after they fell behind (1974-5, 1993-6) or because it was feared they might not win or might only win narrowly (1980-3).   All these Oppositions won.  

* Six cases where the Opposition never held a large lead under either the original leader or the replacement.  All these Oppositions lost.  

I hope this article has shown that the widespread intuition that Labor probably won't win the next election is consistent with the recent polling, and there is no need for those driven by the former to invent ways in which the polls must still be wrong.  A bunch of 50-50 polls way out from an election just does not mean a 50-50 chance.  

5 comments:

  1. Just to point out a small error in your table, Kevin – you have Rudd losing in 2007.

    ReplyDelete
    Replies
    1. Ha, I redid that table so many times and that still got through. Thanks, fixed.

      Delete
  2. Excellent (if rather disheartening) article, Kevin, though one minor correction: Chifley was opposition leader in the 49-51 term, not Evatt.

    ReplyDelete
    Replies
    1. Ta, another gremlin removed. I was thinking of setting up an email service in the sidebar for readers to notify me of gremlins and typos without having to leave a comment, but so far I haven't found any way to do it that prevents people sending bogus emails without an account.

      Delete
  3. I agree with most of your analysis here; however, I do have a slight disagreement with your analysis in point 2 ("The pandemic messes with the pattern").

    I've been analysing polling from democracies around the world (I'll release this as part of a follow-up Pandemic Politics piece), but on average, incumbent governments' first-preference polling (for govts not facing re-election in 2020) jumped by 4.6% (+/- 5.1%) from Dec 2019 to Apr 2019, when the WHO declared COVID-19 a pandemic, with just 6 of 24 countries' incumbents seeing a decline in their first-preference vote (or 7 of 39, if you include govts with an election in 2020).

    In contrast, the Coalition's primary was pretty much the same in both Dec 2019 and Apr 2020 polling. If you assume that the COVID-19 boost is roughly the same everywhere, then the Coalition primary should look a lot worse once the COVID-19 bounce wears off (roughly in the high 30s, which coincidentally is only a couple points lower than where it is today).

    Of course, this isn't bulletproof by any measure. Still, I do think there's a valid argument to be made that given how small the Coalition's COVID-19 bounce was (relative to other incumbents around the world) and how close the opposition has been to it on a 2-party-preferred basis, it's plausible that once said bounce wears off the government will be in a fair bit of trouble electorally. Some other factors may or may not save it, but there's still a reasonable argument that the pandemic has messed with the pattern enough to keep the govt afloat - for now.

    ReplyDelete