Wrong In Both Directions: Richo On Labor's Chances

 Advance Summary

1. An article by Graham Richardson in The Australian argues that Labor has no real chance of winning the next election because the swing required to win on the Mackerras Pendulum is too large.

2. However, the comparisons Richardson makes disregard the relationship between a prior election result and the swing to (or against) Labor at the next election.

3. Indeed there is actually little if any historic relationship between the 2PP result at one federal election and the next.

4. Considering only the problem of gaining a swing of the size suggested by the pendulum, electoral history gives Labor a substantial (28%) chance of winning.

5. However this chance may be reduced if swings against first-term governments are usually smaller than a given base vote implies.

6. Furthermore, because Richardson relies on the Mackerras Pendulum, which is not adjusted for the personal votes of new sitting members, the target swing for Labor is probably higher than he thinks it is, and this reduces Labor's chances.

7. While it is therefore unlikely on paper that Labor will win the next election, it is unrealistic to assess it as a "virtually impossible" event on the basis of past electoral data.

=====================================================================



In today's Australian, Labor number's man turned pundit Graham Richardson has used the current Mackerras Pendulum to deduce that the Coalition has the next Australian federal election  more or less already in the bag.  

Richardson's article gets off to an embarrasing start when, having used said pendulum to set 4.1% as the target for victory, it asserts Labor has achieved "a swing of this magnitude on only four occasions" since World War II, and then goes on to list five of them.  Richo also claims:

"To win the next election Labor needs a swing of 4.1 per cent and that swing will have to be perfectly uniform, which it never is."

What he should have said:

"To win the next election Labor needs a swing of 4.1 per cent assuming that that swing is perfectly uniform, which it never is."

Now, saying that Labor has only achieved a swing this large four five times in 26 elections is hardly crushing evidence that it can't be done, only that it is relatively unlikely.   The natural response to a historic 19% chance of a certain swing (even if one has miscounted it as only a 15% chance) is to argue that victory is a slim but realistic chance, and that Labor should do their best to take that chance, while at the same time realising that success after a second or third Coalition term is more historically likely.  It's the kind of prospect that justifies avoiding taking big risks that only might pay off, but it's a big enough chance to not put all your eggs in the two-term-strategy basket.

Against that sort of implied assessment, this is meant to be the kicker:

"In trying to work out just what Labor's chances are in 2016, it is worth noting that even Bob Hawke, arguably the most popular politician of the past three of four decades, could manage only a 3.6 per cent swing in 1983 against Malcolm Fraser. He won three more elections, but the Labor vote declined each time and in 1990 Hawke just failed to win a majority of the two-party-preferred vote.

To do better than Hawke, Shorten must know, is an incredibly big ask. There has to be a two-term strategy for Labor. One term is virtually impossible."

This is what's wrong with this argument in one graph:

This graph shows the two-party swing to Labor at a given federal election as a function of the ALP's two-party preferred result at the previous election, for all federal elections since World War II.  I've subtracted 50 from the Labor 2PP just to make the axes neater with my prehistoric software; the good results at the previous election are on the right and the bad ones on the left.  There is a very strong inverse correlation between how Labor did at the previous election and the swing at the next one - if they did badly, they usually bounce back, if they did well, they have always gone backwards.

There's actually some asymmetry in the pattern in that Labor has not (since WWII) improved from a 2PP of much greater than 50% (the best 2PP they picked up a further swing from was the 50.2% in 1969) while the Coalition has increased already winning margins three times - in 1958, 1966 and 2004.

A simple linear regression gives:

Swing to Labor = 38.222-0.7781*Previous 2PP +/-2.73

with 46% of variation in results explained solely by this tendency for big wins by either side to be followed by readjustment towards somewhere around the middle.

The Hawke result in 1983 is the red dot just below the 4 on the y-axis.  By the historical standards of the sort of swing Labor gets from a given 2PP, 1983 is actually among the party's best results.  Off a 2PP of 49.2% from the 1980 election, the median historical result would have been for Labor to go nowhere and lose again.  The 3.9% swing the party actually got from that base vote was, all else being equal, only something that would happen about 8% of the time.  All else was not equal; Hawke was a very popular leader facing a third-term government that had made a farcical mess of the Australian economy.

If the swing he recorded was exceptional in raw terms and was so as a result of his great popularity, then such a swing would be a very rare event in elections since World War II.  In fact it isn't; 40% of the rest of the elections saw larger swings than Hawke recorded (half of those in each direction.)  If getting such swings is only for the mega-popular then how come relative plodders like Calwell, Hayden and Beazley all did it?

From the 2013 election result of 46.5%, Labor would be historically expected to get a swing back of about 2%.  Thus, by historic standards Labor should still lose, but the 4.1% swing Richo reckons Labor needs is much closer to a projected 2% swing than Hawke's 3.6% was to a projection of -0.1%.  When we consider that big swings to Labor normally follow the party's bad results rather than its mediocre ones, and that 2013 was a bad result, the probability of a swing of the size Richo mentions rises (all else being equal) to 28%.  Not that different to bookies' odds that currently loosely imply something around 33%, and very different to Richo's assessment of more or less nil.

Here is another graph I constructed in the course of looking into Richo's claims, and it surprised me I had never done this one before.

This one shows how Labor goes in 2PP terms at a given election in 2PP terms as a function of how well or badly they did at the last one.  There's actually almost no relationship at all. A very weakly positive relationship with less than 7% of variation explained probably has not much meaning to it other than that monster 2PP wins aren't followed by monster 2PP losses, and vice versa.

So if the flow of 2PP outcomes from election to election is actually mostly noise and nearly random, why are first-term governments since World War II batting six out of six?  It really has not as much to do with the size of the swing required to win back office, and more to do with a couple of other things.

The first one is that it could be that first-term governments are more resilient in terms of the two-party preferred vote alone.  Comparing the performance of first-term governments to the projection above, Fraser, Whitlam and Hawke outperformed the projection by 2.65 points, 1.78 points and 1.77 points respectively.  Menzies and Rudd/Gillard (!) outperformed it marginally and only Howard, who underperformed by 2.48 points, did worse than the projection.

The projection says all three first-term ALP governments should have only got 50-50ish results at their first re-election attempt; in fact two won comfortably.  The projection suggests that on average, two or three out of the six first-term governments should have recorded a sub-50% 2PP.  In fact, only one did.  From such a small sample, I can't say if first-term governments are really more resistant to 2PP swings, but it makes sense that they could be, and we cannot rule it out.  If it's real, then that would weaken Labor's expected position by 0.7%, again, all else being equal, knocking the chance of a 2PP of 50.6%+ down to about 16%.

The second factor (and this one is certainly real) is that first-term governments have sophomore effects on their side.  Although one of the six first-term governments in the sample lost the 2PP vote, that government won re-election pretty easily.  The 1998 election win by John Howard was a serious (if rare) failure of the pendulum as a tool for predicting the impact of a given swing.  A 51% two-party preferred result for Labor "should" have just got it over the line but instead delivered just 67 seats to the Coalition's 80.  Labor failed to win twelve seats that were on margins under the swing obtained and won only two that were above it (not counting the anomolous Oxley, which was borrowed by Pauline Hanson but had been redistributed to notionally Labor anyway).  Of the twelve key marginals Labor failed to win, nine had a first-term Coalition incumbent, and seven of these incumbents had been elected in place of an ALP member in 1996.  Without this factor Howard would have probably lost the election.

While I have dismissed Richardson's argument above on the grounds that a 4.1% swing is actually not all that historically unlikely, a factor that cuts the other way is that a 4.1% swing would probably not be enough.  Looking at the ten electorates that the Coalition holds by between 2.5% and 4.1%, eight were won from Labor at the most recent election, while in a ninth the sitting member retired and was replaced.  Adjusting the margins in these seats to take sophomore effects into account moves the uniform swing needed for victory up to about 4.6%, or a Labor 2PP of about 51.1%, not 50.6. And since Labor would be getting over the line by less than 0.3% in four or five seats to pull this off assuming uniform swing, even 51.1% is probably a conservative estimate, by a tenth or two of a point.  Mid-term redistributions may change that.

What Chance Labor Really?

The difference between what the pendulum reckons Labor needs (50.6%) and what they are actually likely to need may seem small but the impact on the implied chance of victory according to the historical projection is rather high.  It's enough alone to knock that in-theory 28% estimate above down to about 17%.  If we also assume that first-term governments really are resilient to 2PP swings then this could go down even to 11%, but we don't have enough data to know that.  Perhaps around 14% might have been a fair starting estimate of Labor's chances based on projections from historic data, ignoring personality factors, issues, the economy and everything else that will come into play in the next year or two.  This is certainly slim, but it's not "virtually impossible", and least of all for the reasons advanced by Richardson.

Since then we have seen the Coalition poll some remarkably poor early polling.  Incumbent governments that poll poorly early in their terms have only a 50% record from six re-election attempts, but that isn't much of a sample size.  Then again, the evidence for first-term resistance against 2PP swings to the opposition for new governments is no better.

There are also valid arguments that Labor's bad performance at the last election (even by Labor standards) was caused largely by factors that no longer apply, and therefore that  Labor will get back some of the swing for free.  All this in my view justifies estimating Labor's chances at least as more than 20%, but I am cautious at this stage about going higher than that.

Given that betting markets give Labor about a 33% implied chance, it's possible that the markets are being a shade generous.  But if so, this could well be because a Coalition at short odds is an unappealling investment for a gambler who considers themselves informed, given that they can make about half the profit for no real risk by investing the money instead.  Alternatively, the markets could be right in placing more predictive weight on the early subjective evidence that this government might not be up to much than I am doing at the moment.

By the way you can see Peter Brent also refuting Richardson here.  Having written my article without having seen his I'm pleased how little they overlap.

---------------------------------------------------------------------------------------------------------------

Note re The Australian and paywalling:   If an abbreviated version of an article comes up when you click on links to their site, copy either the title or the URL and search for what you have copied on Google or Google News.  This frequently brings up a link to the full article.