Do Greens Do Badly At By-Elections When Both Major Parties Run?

(This article has been graded Wonk Factor 3/5.  It contains plenty of number-crunching and the odd statistical concept.)

---

Today's article is a test of a hypothesis I thought was worth looking at.  In the recent Fadden federal by-election, the Greens were among the obvious losers, copping a 4.56% swing against them and falling to fifth on primaries behind Legalise Cannabis.  Then came the Rockingham state by-election in which the Greens were again smoked by the dope party despite having outpolled them at the previous upper house contest for the seat. The Greens did pick up a swing there (currently standing at 1.7%) but that was nothing but crumbs given that Mark McGowan's departure left a 33.4% swing for other parties to feast on.  On the other hand, there was a significant local-government based independent, Hayley Edwards, running.

Social media has been awash with Labor stans and the occasional op ed or media hack claiming that one poor result and one ambiguous result are clearly all about the party's stance on the Housing Australia Future Fund bill and are a portent of incandescent doom for the Greens at the next Senate election!  Given the variability of by-election outcomes and the fact that the Green vote is stable at its election level in national polling, those involved could as always improve the standard of Twitter psephology by desisting from this untestable game and deleting their accounts.   

On the other hand, I have seen a theory that the Greens will often do badly in by-elections because their by-election campaigns tend to be token attempts in seats they can't win and without the accompanying upper-house focused campaigns to drive up the vote.  So is there any truth in this overall, or is it really just the case that the Greens vote easily goes soft when there is even modest new competition?

Well as far as I can tell, it's the latter.  The average Greens' performance in by-elections through their history has been only insignificantly down on their general election results.  

In 2020 Greens leader Adam Bandt falsely claimed that "By-elections are always difficult for us". His Sky News interview was deservedly ratioed on Twitter for claiming that Eden-Monaro 2020 where his party dropped 3.1% to riffraff was a message to the government (which had even got a rare swing in its favour!) about climate change.    One obvious counter to Bandt's claim was to point to cases where the Greens have gained seats at by-elections (Cunningham federal 2002, Fremantle WA 2009 and Northcote Vic 2017, all of which returned to their usual owners at the next general election).  But these were all cases where the Coalition did not field a candidate. 

Similarly there have been cases like Mayo 2008 where in the absence of a Labor candidate, the Greens picked up large primary vote swings and were surprisingly competitive for the seat.  But these are irrelevant: they are cases where the Greens were likely to make a bigger effort, and even in similar cases where the Greens were uncompetitive there is no way to benchmark them.  The relevant test data set is by-elections where the major parties, Labor and Greens all recontested.

Test criteria

For this article I identified a total of 64 such by-elections at federal, state and territory level.  My first condition for including a by-election was that Labor, the Greens and at least one Coalition partner must have contested both the general election in the seat and the by-election.   

I required that the party name of the Greens candidate must in each case have included the word "Green" or "Greens"; therefore I did not include the Fremantle by-election 1990 where Wikipedia (not unreasonably given that the previous candidate was later a Greens Senator) gives the swing to the Greens from a thing called "Alternative Coalition".  I accepted cases where the number of Coalition parties contesting went from two to one or one to two as I doubted this would have much impact on the Green vote.

I also did my best to identify some where I thought the Greens might be especially likely to do well or badly:

A major retiring player (MRP) case is one where (i) the departing member was a Labor Prime Minister, Premier or Chief Minister or (ii) at the general election there was a compatible non-recontesting independent or third party who polled more than 10%.

A major new player (MNP) case is one where at the by-election there was a compatible independent or third party who polled more than 10%.

Under the heading "compatible" I exclude obviously right-wing or anti-Green outfits such as One Nation and the Shooters, Fishers and Farmers (though there is probably some minor competition between the Greens and even these) and also anyone with an obvious current link to such a party at the time of the by-election (such as in one case a UAP-supported independent).  I include, for instance, Tony Windsor, Joe McGirr and at party level the Territory Alliance.  All these could be seen as conservative, but also as embracing environmental concerns.  

There's a case for classing Fremantle 1994 as a sort of MNP reverse of the MRP Labor rule given that former Premier Carmen Lawrence was the Labor candidate.  But outgoing Labor Minister John Dawkins would have had a high profile himself so that is not comparable to the replacement of a Premier or PM with somebody new to politics.

I pre-set these definitions before compiling data and there were some remarkable near misses - for instance Bundamba 2020 missed being a MRP by-election by four votes courtesy of Patricia Petersen's 9.99% at the general election. On the other hand Strathfield 2022 missed the MNP cut by 65 votes (Elizabeth Farrelly 9.85%).  There were also lots of cases where competing parties like the Democrats jumped in or out without disturbing the 10% bar.  But on the whole these rules should have generally picked up the most extreme cases, and have shown they are about equally common.

Two by-elections met both the MRP and MNP criteria and were treated as neither.  These were Fisher 2014 (where there were different high-polling independents in the two elections, albeit with a lower independent vote in the by-election) and Rockingham 2023 (a by-election for a former Labor Premier but with an independent making double figures).  

I did not treat Coalition Premier or PM resignations as MRP because I expected Coalition parties to compete less with the Greens.  Including Coalition Premiers and PMs would have disqualified Wentworth 2018 from the MNP category and would have also made Nedlands 2001 an MRP.  In Nedlands 2001 there was a large swing to the Greens in the absence of Richard Court, but the swing looks like it most likely came mostly off Liberals for Forests.

Results

The following table shows the average swing for the Greens at all the included by-elections, including crosstabs by state and category (MRP, MNP and neither).

Because of the tiny sample sizes nothing should be read into the variations by state and territory, other than that the Queensland and NT results when combined do nothing to nourish the "no upper house campaign" theory.  The data do however show that as expected the Greens tend to benefit when major opponents stop contesting (by an average of about 3% in this dataset) and to suffer when new major competition arrives (by about the same amount).  Nothing remotely surprising in all that, and it flows through to figures for swings for and against.  The Greens had swings to them in 7/9 MRP by-elections, 1/9 MNPs and 20/46 among the rest.

MRPs with swings against the Greens were Victoria Park WA 2007 with a field of 11 candidates compared to 5 at the general election, and Griffith federal 2014 where Terri Butler as a Left Labor, female and comparatively young Labor candidate was probably more appealing to Greens voters than former PM Rudd.  The sole MNP with a swing to the Greens in my dataset was Frome SA 2009 where despite the appearance of independent Geoff Brock, the Greens hauled their vote upwards from 3.80% to the dizzying heights of 3.86!

MNPs unsurprisingly account for the worst two swings in the sample (Surfers Paradise 2001 and Wentworth 2018).  The worst non-MNP result is Lytton Qld 1996 with a swing of 5.80%.  However that was with the Democrats and one Nigel Freemarijuana entering the field and taking 9.3% between them, so that was a case with MNP-like suppression of the Green vote.  Fadden 2023 (-4.56%) is the second worst non-MNP result in the dataset, and again featured competition of the leafy variety.  Both these cases saw a large increase in candidates (3 to 9 and 7 to 13 respectively).  Two other over-4% non-MNP swings against the Greens were Pittwater 1996 (large swing to a high calibre Democrats candidate Vicki Dimond) and Miranda 2013 (former 12-year Labor incumbent Barry Collier returning to the fray after retiring at the previous election).  The next biggest is the 3.11% at Eden-Monaro 2020, which featured both a large increase in candidate numbers (8 to 14) and also an unusual Labor candidate in Kristy McBain, a local Mayor with a limited party past, who was highly likely to appeal to Greens voters.

The Greens' two biggest swings were also unsurprisingly in MNPs where Labor heavy hitters left (Fannie Bay 2022 +9.7% and Lakemba 2008 +8.40) followed by the arguable MNP case of Nedlands mentioned above.  The best Greens result without such a departure is a 6.57% swing in Penrith 2010 at the expense of NSW Labor's collapsing vote through that term.  The next best is Ryde 2008 (3.34%) in the same term and there are no other non-MNP cases above +3%.

As a summary of the above, where there have been unusually large swings to and from the Greens in by-elections, there have always been plausible reasons.  Those results may still have been good or bad independent of those reasons, but not by as much as the swing figures suggest.

Overall Finding

The overall result is an average swing of 0.2% (standard deviation 3.19%, standard error on estimate of mean 0.40%) against the Greens in 64 by-elections, with 28 swings to the party and 36 swings against.  Excluding MNPs and MRPs the result is an average swing of 0.33% against (SD 2.66, SE 0.39).  So there is not even statistically reliable evidence that the Greens perform badly in by-elections across what is now quite a large sample.  This article has identified many more factors that could be controlled to try to work out what happens to the Green vote in by-elections when there are no unusual candidate or party factors but that way lies overfitting, not to mention madness.  Overall the Greens' results at by-elections and general elections have been more or less the same, provided that both major parties recontest.  

(NB This article has been researched and written entirely in about four hours and may contain some errors, but any errors are unlikely to affect the overall findings.  Any mistakes that are spotted will be corrected.)