Senate Reform Performance Review Part 1

The results of the Senate election are all in and so it is timely to review how the new Senate system performed in full detail.  No system should be expected to perform flawlessly on its first attempt.  However, the new system has generally exceeded the expectations of its most ardent supporters (save any who supported it for the wrong reasons) and made its opponents' pre-election arguments look very silly indeed.

Two major models of reform were canvassed in the Senate reform debate that ran through the last term of parliament.  The original JSCEM model released in 2014 allowed for fully optional preferencing above the line with semi-optional preferencing (six squares for a half-Senate election, twelve for a double dissolution) below the line.  The revised model released in 2016 initially allowed for semi-optional preferencing (1-6) above the line, but essentially maintained compulsory full preferencing if voting below the line.  After many complaints from the psephosphere (and I especially give credit to Michael Maley and Antony Green here) the final version as amended allowed semi-optional preferencing (1-12) below the line as well.  This change, in the end, allowed Tasmanian voters to overturn the contentious demotion of a sitting high-profile Senator.



The final savings provisions were that any vote including a single 1 above the line would be formal (even if it included two 2s, or only a 1 and nothing else), and that any below-the-line vote would be formal if the voter correctly numbered from 1-6 with each of these numbers once and once only.

There was relatively little difference in the objections raised to the two models, and indeed many objectors to the first model continued making the same objections to the second as if nothing had changed.  Some of the objections made were:

* that the system would advantage the Coalition by making it easy for it to win a majority
* that the system would allow for a permanent Coalition blocking majority (50% of all seats)
* that the system would prevent micro-parties from winning, thereby "disenfranchising" "25%" of all voters who voted for such parties
* that very large numbers of voters would just vote 1
* that very large numbers of votes (estimates were frequently well into double digit percentages) would exhaust
* that the system would be effectively first past the post on quota remainders
* that there would be a massive increase in informal voting (perhaps even to 10%)
* that the AEC would be unable to educate voters or complete the count in time
* that the system would advantage One Nation because votes would exhaust thereby preventing other parties catching them on preferences

Every one of these objections has proved false by a large margin.  Later on I will be running through some of the specific false predictions (possibly in the end-of-year Ehrlich Awards, for which dumb claims about Senate reform may just about scoop the pool for this year, but possibly earlier).  Because of the amount of material I am posting here I am splitting this article into two parts.  This part covers proportionality, half-Senate projections, winning vote shares, the impact of preferencing, just-vote-1 rates and exhaust.  

Part 2 covers informal votes, below the line rates, how-to-vote cards, Labor's attempt to blame the new system for the election of four One Nation senators after first insisting the new system would prevent all minor parties except Greens and NXT from winning, the two most unusual victories (Lisa Singh and Malcolm Roberts), and areas for improvement.  It also discusses the impact of the fixed quota and the question of verification, and I'm open to suggestions for other areas worth canvassing.  (Please bear in mind that my computing skills and facilities are limited, and my time to improve such even more so, so I can't rerun elections to test scenarios unless the answers are pretty obvious.)

I have already extensively discussed donkey-voting, proximity preferencing, the abject failure of most parties' how-to-vote cards, and preferencing patterns by state on the Button Press Week thread. Also, lots of Tasmanian-specific stats can be found on the Tasmanian button press thread.

In passing, there has been speculation that if Rodney Culleton (One Nation) is declared ineligible, his seat might be won by a different party.  This is nonsense.  Culleton won with a massive margin of over 37,000 votes and nearly all his votes were above the line.  Rerun the count without him and One Nation lose a few thousand votes becuase of different BTL patterns, but they would certainly still win a seat (as confirmed by Grahame Bowland.)  Things might get interesting only if it turned out that all the WA One Nation candidates were ineligible.

Proportionality

The following chart shows the share of primary votes won by each party and the number and share of Senate seats won by each party.  I will argue below that there is a better way to do this, but this is the one everyone else is looking at, so let's talk about it first:

There is a general tendency here for the large parties to get a slightly higher seat share than their vote share.  Not all parties can be represented and the preferences of the many parties that were not close to winning a single seat had to be distributed among all the others.  Had these preferences gone mainly to other micro-parties, it would be fair for micro-parties to punch above their weight in seat share terms.  But in fact, generally, the bigger a party's primary vote share, the better it was at gaining preferences from micro-parties.

So the results above are remarkably close to proportional - given the breakup into states - in a system that considers both primary vote and preferences. There is virtually no difference in the ratio of seats to votes between the two major parties.  Every party with significantly more than 1/76th of the vote has won at least 1/76 seats.  Of two parties on more or less that ratio, one won a seat and the other didn't.  Of parties far below that ratio, only one won a seat.

The result is, however, somewhat different to what you would get by electing the whole Senate nationwide with a single PR quota of 1/77th.  That would most likely have resulted in three fewer Coalition, three fewer ALP, two fewer Greens and one more seat each for a bunch of micro-parties (maybe LDP, SFF, AJP, CDP, Sex, ALA, DLP and (shudder) Health Australia), bringing the total non-Green crossbench up to 19 seats held by 13 different parties.  It's stuff like this that drove much of Europe to primary-vote threshholds.

(A note on the Sex Party and HEMP, which also applies to Science/Cyclists.  Sex Party and HEMP ran joint tickets in several states, but also ran separately in NSW.  The AEC data treats the joint ticket as a separate entity from Sex Party and HEMP individually.  Some people think they should be treated as a single party like the Coalition, in which case they have 1.44% of the national vote and no seats.  I don't agree with this, because where the two parties did run separately the preference flows between them were weak, and also they do not have any kind of evident coalition status beyond that they would run joint tickets and cross-preference. Effectively, these joint tickets were just an agreement that in different states one of these parties would endorse the other and abandon all hope of a seat for itself.  I have therefore just given each party the votes actually credited to its candidates on the joint tickets - so for WA and SA most of the Sex/HEMP votes are allocated to HEMP, and for Queensland and Tasmania most of them are allocated to Sex Party.)

Proportionality: a better way to look at it

To get serious about assessing how proportional the outcome is, we need to bear in mind that the Senate isn't meant to be a national proportional representation house.  It's meant to provide proportional representation within each state, but the requirement that the States have the same number of Senators as each other makes it easy for, for instance, Jacqui Lambie Network to win a Senate seat with a small national vote share.  It also means that a party polling one and a bit percent everywhere isn't going to win a seat in any state.  Also the Territories only elect two Senators each, which favours the major parties.

So a better way to judge proportionality is by each party's average of vote shares by state, compared to its share of seats won in the states.  And that's what that looks like:

(Note: "The Nationals" = WA Nationals, not formally part of Coalition.)

We now have a striking result: out of the 72 seats, every party that has averaged over 1/73rd of the vote per state has won at least one seat, and every party that has averaged less has won none.   NXT has won three seats instead of a proportional four (mainly because outside SA its 1.8% vote was too evenly spread) and One Nation has won four instead of three (on superior preferencing performance by collecting votes from a lot of right-micros that won nothing).

The same slight skew to the big three and against parties on 1% or so of the vote is seen as above, with Labor, the Liberals and the Greens each getting three more than proportional, partly because big parties outperform tiny ones on preferences, and partly because the Greens benefited from having three good states and three weak states rather than getting just over a quota across the board.  Again, I argue that this is a good thing, as a parliament that lets in every movement that can scrape 1% is bound to be extremely messy.  The seat result seems to have slightly favoured the left, if anything, but the left were very lucky not to drop a seat to One Nation in Tasmania, which would have made it pretty much even.  For most micro-parties that polled below 1%, especially on the right, the message is clear: start merging or stay in the bin.

Half-Senate

Of course some of those who made false predictions about the impact of Senate reform were basing them on the normal cycle of half-Senate elections, rather than a double-dissolution (which is a fairly rare event).  But had this election been a half-Senate election, the Coalition wouldn't have got anywhere near a blocking majority either.  Based on the results the most likely seat breakdown with the same candidates standing would have been Coalition 17 Labor 14 Green 4 NXT 2 and one each for Hanson, Lambie and Hinch - a more or less dead even and extremely fair left-right split from what was a close election, and with three non-"big four" crossbenchers still elected.  However, had 2016 been a half-Senate election, Lambie herself would not have been up for election, and it's doubtful her brand would have transferred to a candidate endorsed by her if she was not on the ballot.  So more likely, 15 for Labor with Lisa Singh re-elected.  (Note that the ongoing target for a blocking majority for either side is 20 seats per half-Senate election, not 19, because of the cycling of the Territory Senators.)

Compared to a half-Senate election with similar vote shares then, the Coalition is down one seat each to Labor and the Greens as a result of "throwing away" its 2013 result.  This means it now needs 9/11 non-Green crossbenchers to pass bills instead of 7/11.  The difference in the crossbench is that three One Nation Senators have taken the places that would have been kept by Wang, Lazarus and Muir.  On the other hand, a half-Senate election would have resulted in seven non-Green crossbenchers elected in 2013 staying there til mid-2020, with five of those angry and perhaps obstructive over Senate reform.  As it is, either six or seven non-Green crossbenchers will see their terms end in mid-2019, and three of the crossbenchers who opposed reform are gone. Being re-elected may give the rest some perspective.  So in a simple numbers-based sense it's not so clear that one path was better than the other - the real concern for the government will be the protectionist leanings of the new crossbench and the extent to which it may have to change policy course.

The above also assumes that had the Government gone to a half-Senate election, with no plan to even try to stand up to the Senate, its vote would have held up.  Based on what we now know, it may have simply lost.

Winning Vote Shares

I wrote the following in February:

Under the proposed new system, as with the JSCEM model, minor parties with substantial support would still be potent forces at double-dissolutions.  With a quota of just 7.7% and relatively strong preferencing, it's entirely plausible parties in a state or two would win seats at double-dissolutions with, say, 2.5-4% primary votes.  A double-dissolution under the proposed system would return a crossbench with a more democratic mandate, but whether it would return a much smaller crossbench is not so clear - especially not with the prospect of a three or four seat Xenophon block.  What it would return is a crossbench elected under the new system and therefore less likely to be uncooperative with government legislation on account of the system change. 

This sort of point did not stop several Labor MPs continuing to treat the reforms as certain to destroy all micro-party winning chances, even in the knowledge that a double dissolution was coming.  An increase in the "others" vote above what I expected at the time made things just slightly easier for micros than the above forecast, and what we finished up with was two wins from just above 4% (One Nation in NSW and WA) and two wins from around 3% (Bob Day 2.87% and David Leyonhjelm 3.09% - both sitting Senators).  There were no cases of a party polling over 3% in a state and winning no seats, but the LDP in NSW was actually the only case of a party polling between 3 and 4% at all.

Alongside Day's win, in seven cases parties polled between 2 and 3% and lost, and every party that polled below 2% in a state lost.  In Victoria there was a small surprise - Family First (1.14%) overtook five parties that had between 1.55% and 1.81% each and finished 13th, missing election by 0.1 quotas (1.3%). So the suggestion there is that about 2.3% with some of it coming from the Coalition would have been enough to win a seat.

This was mostly consistent with what I found in simulations of past results - it was fairly common for a party to win around the 0.4 quota mark (3%) but that wins from much below that were very rare.

In the 18 cases where a party had a surplus after polling one or more quotas, a surplus exceeding 0.3 of a quota (2.3%) was enough in 10 of 11 cases. Labor lost in South Australia with a notional .551 quota (4.24%) surplus. However in that case Labor actually shed .056 of a quota (0.43%) from a massive leak of below-the-line votes from Penny Wong, chiefly to Sarah Hanson-Young, and thus weren't anywhere near as far ahead of the chasing pack as they appeared.

A surplus below 0.3 of a quota (2.3%) was not enough in 6 out of 7 cases - the exception being One Nation winning two in Queensland with a .194 quota (1.49%) surplus, of which plenty more to say later.

In general then, the break-even point seems lower for the extra candidates of parties with one or more quotas than it does for micro-parties with less than a quota.  This is entirely fair, because parties with more primary votes, on average, get more preferences.

First Past The Post?

After the Tasmanian result, in which the parties with the largest remainders had - after everything - won the remaining seats, there was some speculation that other states would follow this pattern too.  Two of them didn't.  In Victoria, NSW and WA the largest remainders did win, but in Victoria one party overtook five others before losing by a modest amount (as discussed above).  In South Australia Bob Day (who had argued this effective-first-past-the-post silliness before the High Court without success) ironically disproved his own claims by overtaking One Nation and Labor and winning. (Day actually predicted he would win, but the main reason he won was Liberal card preferences, not the micro-party card preferences he refers to, as they flowed very weakly.)  In Queensland, One Nation overtook five micros to win, and Family First (which was one of these) overtook two other micros to again finish 13th.  Furthermore in Western Australia the Greens greatly stretched an initially small lead over the WA Nationals, and hence would have easily won had they been slightly behind.

The evidence is that the impact of preferencing is alive and well - and that impact comes from voter choice, and not from bogus backroom deals that created completely fake near-100% preference flows between parties in the old system.  A lot of credit for the effectiveness of preferencing belongs to those in the parliament who pushed for and agreed to a soft requirement (with savings provisions) to number six boxes above the line.

Just-Vote-1

There was a widespread expectation that a great number of voters - perhaps most, certainly well into double digits - would just vote 1 for a single party above the line, with an increased chance of their vote exhausting.  This was more of a concern with the original JSCEM proposal.  Objectors underestimated how readily voters would simply follow the instructions under the new system.  The just-vote-1 rate (1 above the line then vote exhausts) was just 3.0% nationally, and that includes voters who, for instance, voted 1 for a party above the line then started numbering boxes below the line (I saw a few of these).

Here is a breakdown of the just-vote-1 ATL rate per state, including the rate for each of the major parties.  These rates are out of all votes (including below-the-lines), though outside of Tasmania the difference is small.

Overall the just-1 rate is by far the highest in New South Wales.  A major cause of this is probably that NSW has optional preferential voting at both lower and upper house level in its state parliament, so voters are very used to just voting 1 for a party above the line.  Another factor would be the very high rates of voters from non-English speaking backgrounds.  Thus in each of Blaxland, Fowler, McMahon, Chifley and Watson there were more just-1 Labor ATLs than there were for all parties in all of Tasmania combined.  A third factor may have been prominent coverage by shockjock Ray Hadley of the just-vote-1 option.  I have seen many claims that Hadley actually advocated just-vote-1 though I have not yet found a primary source for them.  At the least he put a lot of attention on the fact that a just-1 ATL would still be counted.

The rate of all stopping before 6 in above-the-line voting was low.  According to David Barry's figures, NSW had the highest rate here by far (9.3%) with ACT on 4% and all the others between 5% and 6.3%.  This would have included voters whose vote stopped because they accidentally doubled or omitted numbers above the line.

Exhaust

Concern about exhaust rates was such that a short-lived "three million voices" campaign was set up to highlight fears that "millions" of votes for micro-parties would exhaust. Labor also got involved with claims that the system risked exhaust rates of "14-20 per cent", which I refuted at the time. In fact 5.1% of votes nationwide exhausted (just over seven hundred thousand) by the point at which the winners in the various states were determined.  A further 2.4% (just over three hundred thousand) exhausted during throws that did not determine the winners but that either determined election order for the final candidates, or else were simply required by the Act.

Exhaust rates by state (effective exhaust with total exhaust in brackets) were NSW 7.3% (9.2), Vic 5.2 (8.6), Queensland 4.2 (7.6), WA 3.6 (6.3), Tas 2.8 (2.8), SA 2.0 (2.0), ACT 0.0004 (0.0004 - very few preferences distributed) and NT 0 (0 - no distributions required.)

Exhaust was high in NSW not only because of the state's high just-vote-1 rate but also because the final seat race was between micro-parties.  A lot of voters would really not care between a Liberal Democrat and a Christian Democrat.  If they don't care then why make them preference?

This concludes part 1; part 2 is now open for business.

Added: 1987 Comparison

The purpose of my comments about proportionality was not to demonstrate that the new system was more proportional than the old one, but just to show that it was not as unproportional as some critics had implied (especially those who thought the Coalition would easily win a blocking majority).  Comparing proportionality with the old system is difficult because we should expect double-dissolutions to be more proportional than half-Senate elections, but there had not been a comparable double dissolution since 1987.  The world was very different back then, with nowhere near as many minor parties, and hence nowhere near as many opportunities for the system to produce a silly result.

Here, however, is what the state-based method of assessing proportionality comes up with for 1987:

The 1987 comparison is a bit messy because of the prevalence of independents who ran in one state only, whereas in the 2016 election Lambie, Hinch and Xenophon had very localised support but ran nationwide.  So it's not a problem that Jo Vallentine won a seat off an average of 0.81% of the state vote in all states, since she got all of this in one state.

The NDP's seat win though, by preference-snowball from a 1.53% primary in New South Wales, is a problem.  That seat should have gone to Labor (or just maybe, arguably, the CTA), as there is no way the NDP's .199 of a quota would overtake Labor's surplus .514 by voter choice.  Artificial 100% ticket preference flows from the Democrats and Greens meant that they did.

Labor were also diddled in Victoria.  Starting with 5.719 quotas to the Coalition's combined 5.365 (separate tickets), they ended up not getting six because artificial 100% ticket preference flows from DLP, CTA and various non-party ATL groups put the Nationals (0.738 quotas) over quota and their surplus elected the Liberals.

As a result Labor wound up with a state seat share below their average state vote share, while the other two large parties got the normal edge for larger parties. The new system would have produced a fairer result in 1987 with 34 Labor (including the two territory Senators), 33 Coalition and no seat for the NDP.  (It happens also that Labor won the national Senate vote, 42.8 to 41.9, in that year.)