Wonk Central: How Should Parties Count Member Ballots For Senate Tickets?

====================================================================

Advance Summary:

1. A recent Tasmanian ALP member/delegate ballot for Senate ticket preselections has raised the question of how member ballots for Senate tickets should be best conducted.

2. The use of standardised Hare-Clark (or other similar STV systems) for these ballots should be avoided, because such systems are designed to conduct elections in which all positions won have roughly equal value.

3. The use of standardised Hare-Clark can therefore mean that a minority-faction candidate gets either an easily winnable or an unwinnable (without a high below the line vote) position, depending on the way votes split up between other candidates.

4. Such a system therefore creates a big risk of tactical voting.

5. This article suggests an alternative, which is to set the quota off the number of positions on the ticket that are expected to be automatic wins, rather than off the number of candidates to be preselected from the cutup.

6. This article also discusses (scroll way down) the Tasmanian Greens' Senate preselection system.

========================================================================

This one's hugely technical, and is not aimed at a general audience.  Please don't say I didn't warn you.  There is just no other way.

A Senate Preselection Test Case

Here's a little electoral maths problem.

A party wants to conduct its Senate preselections not by using a preselection committee, but by a vote of its members.  It sends out a ballot paper to its financial members with a list of candidates, and gets back a large number of ballot papers where the voters have listed some or all of the candidates in order of preference.  The vote is for a four-candidate Senate ticket, where candidates 1 and 2 will more or less certainly get elected, candidate 3 is expected to have a 50-50 shot, and candidate 4 has no realistic chance.  The party has two factions, A and B.  Just over three-quarters of voters (let's say, exactly 77%) prefer every candidate of faction A to every candidate of faction B.  The remainder prefer the reverse.  It's pretty obvious that faction A should expect at least three of the top four positions, and that they should expect the top position to certainly be one of those.  But should a candidate from faction B be on the ticket, and if so, in what position?

As specified above the problem isn't too well defined.  The first issue is that we need to know whether the party sees representing all views proportionally as important, or doesn't care, or actually wants to let a minority faction punch above its weight.  Let's say the answer is the first. 

Where does this scenario come from?  Actually it's very loosely inspired by the recent Australian Labor Party Senate preselection in Tasmania, in which, based on a preferential ballot formed out of a mix of member votes and state conference delegate votes, the party elected a ticket with the Left's Anne Urquhart first, the Right's Helen Polley second, the Left's John Short third and leftish unaligned Senator Lisa Singh relegated to the "unwinnable" (except by BTL votes) fourth position.   I've changed numerous details in the above problem for the sake of keeping the maths simpler.

Before going further, I'll note that the viable debate in the Polley case was not whether Singh should have been higher up based on the known votes cast (she shouldn't) nor whether Singh's demise was primarily caused by factional deals (it wasn't).  Rather, it is whether the ticket should have been Urquhart-Polley-Short or Urquhart-Short-Polley.  There actually aren't enough figures in the public domain to usefully answer that question, but there are enough figures for the rank-and-file vote to question whether Polley's election in second was the will of the rank and file (which I did, see comments at the bottom). 

The ALP's Solution

As noted in the Singh-dumping article, the ALP appears to have selected four positions on the ticket (two more or less certain wins, one competitive and one "unwinnable") using Hare-Clark for four positions, with a quota of effectively 20% (in practice, fractionally over).

I noted in comments in the body of the article that this approach is risky.  Here's how the risk plays out in my scenario above, for various distributions of the votes among the A-faction.  I'm setting the quota at 20.1% and assuming in every case that the B-faction voters all vote first for candidate B1.  I'm assuming that every voter preferences remaining candidates on their ticket in order, and from the other ticket, just for nuisance value, in inverse order.

Case 1: Primary votes for faction A all go to candidate A1:

Count 1: Primaries counted, A1 elected first (77%), B1 elected second (23%)

Count 2: A1's surplus distributed (56.9%), A2 elected third (56.9%)

Count 3: A2's surplus distributed (36.8%), A3 elected fourth (36.8%)

Election finished: A1, B1, A2, A3

Case 2: Primary votes for faction A are split almost equally between two candidates:

Count 1: Primaries counted, A1 elected first (39%), A2 elected second (38%), B1 elected third (23%)

Count 2: A1's surplus distributed (18.9%), goes to A3

Count 4: A2's surplus distributed (17.8%), goes to A3, A3 elected fourth (36.8%)

Election finished: A1, A2, B1, A3

Case 3: Primary votes for faction A are split almost equally between three candidates:

Count 1: Primaries counted, A1 elected first (27%), A2 elected second (26%), A3 elected third (24%), B1 elected fourth (23%)

Election finished: A1, A2, A3, B1

So far it looks like splitting the vote between candidates from faction A helps them push B1 further down the list.  But this can be taken too far ...

Case 4: Primary votes for faction A are split almost equally between four candidates:

(A1, A2, A3, A4 have 22, 19, 19, 17%

Count 1: Primaries counted, B1 elected first (23%), A1 elected second (22%)

Count 2: B1's surplus distributed (2.9%), goes to B2

Count 3: A1's surplus distributed (1.9%), goes to A2, A2 elected

Count 4: A2's surplus distributed (.8%), goes to A3

All other candidates excluded with no votes

Count 5: B2 excluded (2.9%), goes to A4 ahead of A3 

A4 finishes with 19.9% to A3's 19.8%.

Election finished: B1, A1, A2, A4 

---------------------------------------------------------------------------------------------------------------------

Well, that's just a bit of a problem there.  In my example, the minority faction candidate always gets some position, since they always have a quota.  However, depending on the split-up of votes between the majority faction, they could get a position anywhere from first to fourth on the ticket, and the preferences of the B-ticket voters could also affect which A-ticket candidates get up.  That all creates a huge potential for tactical voting by the A-faction voters, and indeed it could be that the A-faction voters by honestly preferring A1 over the other A-candidates, are helping B1 come a questionable second.

If this is a problem, why doesn't it occur in normal Hare-Clark elections like for State Parliament? The answer is that in most Hare-Clark (or similar Single Transferrable Vote multi-member elections) the order candidates cross the line in is meaningless.  Topping the poll with three quotas has the same value - a four-year seat in parliament - as limping across in fifth place by 50 votes.   

In the Singh article I argued that the use of the standard Hare-Clark quota for four places is wrong, because one of the "winners" of the election (the person elected fourth) has actually won the booby prize - an "unwinnable" position.  Really, the election should be considered as being primarily for three positions, with the fourth of little value.  

If we instead set the quota for the above election at just over a quarter (for three winnable positions) the same thing happens in every scenario.  The A faction with just over three quotas takes positions 1, 2 and 3 and the B faction with just under a quota always ends up fourth.

But Is That Fair?

Well, actually, despite me previously suggesting 25% as a better quota than 20% (which it is), probably not.

The first thing to note here is that while Hare-Clark is famed as a proportional representation system, on an electorate-by-electorate basis it's debatable how well it scrubs up.  If we're electing precisely three candidates to winnable positions, it could be argued that a 2-1 split (66.6:33.3%) is much closer to my example's 77:23 than a 3-0 split (100:0), and therefore that faction B should be getting the third Senate seat and not the fourth.  

The second thing here is that the third Senate seat is actually not worth as much as the first two.  In my example, it's half a win.  So counting the first two seats as full wins, the third as half a win and the fourth as nothing, the various outcomes might be ranked as follows in terms of the division of seats:

B1 first or second:   A 60% (1.5/2.5) B 40% (1/2.5)

B1 third: A 80% (2/2.5), B 20% (.5/2.5)

B1 fourth: A 100% (2.5/2.5), B 0.

The outcome "B1 third" is fairest.

A Possible Solution

A solution that may work better is as follows:

Where the preselection has v votes, and is for p positions considered to be automatic wins and t total positions, conduct the election with the quota used for electing p positions, but actually elect t positions, with a full distribution to determine election order even after all the elected candidates are known.

Going back to my four hypothetical examples then, with a quota fractionally over a third (say, 33.5%) but electing four positions:

Case 1: Primary votes for faction A all go to candidate A1:

Count 1: Primaries counted, A1 elected first (77%), B1 has 23%

Count 2: A1 surplus (43.5%) distributed, A2 elected (43.5%)

Count 3: A2 surplus (10% distributed), A3 receives 10%

All candidates below A3 and B1 have no votes, B1 is third without quota, A3 is fourth without quota

Result: A1, A2, B1, A3

Case 2: Primary votes for faction A are split almost equally between two candidates:

Count 1: Primaries counted, A1 elected first (39%), A2 elected second (38%), B1 has 23%

Count 2: A1's surplus distributed (5.5%), goes to A3

All candidates below A3 and B1 have no votes, B1 third without quota, A3 fourth without quota

Election finished: A1, A2, B1, A3

Case 3: Primary votes for faction A are split almost equally between three candidates:

A1 has 27%, A2 26%, A3 24%, B1 23%

Count 1: Primaries counted, all other candidates excluded with no votes,B1 excluded fourth

Count 2: B1's preferences distributed to A3, A3 elected (47%)

Count 3: A3's surplus distributed (13.5%) to A2, A2 elected (39.5%)

Election finished: A3, A2, A1, B1

Case 4: Primary votes for faction A are split almost equally between four candidates:

(A1, A2, A3, A4 have 22, 19, 19, 17%, B1 23%)

All other candidates excluded with no votes

Count 1: Primaries counted, all other candidates excluded with no votes, A4 excluded fifth

Count 2: A4 preferences distributed to A1, A1 elected (39%)

Count 3: A1 surplus (5.5%) distributed to A2, A2 has 24.5%, A3 excluded fourth

Count 4: A3 preferences (19%) distributed to A2, A2 elected (43.5%), B1 left in third place with all others processed

Election finished: A1, A2, B1, A3

We can see here that in three of the four cases B1 came third.  In fact, it is impossible for B1 to do better than that, since B1 never reaches quota, and in all cases at least two quotas pool with the A-team.  Also, the scenario in which B1 comes fourth is pretty artificial.  Nearly always at least one of A1 and A2 will be a sitting Senator and such an even spread of A-team votes as in case 3 is very unlikely.  It's doubtful they could calibrate this sort of result to happen tactically if they tried.  

So for the assumed numbers in my example, this method has overall worked pretty well - it's not bulletproof, but it usually gets the right answer.

Of course, I could have used very different numbers:

* Both factions have more than 33% support

In this case, both will win one of the top two positions, which is hardly unfair.  It's possible that B1 will be top of the ballot, but this is irrelevant, since A1 and B1 will both win Senate seats anyway.  The most popular faction will get the third spot. The second most popular may get the fourth or may not, depending on the gap.

* The B-faction has very low support

If the B-faction has below a certain level of support then it cannot win the third position.  The A-team are guaranteed the top two if they have two quotas (=a bit over 66.67%) and also secure the third if they have more than half the remaining votes.  So if the B-faction has below about 16.5% of the vote, it won't win one of the top three places.  That's slightly disproportionate, but better than straight Hare-Clark for three places, in which below 25% means missing out, or even straight Hare-Clark for four (20%).

Really to test this system properly, it would be good to use an actual complete set of ballot papers, with complications like leakage, different preference orders in the same faction, candidates who aren't clearly of one faction, and all that jazz.  But I think this method (using a quota as if electing only the "automatic-win" positions, but actually electing all the positions) might be a good one.  

Getting Beyond Quota Fetish

In discussions of electoral system design, especially Hare-Clark or the Senate system, I often find that people do not think about the quota very well, and elevate it to something more important than it is. People get hung up about systems in which votes can exhaust because someone might get elected without quota.  The same people might regard an election for four places with a quota of 33+% as heresy.  They're wrong.

Hare-Clark recount systems have some especially odd quota fetishes.  For instance when a candidate was elected on primaries the first time around, their vote total is brought down to quota for the recount, a totally pointless practice that actually distorts the results very slightly through rounding.  If a candidate was elected without quota, then votes of remaining candidates are thrown to try to get them over quota, even if those candidates are the elected candidates' worst enemies, and even if this results in a vote getting smuggled into someone's recount when the voter put the recounted candidate dead last.  And so on.  

This is what the quota actually is: it's a shortcut.  The level it is set at does two things.  Firstly, it saves time in the distribution of preferences by declaring a candidate elected once there is no way that they could not be, no matter how any other votes flowed.  Secondly, it's set as low as possible not just for the first reason, but also so that as much of the candidate's vote as isn't needed flows on to someone else.  (More advanced systems like Meek continually update the assessment from previous counts as votes exhaust.)  The idea that there is something wrong if the last candidate gets elected without reaching their quota is irrelevant, so long as they deserved to be elected.

The quota works that way in Hare-Clark because what matters is whether a candidate gets elected, and not so much where in the order.  But when where in the order someone gets elected matters a great deal, then there's no reason not to mess with the usual quota.  

Tasmanian Greens Ballot System

The Tasmanian Greens also had a member ballot for a Senate preselection (and, indeed, a Senate casual vacancy) very recently.  This ballot, which I covered in much detail, was conducted with a widely and deservedly ridiculed level of secrecy*.  However, after much effort (most of which consisted of implying something unflattering so someone would have to deny it) I managed to find out what their ballot system was.

The Greens were selecting one position that entails an immediate Senate seat, and one that provides an outside chance of election to the Senate (mainly in the case of a double dissolution).  They conducted their election by using a single ballot paper with semi-optional preferencing and exclusion from the bottom up to find the winner (basically the same as Tasmania's Legislative Council system).  

After the first candidate, former state leader Nick McKim, was elected, they simply reused all the ballot papers with McKim removed, to find the candidate in second.  This didn't tell us much because apparently Rosalie Woodruff (who is almost certain to take McKim's state seat in Franklin) came second, meaning that the identity of the real Greens Senate #2 candidate most likely remains unknown for an unknown period.

However, the counting system they used is quite a good one, and also elegantly simple, if a party isn't very factional and preselections are more about candidate quality or recognition.  In the case of factions, such a system becomes too winner-take-all.  It is very much like the old Senate "slate" system from the first half of last century; if a faction gets 51% then they get the first position, and the second, and so on if there are more positions and it has more candidates to fill them.

If a party prefers more representation for dissenting positions within it, then an alternative is to simply give the #2 spot to whoever finished second after preferences in the original cut-up.  This way, if two factions each have at least a third of the vote, then both will at worst secure the #2 ticket position.  However, if everyone knows who is going to win, this system will result in tactical voting to attempt to influence the #2 position instead.

A middle option is to use the system I have suggested for the Labor case.  Hare-Clark (or a similar STV system) can be used to conduct an election for two positions with a quota of just over 50%.  This means that once the winner wins, their victory margin is distributed as a surplus, and can influence who finishes second.  (Here I would strongly recommend using a system that throws all a candidate's papers as a surplus, rather than last-bundle Hare-Clark).  This system gives some weight to factional issues, but reduces the problem of "wasted votes" for the winner.

Feedback Welcome

For the arcane few who managed to get this far, I welcome any further suggestions on good ways to conduct proportional rep elections in which the order of election is critically important.  (This is not an invitation to general debate about the merits of HC, Wright, Meek etc, or any Hare-quota-vs-Droop-quota stuff that fails to engage with that aspect.)  I think the solution I've come up with is a good one, but perhaps someone can punch a hole in it, or come up with a better one.

This is a new thing to think about, because the demand for member ballots in parties that have winning chances is pretty fresh.  Mistakes will be made by parties in the early days of it, and systems will change over time.

Feedback Received

There has already been one suggestion posted in comments.  Also on Twitter, Ben Raue has drawn my attention to the NSW Greens system.  The ballot is repeatedly recounted using a different quota - 1/2# to elect one candidate, 1/3 to elect two, 1/4 to elect three, 1/5 to elect four, and so on.  At each stage of recounting, the winner of the next spot on the ballot is the first candidate elected who was not already elected at a previous stage.  This has the advantage that a minority faction candidate can't get a higher place than they deserve.  In cases like my 77/23 example, they may (arguably) get a lower one.

(# Plus the extra vote or fraction of a vote as normal)

----------------------------------------------------------------------------------------

* Irrelevant footnote, but I couldn't resist another swing:  The Greens' choice to allow candidates to nominate secretly is derived from a perception that identifying as a Green in Tasmania has traditionally "killed off chances of jobs in the public sector and in certain parts of the private sector".  In my view this is just another case of the Greens playing at being unique and beautiful snowflakes when they're not.  For starters, quite a few politically active Greens do work in the public sector, especially in academia and health.  Discrimination in employment on the grounds of political affiliation, belief or activity is illegal in Tasmania, and the Greens can hardly think those laws are useless since they unsuccessfully supported using them to wreck free speech in the state.  Some would say the Greens are pretty good at killing off some people's job chances themselves, and usually doing so in a way that affects people earning a great deal less than them.  And so on.  

The point for me is that risking employment by being politically active is in the nature of choices made to go into public life.  I personally tend to avoid working under nebulous public-sector laws if I can, so that I can comment freely on politics without risk to my work.  The benefit to a party of making the names of all preselection-seekers known is great because it gets to gauge public reaction to the candidates, which may even reduce the risk of screening failures.  

------------------------------------------------------------------------------------------

Note added 2021: This article was written before Senate reform.  Some references to "unwinnable" positions have been edited as now no position is unwinnable if a candidate can get a high enough below the line vote.