All Polling On The Plebiscite Has Problems

In the last few weeks we've seen some new polling results concerning the Coalition's proposed plebiscite on legalising same-sex marriage (or as it is more accurately described, marriage equality).  It is extremely well-established in polling that a clear majority of voters support legalising same-sex marriage, but whether voters support deciding the matter by plebiscite or parliamentary vote has been less obvious.  The very inconsistent results from various polls on this subject are causing a fair degree of interest and confusion.

In this article I suggest that the range of results we are seeing on the question of a plebiscite vs a parliamentary vote is largely a result of differences in design between different polls.  Which poll is right and which poll is wrong?  My view is that all of them are suspect.   The question is a difficult one to poll and none of the polls offered thus far have even got close to a design that accurately reflects the choices the parliament, voters and activists face.



As noted previously (eg in my piece on a postal plebiscite proposal) I am not at all neutral on this issue. Although not personally affected by the issue, I very strongly support legalising same-sex marriage by an immediate parliamentary vote, and have written about this at length on here before.  Indeed, I consider politicians who oppose same-sex marriage to be simply unfit for office in a modern liberal democracy, because politicians should be protecting people from unjust discrimination, not entrenching it or outsourcing decisions on it to the people because they are too weak to act.   Yet while politicians have failed on this issue in the past, and both major parties (the Coalition especially) should be noted with disgust by history for this, they cannot keep failing for long. It is for this reason that I prefer the parliamentary vote road to a plebiscite which would confer respectability and seriousness on views which don't deserve it.

As will be seen below, my own anti-plebiscite views don't stop me pointing out when a poll's design is skewed against the plebiscite.  Polling should always be scientific and fair.

The Polling

The following are examples of the polling that has been conducted on the question of a same-sex marriage plebiscite versus a parliamentary vote:

1. Newspoll

In late September 2016, Newspoll asked the question "The Government has proposed a national plebiscite to be held in February this year to determine if Australians are in favour of a law to permit same-sex marriage.  Others have suggested that we do not need a plebiscite as politicians can make the decision in parliament.  Is your preference to have the plebiscite next year or have politicians decide the matter in parliament?"

The result was 39% support for the plebiscite, 48% for a parliamentary vote, and 13% uncommitted.

2. Essential

Essential has consistently asked the question "Q. Do you think the issue of same sex marriage should be decided by Parliament or should there be a national vote?"  

Results have continually shown strong support for the national-vote option, although its lead has eroded slightly but steadily over time from 67-21 in September 2015 to 59-29 now (an eight point swing in nearly two years).  Currently, those who are most supportive of same-sex marriage, and those who support parties who are most supportive, are the most likely to prefer a Parliamentary vote, with a narrow majority of Greens voters preferring that option.  However, even those who support a "yes" vote break 57-35 in favour of the parliamentary option on this question.

When the cost of the plebiscite was mentioned, support dropped sharply to 43% in favour, 41% against.

3. ReachTEL

ReachTEL recently asked a question which they summarised on Twitter as "How would you prefer the question of SSM to be decided? A vote of MPs in Parliament or vote of Aus people in a plebiscite?"  This was a forced-choice question, meaning that any voter who refused to answer this question or any other forced-choice question in the same poll was excluded from the poll (ReachTEL have told me that once respondents get past the voting intention question, the rate of hangups is trivially low.)  The result was 59.1% for the plebiscite and 40.9% for the vote in Parliament.

In late August 2016 a ReachTEL commissioned by the Australia Institute (a left-wing think-tank) found 51.3% support for a plebiscite.  The question was "Do you think the legalisation of same-sex marriage should be decided by a national vote, called a plebiscite, or should it be decided by a vote in parliament?"  Supporters of same-sex marriage broke 61.7% for a parliamentary vote, and opponents broke 71.4% for a plebiscite.  It should be noted that the baseline question on same-sex marriage was closer in this poll (59.7% support, 40.3% oppose) than in most polling. The baseline question was the referendum-style "Do you approve of a law to permit people of the same sex to marry?"

4. Fairfax - Ipsos

A Fairfax-Ipsos poll on the eve of the 2016 election found 69% support for a plebiscite with 24% support for a parliamentary vote.  At that time young voters heavily supported a plebiscite.  I have not located the wording of this question.

5. Galaxy

A commissioned Galaxy for PFLAG just before the election found 48% in favour of a plebiscite and 30% against.  Support dropped to 33% if the respondent was told the plebiscite wouldn't be binding, and 25% when informed of the expected cost.  I have not found the question wordings.

A later commissioned Galaxy asked voters how they would like it if their own marriage decisions were subject to a popular vote, with only 13% happy and 76% unhappy with the concept.  The breakdown on a plebiscite was 36-35 in favour.  Again I have not found the question wordings, and in cases like this the order of questions can also have a massive influence.

Problems with this polling

The polls quoted above, with the exception of Newspoll, demonstrate a basic desire of voters for a national vote. This could be seen as representing a well-earned distrust of politicians to get it right on this specific issue, or could also be seen as representing some instinctive populist desire for more participation in politics generally.  However, in terms of finding out what people think about how parliament should deal with the issue, I will argue that these polls are useless, because they do not capture the choice the parliament in fact faces.  They - usually unintentionally - prime the respondent with dubious selections of facts and misrepresentations of what is actually going on, and hence none of the results can be trusted.

Some of the polls have specific issues too.  Essential does not use true random sampling (though the randomness of any form of sampling has always been and is increasingly distorted by different levels of contactability and willingness to respond) and there is some debate as a result about whether its issue-question results are ever reliable. However the Newspoll question has the most design issues, which may explain why its result is the odd one out:

* The Newspoll question unnecessarily links the plebiscite proposal to "the Government", which could result in voters reacting against the plebiscite because they disapprove of the government (or for it because they approve, but at the time of the poll, more disapproved.)

* The Newspoll question is time-specific.  Some voters might support a plebiscite but have preferred that it be held in conjunction with an election, or after longer debate, or at some time other than February 2017 for some idiosyncratic personal reason.

* The Newspoll presents only a weak argument for the plebiscite: "to determine if Australians are in favour".  Some supporters of a plebiscite would consider this is a settled question (which it presently is based on abundant polling evidence) but would nonetheless believe that there are stronger arguments for having a plebiscite rather than a parliamentary vote.  For instance they might argue that there is a strong public legitimacy advantage in a national vote.

* The Newspoll then presents an argument for the plebiscite "we do not need a plebiscite as politicans can make the decision in parliament".  While this is also a rather weak argument by itself, the structure of the poll is a common one seen in preambles.  The standard form is "A says X because Y but B says X is silly because Z".  B gets to reply to A but A doesn't get to reply to B.  B tends to do rather well in polls that use this structure.

When it comes to the polls that have shown results supportive of a plebiscite, they have generally referred to a plebiscite deciding the issue.  Yet most forms of the plebiscite that have been presented (and indeed the only forms possible while the issue remains blocked by the Senate) have involved a non-binding plebiscite followed by a parliamentary vote.  If someone's support for a plebiscite is outcome-focused then this is a largely irrelevant distinction, since if a plebiscite passed by any sort of margin at all then only a small number of MPs would be likely to still vote against.  But if someone's support for a plebiscite is process-focused (ie they believe that the people should decide directly) then they might well feel that a plebiscite that does not directly decide the issue is pointless.

But the biggest problem with these polls is that they don't represent the reality of the parliamentary vote scenario.  Voters are probably concerned about whether an immediate parliamentary vote would pass, but a failed vote in this parliament would do absolutely nothing to prevent a vote in the next parliament after the next election changes the numbers.  If Labor wins decisively, the bill will pass.

So the idea implied in these polls that any one vote in parliament decides the issue is a false scenario.  Taking a position in favour of a parliamentary vote doesn't mean arguing that a single vote in parliament should decide the issue for decades to come.  A yes vote in parliament would very probably be the end of the matter for a long long time, but a no vote would probably not.

The choice is really between:

* Parliament voting on the issue whenever it chooses to, possibly with multiple future votes, until same-sex marriage is passed (which would then almost certainly be the end of the matter for a long time), or

* A plebiscite of some kind, maybe binding, maybe not, where the result of a single vote whatever it was would very probably be the outcome for at least a decade to come.

No poll is really describing what is at stake in supporting or opposing a plebiscite correctly.

This would not be a problem if the question was something like "Do you support or oppose the holding of a plebiscite on the question of legalising same-sex marriage?", because in that case the voter can make their own unprodded decision based on their own opinions of what might happen if a plebiscite was held or wasn't held.

However that sort of poll design would be open to a different accusation - that of "pony polling".  Voters may instinctively go "a national vote! yay!" (on certain types of issues at least) but if they are presented with an even selection of arguments for or against, their view might be different.  Voters generally go "yay!" to proposals to give them things that sound good, but they also go "yuk!" to high government spending.  These different kneejerk reactions are inconsistent with each other.  Ask voters if they would prefer the government to spend $160 million on a plebiscite or to spend the same money on health and education and see what result you get.

If polling on the plebiscite question is to be more useful it needs to start drilling down into why there is a yes result for the plebiscite when it is not supported by those who are most affected.  Plebiscite supporters should be asked whether their support is because they hold really general views that there should be public votes on more or less everything, or whether it is just that they do not trust politicians to get this particular issue right.

Update 11 June: Newspoll has re-polled almost the same question it originally polled (on which its original poll was the outlier), this time finding 46-39 in favour of the plebiscite.  As William Bowe notes, this version differs from the original by not referencing any specific month.  This means that no pollster has consistently found support for a parliamentary vote.

Update 18 July: Essential has asked a very interesting question in which respondents choose between a parliamentary vote, a binding plebiscite and a non-binding plebiscite followed by a parliamentary vote.  The latter (via the defective postal plebiscite proposal which got another airing from the Queensland LNP today) is the only viable plebiscite option given that a binding plebiscite would require legislation, and the Senate has blocked the plebiscite already.  The poll finds that support for a binding plebiscite is far greater than support for a non-binding one, including among both supporters and opponents of marriage equality.  For opponents a non-binding vote would seem to offer an extra path to blocking reform, so it may well be opponents actually think they would win a national vote.

One issue with the concept of a "binding" national vote is that the parliament can't bind a future parliament except through a constitutional referendum.  So while a yes vote in a "binding" vote obviously means that same-sex marriage becomes law, what does a no vote mean, and how is it actually "binding" for more than the rest of a term, if that?  Opponents may not have considered this aspect.  The only other thing I'd say about the poll question is that I would have slightly preferred "a decision on whether or not to legalise same-sex marriage" to the actual wording "a decision on legalising same-sex marriage" as the latter just might be interpreted as asserting that legalisation is inevitable and asking how legalisation should be done.  I don't think it matters though, especially not given the strength of patterns shown.