Wednesday, October 4, 2017

Marriage Law Survey Turnout Is High ... But Not That High!

(This article is being progressively updated through the survey - KB)

The first release of turnout estimates from the Australian Bureau of Statistics for the same-sex marriage postal survey has created some interest.  The ABS estimates that as of Friday 29 September, 9.2 million survey forms have already been received, 57.5% of all enrolled voters.

The ABS notes that this is an estimate only.  It may not represent the received yes/no vote as not every single one of the forms submitted will be valid (some small percentage may be posted back deliberately blank, for instance, or even with both boxes ticked.)  It is also unclear to me whether this estimate is based in some sense on a count of forms or on a count of envelopes, since there could well have been some cases of multiple forms being posted back in the same envelope, or of empty envelopes being posted (heck, I used to do this to junk-mailers who included a reply-paid all the time).  It also doesn't include anything that has been posted but was still on its way to ABS as of Friday.  So the figure is likely to be at least a few percent short of the number who have now voted.



This figure has attracted attention because of discrepancies with turnout figures in some of the polling.  Some early turnout estimates and projections were reported in a previous roundup, but we're now at the stage where almost everyone got their survey over a week ago.  Yes campaign internal polls have reported up to 77% of enrolled voters voting.  A Sky ReachTEL with a massive sample size reported 79% having already voted.  This contradicted a separate, unsourced ReachTEL with a smaller but still decent (1000) sample size that found 72% saying they had voted or definitely would vote, including c. 60.5% who had already voted.  (That other ReachTEL also had a much more normal Yes vote for this pollster - 61%).

If the Sky ReachTEL is to be believed as reported, then the postal vote is already over as a contest, with a massive 64% of enrolled voters already voting Yes, with Yes running at 80.5% among those who've already voted, and projected to finish at 76.8% including those who say they will vote.  That would be the second-highest Yes vote on a two-answer basis in any poll in the country on the issue.  If the other ReachTEL is closer to the mark then No is still mathematically in it if it can get some of those who aren't sure they will vote to do so - but even then it would need a very large majority of the remaining votes.

There aren't any polls yet that are giving No a sniff of actually winning, but even assuming Yes does win comfortably, the final turnout and the magnitude of the result are likely to be significant in terms of the pressure on government for a quick solution following the vote.  Especially, should there be more Yes votes than No votes, the condition for allowing a bill will have been activated and a bill of some sort should then pass. But if the total number of Yes votes is less than half all the enrolled voters, some mendacious goalpost-shifting MPs may seek to argue that that's not good enough and use this as an argument for voting against same-sex marriage or mucking about to attempt to delay legislation.  Or someone might say that, say, 66% is not enough, it should be 70%.  If you think nobody could possibly be that stupid, I've heard there was a lengthy discussion of this concept on Sky!

The higher the turnout, the higher the chance of the Yes vote exceeding one in every two enrolled voters and making things really clearcut for even the most evasive pollies, but the chances of this happening are unclear given the widely divergent turnout figures we are seeing.

So who is right and who is wrong?  The ABS count is bound to be a few points short of the current turnout, but the accuracy issues with its estimates shouldn't be enough to explain gaps of twenty points.  I think that some of the polls have turnout a little too high, and that the most likely reason for this is that they are undersampling politically apathetic voters, who will tend not to respond to this survey at all.  These voters might either not be opting into polling in the first place, or might be part of online panels but rarely online.

Anyway, we have at least another five of these updates for pollsters to calibrate their estimates against!

The SkyNews ReachTEL is so strong for the Yes vote that I am treating it with caution pending at least the release of full details of the poll.  It's very disappointing that for a poll of such a size we have not yet had a verbatim release of the questions asked and a publication of the original tables.  It's one of the largest polls conducted in Australian history and I cannot even use it in my voting intentions aggregate because no primary votes have been released! (Edit: some have now.)

But that said, it is clear that turnout for the survey will be at least reasonably high, perhaps very high - it could still at this rate get to, say, 75%.  It is higher than for other postal vote processes in Australia, which were never that comparable as they mostly involved more obscure issues.  We're also not seeing anything yet in the last week or so to suggest that the narrowing of support for Yes being found by some polls earlier on has continued, nor anything in the public debate to suggest that it should.

Perhaps the campaign doesn't matter all that much and people are just answering the question.  But assuming it does matter, I have found it difficult to judge the quality of the campaign because I view the entire No case as completely unjustifiable, and I have had to make severe adjustments to even try to account for this.  With a handicap larger than mine would be at golf applied, I thought that the No case had a good run of the media cycle early in the response phase, but has looked silly even by its own standards in the past several days, including being seen to contradict itself on free speech by whinging about Macklemore and being reduced to scouring social media to advertise evidence that it was being hated.  So if some of the move back to Yes implied by the Sky ReachTEL is real, it's not too hard to say where it might come from.

I'll update this article over the next several days with more comments on new polling as it comes out.  I've meant to also write about security issues in the process but I have been extremely busy lately and this isn't likely to change any time soon.  Comments on those remain very welcome!

Updates (Wednesday Oct 4): New today we have Essential (here, here, here) and this is the first attempt to cross-ask questions about in-principle support and actual voting.  The poll finds a 61-32 breakdown in in-principle support with a 66-30 breakdown among the (low if the ABS are correct) 47% who have already voted.  (The poll was taken from Wednesday to Sunday, so given postal delivery lag should ideally have had a slightly higher turnout than the ABS).  Those who voted reported voting Yes 64% No 30% "Prefer not to say" 6%.  There were actually more who opposed same-sex marriage but said they voted Yes than who supported same-sex marriage but said they voted No.  However, Yes lost more votes to "prefer not to say" (whatever that is).

There is also some fossilised internal poll stuff being spruiked by Miranda Devine - see comments.

Week Two Update

The week two ABS update estimates only another 5% have been received in the past week, bringing the total up to 62.5% as of Friday.  This will probably tail off further from this point, so I am still expecting something around the low to perhaps mid 70s.  Unless the ABS are making a serious error then this new figure confirms that the turnout figures in some of the polls were too high.

There is no new reliable polling (though more unsubstantiated Devine spruiking of No campaign internals has been sighted) with Essential this week not polling the question. I've just taken one for the team by watching 34 minutes of Q&A (aaaaaaaarrrrrrggggghhhhh!) to determine that Michele Levine of Roy Morgan said that according to their polling same-sex marriage had over 80% support prior to the debate surrounding the postal survey but that this had dropped to a level consistent with a Yes vote of 60% or more.  This should be treated with caution since Morgan's only published polling on this in the last few years was an SMS poll with an element of opt-in and even that only got a 76% Yes vote.

Added Oct 14: Some more details re Morgan polling here.  Note that it actually implies a Yes vote of higher than 61.5% given that 21% either didn't answer or aren't voting.  How much higher is a matter of speculation.

Week Three Update

The week three ABS estimate update adds another 5%, the same as last time, bringing the total up to 67.5% of all enrolled voters (10.8 million)  This hasn't tailed off in the way I expected, though it's always possible the tailing off has been lost in the rounding.  Because the number of surveys estimated to have been returned is rounded to the nearest hundred thousand, in theory there could have been just over 700,000 this week and just under 900,000 last week (or, less probably, the other way around).  Indeed, we don't know for sure whether these estimates are necessarily accurate to the closest or even second closest hundred thousand.

Hitting the turnout bullseye wasn't too difficult for YouGov which reported 67% of its sample had voted with 61% yes, 35% no.  Nothing new from Essential this week.

Newspoll has just come through reporting 65% of its sample had voted with 59% yes, 38% no. On the in-principle support-oppose question it has 56-37 for Yes (compared to 57-34 in its previous poll).  The 35 points that have not voted include 19 points claiming they definitely will and 6 points claiming they probably will, which would make a total 90% turnout.  Those who have not voted split 49-37 to Yes.

Week Four Update

As noted in comments, the week four update has 11.9 million forms returned (that's 74.5%!)  The ABS says only 300,000 were received in the last week, meaning that last week's estimate was 800,000 short.  The explanation for the difference would be errors in the ABS's weight-based method for previously estimating turnout, so really there has been a big tailing-off this week.  The question now is whether there will be a late rush that might lead to a further pickup and push it to above 80%, or whether there might be only, say, 3-4% still to come.

Essential has a 60-34 yes-no response among those who say they have voted, which is 75% of their sample (a figure that is looking very good after today's release!), with another 8% still saying they definitely will vote.

Week Five Update

The week five update has 12.3 million forms returned (77%).  I have seen no new polling this week yet. There was, however, a report that only 0.3% of the returned forms are blank or invalid.


30 comments:

  1. I question the release of official figures while the vote is still open. This could effect people's decision to vote or not to vote and even how to vote.

    ReplyDelete
    Replies
    1. There was a lot to question when it comes to something like a postal vote (people really need to stop calling it a plebiscite because it technically isn't one).

      First things first: Voter fraud. Whilst there are 'safeguards' in place which punish people who get found doing it, there is absolutely no way to prevent it in the first place.

      Second: Given the very short "speculation to action" phase, (or, at least, serious public speculation of a postal vote) the yes campaign in particular were left somewhat on the backfoot. Many government officials could have been assuring the no campaign of a postal vote months before Dutton even mentioned it (this is, of course, speculation - but politics is politics).

      Third: The fact that with anything labelled referendum or plebiscite with a simple yes/no question, Australian history has shown that no has a decisive advantage. I guess we, as a voting population, prefer the monster we know to the unknown.

      Fourth: Doing anything by post these days plays directly into the hands of more conservative voters. Postal votes in elections lean heavily conservative, largely due to age brackets, and there is absolutely no reason to think this will be any different.

      Fifth: The 'protection' laws put in place do nothing to stop the vilification or fabrication of statistics to win people over. The no campaign have been caught lying through their teeth (some of the yes claims are somewhat dubious too, but not as bad as the no campaigns), and when swastikas and fag hate signs began to appear there was literally nothing anyone could do. The protection laws were, by design, useless.

      Need I go on? :/

      Even if there is a decisive yes vote here, the result is largely null and void. The irony of this is that if a yes vote comes back, the Government drags their heels until the next election, then loses? When Labor (if they keep their promise) bring marriage equality to Australia, it will be the LNP pointing out the flaws in the vote to defend their stance.

      Delete
    2. The hate speech parts of the protection laws seem to have been all for show. They've had negligible impact on behaviour, and seem to have been breached a heck of a lot (depending on interpretation) but no prosecutions have yet been announced. I've sometimes bit my tongue on account of the unknowable risk of breaching them (perhaps needlessly since nobody actually knows what the things mean), but I don't see any sign that many others have.

      The ability to catch multiple voters (people who are stealing survey forms and filling them out) is weak unless the thieves are really stupid about it. I am used to the Tasmanian council system where if there were multiple envelopes recorded stolen it would be easy to intercept the envelopes before opening them, and probably from that capture fingerprints and samples of the thief's handwriting and other clues.

      Delete
  2. What are your thoughts on the NO campaigns internal poll released today by Miranda Devine? I think its probably nonsense but anyway:

    "WITH five weeks to go in the Same Sex Marriage postal plebiscite, the Yes vote is in free fall, especially among mothers, according to large-scale polling by the No campaign, revealed exclusively here.

    Support for changing the Marriage Act has dropped to 51 per cent over the first four weeks of the campaign, with the No vote rising to 37 per cent.

    The poll of 1200 people was taken on September 25, and has a margin of error of 2.5 per cent. It reflects a similar trajectory to a recent Newspoll." http://www.dailytelegraph.com.au/rendezview/mums-fears-could-sink-samesex-marriage-push/news-story/c7ecd12e5d7fad04fb9034ed16154c10

    ReplyDelete
    Replies
    1. Thanks for mentioning that one, hadn't seen it. After Devine's previous effort (pushing findings about LGBTIQ attitudes that were obviously not credible and with no information about the poll's methods or who conducted it) I treat anything she reports about polling on the issue as extremely dubious. But it sounds like it is possibly from the same series as previous No internals that have been reported, as they have also tended to run somewhat below the neutral polls at any given time. We don't have any details about the method used or who the pollster was or even any assurance that it was any reputable pollster. The only information apart from the sample size is that the poll is pretty old rope now - September 25 - raising the suspicion that it might be cherry-picked. Presumably they would have polled more recently than that.

      Also if it was conducted on one day only then it is almost certainly a robopoll, which raises the question of whether it was one of those ones with a suspicious preamble or not (there are a few doing the rounds.)



      Delete
    2. By "suspicious preamble", do you mean push-poll? ;-)

      Delete
    3. No. Many polls are accused of being push-polls that are not.
      There are a number of different poll types that may have suspicious preambles. There is often some overlap, but these include:

      1. What I call skew-polls, where the purpose of the dodgy preamble is to generate a skewed response which can then be used to get publicity for the commissioning source's views and/or pressure decision-makers.

      2. Message-testing polls, where a dubious message is presented to some or all respondents to see how it might affect their response.

      3. Incompetently designed issue polls.

      4. Genuine push-polls, where the poll has a large "sample" size and the commissioning source is just transmitting propaganda and then throwing away the result.

      5. Data-harvesting exercises, where the real purpose of the poll is to obtain data about respondents for use in campaigns.

      The known suspicious-preamble polls that have been reported in this case are mostly of forms 2 and 5.

      Delete
  3. high??? I would have thought it would be a lot higher given you do not have to do a lot of work.

    ReplyDelete
    Replies
    1. That's certainly true, "voting" is easy. But there's also long been evidence that very few people care strongly enough to change their electoral vote over this issue, so a more apathetic response wouldn't have been a massive surprise to me.

      Delete
  4. There may indeed be some reason for scepticism about the return rates reported by pollsters, but I also hae' me doots about the ABS figure. They say the 57-odd % estimate "is indicative only as it is based on the bulk containers of returned forms and not a count of individual or processed forms. It does not include forms that have been posted but not yet delivered by Australia Post to the ABS."

    Now my question is where AusPost does the delivery and when are the containers deemed to be delivered? Out here in the Brisbane northern suburbs I noticed that our reply-paids were addressed to a PO Box at the Northgate sorting centre. Does AusPost then deliver them to the ABS's (relatively small) office in Brisbane, or to the Fuji-X document management place at Parkinson where the machines are(way over the other side of town), or all the way to Canberra - and at what stage does the ABS count them as having been "delivered"? My hunch is that there may be some millions of votes in containers being gradually trucked to Parkinson, or even to Canberra, and they may not be added to the ABS's count until they've arrived even though they may well know that they're on the way. And then there'll be quite a large jump in the numbers. You seem to have made some sort of contact with insiders at ABS Kevin, could you find out more details and pass them on please?

    ReplyDelete
    Replies
    1. The ABS is mainly relying on "Australia Post’s assessment of the number of containers of sorted envelopes." So it probably includes everything that has been mailed by Thursday 28th or Friday 29th and sorted by Australia Post, even if some of those forms hadn't been delivered to the ABS yet.

      ABS Source:
      https://marriagesurvey.abs.gov.au/results-survey#estimates

      Delete
  5. OK, rash prediction for tomorrow's ABS official % of forms received. High 70s, on the assumption that the surveys that said so were about right, and all the forms that had been posted when the surveys were taken have got through all the choke points and been counted. You'd have to expect delivery to be slow - this must be the highest number of items all being delivered to the one recipient in AusPost's history. Even tax return time in the days before online returns wouldn't come near.

    ReplyDelete
  6. Have today's numbers bee released?

    ReplyDelete
  7. Well that was a pretty bad guess! Another 800,000 forms during the week, bringing the total to 62.5%. So maybe the envelopes have been flowing more quickly through the postal system than I'd thought, with less choke points than I was imagining. I would like more info, though, on whether the number thought to be in each bulk container turns out to be true when they open the container. And where are they being scanned, Kevin? All at Moorebank, or at all the capital city FujiXerox document centres like they did for the Senate? (I think I read that the observers-not-scrutineers are being invited just to Moorebank. Is that right?)

    ReplyDelete
    Replies
    1. I'm suspicious of this too. Whether they're checking at Capital city depots, or at Moorebank. That would change the way we understand these numbers completely.

      Delete
    2. Haven't been following the details of the observer process closely enough lately to comment on the scanning process. Anyone who has is welcome to comment.

      Delete
  8. Ah, thanks to the Gwardian at https://www.theguardian.com/australia-news/2017/oct/07/same-sex-marriage-survey-count-watchers-bound-by-lifetime-secrecy , I've found some of the answers. The Grauny page includes a copy of the Observer Guidelines. The ABS had stupidly marked them Confidential, but the Grauny quite rightly ignored that. You can pop the Guidelines out for easier reading by clicking Show Only This Frame or the equivalent in your browser of choice.

    As to where the scanning takes place, p 4 says "Survey responses will be returned to five Fuji Xerox sites located in Perth, Adelaide, Melbourne, Brisbane and Sydney for processing". But then p 8 says "Manual coding of forms will only occur in the Moorebank facility, where observers are present. The other centres are only used for scanning and containment of forms." So presumably batches of disputable ones will be sent from the other centres to Moorebank - presumably. Presumably the staff at the other 4 centres decide whether something is disputable in the first place without observers looking on. And presumably the weekly count is of all the forms that have been delivered to any of the 5 centres. Presumably.

    The document then shows many examples of forms filled in in a variety of silly ways (including a write-in of "Go Tigers", with the way each one should be coded and the reason for that decision in columns to the right. Mostly seems pretty sensible to me.

    ReplyDelete
  9. Update, Kevin - another 800,000, taking it to 10.8 mill or 67.5%. Steady at 800,000 per week for the last two weeks. Good for AusPost's balance sheet.

    ReplyDelete
  10. And this week's ABS alert says 11.9 million (74.5 per cent) which "is based on a count of forms scanned rather than earlier estimates that were based on the weight of the bulk containers of returned forms" and that it estimates that it received 300,000 forms last week. Which means that the previous estimate of 10.8 mill was low by 800,000 - the forms must be 7% lighter than they thought! So the polls have been somewhere close to right all along, and the ABS's guesstimates have been low. The turnout has been pretty close to "that high" ater all.

    Have to have some doubt about the 8% who are telling Essential they "definitely will" return the forms though - if they've procrastinated for this long what's another week or two?

    ReplyDelete
  11. How close to a 3/4 vote efficiency is Australia in the rankings of things? I know we're better than America that barely gets above 50%, but the UK?

    ReplyDelete
    Replies
    1. Postal voting gets better turnouts than booth voting for voluntary voting generally. In my state we switched from booth to postal voting for council elections in the early 1990s and turnout increased massively.

      Election turnouts are higher in European democracies that have voluntary voting than in the USA which has a notably low rate. There's a site with records for different countries here:

      https://www.idea.int/data-tools/question-countries-view/521/228/ctr

      Delete
  12. Re the Wk 4 Essential figures:

    So, 60% of 75% is 45% of the Australian population having voted yes so far. Assuming the same responses in the remaining 8% who say they will vote, that'd be 49.8% voting yes – very close to an absolute majority, and that's before taking into account that 60 and 34 don't add to 100.

    An absolute majority voting yes would make the politics less fraught, looks like it's going to be very close to that.

    ReplyDelete
  13. There's a new Newspoll today -

    Returned: 76%
    Yes, of those voted: 62%
    No, of those voted: 35%

    Sentiment: (Yes/No/Undecided): 59%, 35%, 6%

    http://www.theaustralian.com.au/national-affairs/newspoll/newspoll-samesex-support-grows-as-survey-deadline-looms/news-story/c8efa8998359a13da0541835506a7ce6


    I'm attempting to record all the known polling & participation data here, for anyone interested in viewing:

    https://docs.google.com/spreadsheets/d/19rdAtUXUj3A3Nb26PBaGlqW_emnGTTnitJKZshGzqmw/edit?usp=sharing

    ReplyDelete
  14. Kevin,

    An EMRS poll was published today by the AFR. 64% 'Yes' vs 28% 'No'

    Looks like it was conducted over the same polling period as the Newspoll, although it appears to have asked about general sentiment, rather than how people voted (although it might not make much of a difference at this stage)

    http://www.afr.com/news/equality-and-fairness-key-drivers-behind-the-yes-vote-poll-20171030-gzbgjx

    ReplyDelete
    Replies
    1. A federal EMRS! Haven't seen one of those in a very long time indeed.

      Delete
    2. Thanks Joseph. Other interesting figures - 12% think "No" will win, so now we know how many stick to the "everybody else thinks just like me" fallacy even in the face of strong evidence. (I'm a bit surprised that it's that low; it boosts my faith in the sanity of the average Aussie. Probably higher in the USA!)

      And "91 per cent believed that it was important for the Parliament to reflect the will of the people in relation to the results." Not clear whether that includes the deluded 12% who will presumably scream "fake result" when the result is released. Some cross-correlations between the answers to the different questions would have been nice.

      Delete
    3. I haven't found any info on this EMRS poll aside from the two articles in the AFR:

      http://www.afr.com/news/equality-and-fairness-key-drivers-behind-the-yes-vote-poll-20171030-gzbgjx

      http://www.afr.com/news/politics/national/what-the-high-samesex-marrriage-vote-turnout-means-20171031-gzbse1


      It mentions 1000 voters, which is the same size as EMRS's Tasmanian CATI omnibus according to their website - are we sure this is a national poll? Anyone got any other info.?


      Thanks

      Delete
    4. Update - I emailed EMRS and they said:

      It was a national poll of n=1,000 adults.
      Method was combined online & telephone – 70% Online and 30% Telephone.

      Telephone was done from EMRS's Hobart call centre (so I imagine it was 'live' and not IVR)

      Delete
  15. And apparently 15% of people even in this land of the fair go still think that people engaging in same-sex relationships should be charged as criminals - see https://www.theguardian.com/global-development/2017/oct/31/one-in-four-people-say-those-in-same-sex-relationships-should-be-charged-as-criminals-survey-77-countries
    At least we're equal lowest with NZ.

    ReplyDelete