Poll-Shaped Objects: January 2022

Welcome to a new irregular series for this site.  From time to time I will do a post that covers and comments on all of the poll-shaped objects that come to my attention in a given month.  Where relevant, the article may also discuss real polling on the same issue.  Once I've decided there is enough material and time to put out an issue for a given month, all others that I see in that month will be added to the piece.  However I won't necessarily put out a PSO article every month; usually I will release such articles when there seem to be far too many of the things around and I feel that the proper order of things needs to be restored.  An early version of this idea can be seen in a 2017 article Poll Roundup: Attack Of The Poll-Shaped Objects.

The term poll-shaped object is one that I use to deride the overly credulous reporting of unsound or insufficiently transparent polls in media articles, and also the peddling of such polls by pressure groups, so-called think tanks, parties and other groups that cling to the foolish belief that you can influence public opinion by telling fibs about it.   A PSO can, for instance, be any of (i) something that is claimed to be a poll but isn't (ii) an especially unsound poll or (iii) a possibly sound poll for which the level of reported detail has been woefully lacking.  (The term is by analogy with piano-shaped object, something that appears to be a piano but sounds horrible or is impossible to play - whether by reason of shoddy construction, mistreatment or decay.)

For the avoidance of doubt, I don't think it's acceptable for media to publish reports of polling that only include primary vote figures for two candidates.  Media reports should publish full primary votes or a link to somewhere where they are online at the time of publication.  If full primaries aren't available then a report on the poll should not be published until they are.  

1. Nearly all parents oppose reopening schools?  

Culprits: Single AAP article, widely syndicated with sightings at New Daily, Seven, Ten, West Australian, Canberra Times, etc.

MO: Unscientific and badly designed opt-in

A survey by a parents' advocacy group called The Parenthood has been reported as a "poll", together with unqualified claims that "Two-thirds of parents believe it is unsafe for children to return to classrooms in a few weeks [..]" as if this is a fact about Australian parents in general.  However none of the reports indicate that the survey has been conducted by a pollster, say that it used any form of scaling or provide any detail of methods beyond "sample" size (3043).  Moreover, the details provided exactly match an opt-in survey on the organisation's website.  I was able to take this survey and found that it collected no demographic data whatsoever apart from state of residence and the age and number of alleged children.  It also does not openly block repeat answers.  On my first run through the survey I claimed to be the parent of a six-year old, but on the second run I claimed (correctly) to have no children but said their ages were 0-2 and 18+.  The survey form did not challenge this obviously inappropriate response and let me complete the survey.  

The survey also includes this inappropriate forced choice question - what if the parent isn't worried about either?

There is no evidence this is a representative survey in any way.  It appears to be simply an opt-in survey of people who visit the Parenthood website or are recruited by them in any other ways.  The results should be ignored.

2. More Climate Indie Poll Hot Air

Culprits: KORE, AFR

MO: Panel poll by dubious source

Following a previous wave of poll-related nonsense about climate indies in Flinders, Goldstein and Kooyong (the main issue being that a poll in which respondents were primed with pro-indie statements was misreported as a straight voting intention poll) we now have a second volume with claims about Goldstein, Wentworth, and North Sydney.  The claimed primaries have the independents leading Liberals 40-24 on primaries in Goldstein and behind 32-36 in Wentworth and 24-40 in North Sydney. At this stage the available details are woefully inadequate though fuller details have been promised by the pollster and we are told the sample was around 500 per seat.  I was initially under the impression that Climate200 was involved in this one too, but it seems it was an unsponsored poll by KORE.  

The first problem here is the pollster: KORE is the successor to Voter Choice whose founder has a poor track record in published testable pre-election polls and has made some very weird claims about polls and elections on social media (see Voter Choice section of my pollster guide).  The strange methods of KORE include openly advertising for survey respondents on Twitter (which is just asking for politically motivated flooding, which seems to have happened to it at least twice already, including once by people who disliked Daniel Andrews).  In addition to the social media samples KORE has said these seats received "snowball and email targeted recruitment" but without further detail.  

The second problem is the interpretation.  If Allegra Spender polls within 4% of Dave Sharma on primary votes then she wins very easily indeed (Kerryn Phelps defeated Sharma from 13.9% behind him in 2018).  It is unlikely she would have that sort of support level across Wentworth - more than Phelps' 2018 primary - if she was a candidate who "has generally failed to connect with the Wentworth electorate and is still considered an unknown".  This suggests, in my view, that the poll is simply oversampling voters willing to vote for a generic independent.  On being asked on Twitter to explain why she was projecting a loss for Spender off 32-36 primaries when Phelps won off 29-43, the person behind KORE gave another of her strange responses: "That was a 5 candidate split, not 3".  It was actually a 16-candidate split. An immediate follow-up tweet which I at first missed because I had been blocked said "5 viable candidates" but that makes no sense either - Licia Heath (2.26%) was not relevantly viable on election day, and will be outpolled by whoever is 4th in Wentworth next time.

As with some of the previous Redbridge polls, there is also the issue of the sheer implausibility of the Goldstein result which would be a 28.7% swing against the incumbent.  Perhaps there is a large undecided vote unaccounted for but even so it defies credibility.  As I noted in the previous round:

" Even the 2019 model Tony Abbott, who in personal vote terms seems to have been a worse candidate than the average case where a candidate is disendorsed but remains on the ballot, still polled 39%, and that was a 22% swing against him over two elections compared to the one that he won."

KORE is not a member of the Australian Polling Council so what we do get by way of methods details will probably be idiosyncratic.  (Update: and in the case of these seats specifically it turned out to be relatively little.)

Why are the AFR just printing any old rope that looks like a poll with no more than a (apparently inaccurate) he-said-she-said from one of the incumbents?  When they could get comments on the reliability of the polling for free? Do they know what the word "Review" in their magazine name even means?

(There have also been social media sightings - not yet picked up by media thankfully - of Tasmanian Senate polling (not by KORE) claimed to show a contest between the Liberals, Jacqui Lambie Network, United Australia and the still federally unregistered Local Party for the final seat.  No details have been published and this too should be treated with extreme care until the wordings and results are seen.  I think it is unlikely the Local Party would give an asterisk a run for its money in a poll that wasn't primed.)

Update Care of the Wollondilly Advertiser we have another one of these ... things ... from the seat of Hume.  It claims Angus Taylor is down 21.7% to 31.6, with independent Penny Ackery (whose supporters are said to have commissioned it) on 7%, "well behind Labor, One Nation, The Greens and the United Australia Party." (This seems unlikely when the Greens and UAP scored around 5% last time with One Nation not running).  Why not print the actual numbers for these parties, the poll date or the sample size?  The poll is by Community Engagement who are also not a Polling Council member and have very little verifiable form (see field guide).

Update 2: We now have even more KORE silliness, reported by AAP and picked up by Seven and probably others: a national poll with Labor on 40.1 (42.6 after redistributing undecided) leading the Coalition on 22.7 (24.1).  This would be a 2PP of around 62.7% to ALP, for a 123-22 2PP seat margin (though several of the Coalition's 22 would be picked up by independents or perhaps even Greens or One Nation). Quite what KORE are doing to come up with this nonsense isn't clear but I suspect it is a lot (or maybe in the scaling department, not enough.)  

There has been a lot of debate about this sample being apparently stacked by people on Twitter after a false claim circulated that it was a Scott Morrison internal poll.  While KORE stated they removed 176 responses that "indicated they believed the poll was for Morrison", it is highly unlikely that anywhere near all respondents who believed it was a poll for Morrison would have indicated this clearly, so the size of the unintended viral stack is probably much higher.  Also KORE later described the reason for exclusion as the responses being "abusive or invalid".  Then they went on to quote several sweary messages and say that swearing was fine. 

Whatever, it is clear the poll received a very large response (several times the previous).  A tweet advertising it was retweeted 250 times and quote-tweeted 80 times whereas previous tweets had insignificant retweet action.  This would have given it a massive audience.  The biggest problem with this "poll" is that the sample will include large numbers of social-media opt-ins despite the left-wing skew of some social media sites and the risks for polls to be retweeted into ideological ghettos.  This level of skew cannot be entirely fixed by weighting, and indeed the weights used (age, gender, location and some kind of insufficiently explained correction for partisan skew) would not fix it.  A 75-year old male Queenslander in a left-wing Twitter retweet ghetto is still very likely to be a leftie.

3. Voters Wanted Djokovic To Stay?

Culprits: The Age, Novak Djokovic's solicitors - and on the other side, Herald Sun

MO: Unscientific opt-in "reader poll", skewed wording

-

During the legal shenanigans surrounding the deportation of Novak Djokovic it was amusing to see Djokovic's legal team claim "vocal support" for him to stay that was supposedly based on polling.  However the poll turned out to be an unscientific opt-in "reader poll" where the viewers of a website simply answer questions without any scaling, attempt at representative sampling or quality assurance whatsoever, ie not a real poll.  There were a number of these opt-ins regarding Djokovic, most of them producing results strongly against him.  It was somewhat comical to read the Minister's response which politely said the Age's poll had been taken into account and didn't mention any other polling on the matter.  

Moreover, the Age's opt-in poll question had been framed in a way friendly to Djokovic: "Should Novak Djokovic be allowed to stay and play in the Australian Open?"  Intentionally or otherwise this draws attention to the virtues of permissiveness and the value of letting him compete.  The poll (screenshot above, showing that whoever took the screenshot also threw in a vote in Djokovic's favour) was also embedded in a quote from a Conversation article by Michelle Grattan, an article that was deeply critical of the government's intent to deport Djokovic.

On the other side there was the Herald-Sun, which has a long track record for not caring about whether things it calls polls are polls or not.  The Herald-Sun passed off its own unscientific opt-in as "an exclusive poll conducted by News Corp this week", as if this put it on a level with Newspoll, but it actually wasn't a proper poll at all.

Remotely scientific polling on the Djokovic matter was scarce.  Resolve offered a poll with 71-14 support for booting Djokovic but the poll employed a very longwinded preamble:

"Novak Djokovic, the international tennis player, was recently granted an exemption to travel to Australia without being vaccinated to play at the Australia Open tournament by two review panels. This was challenged by the Commonwealth Border Force, was the subject of a court appeal and a decision by the Immigration Minister.

Do you think Novak Djokovic should be allowed to stay in Australia to play tennis when unvaccinated or not?”

This question twice draws attention to Djokovic's unvaccinated status but does not state what review panels gave him an exemption or why, or give any other argument for Djokovic's cause.  This is likely to lead to a negative response and long question wordings like this in general give the respondent lots of room to go astray.

Utting Research tweeted a claimed 51-31 poll result for booting Djokovic but I have seen no detail bar the tweet, and Painted Dog found 81% of WA respondents according to Poll Bludger (via a West Australian report that I am paywalled out of reading).  

4. Do Voters Choose Australian Choice?

Culprits: Australian Republican Movement, Age/SMH, Canberra Times, SBS

MO: Who knows? They haven't told us.

The Australian Republican Movement recently launched their "Australian Choice" model for selecting an Australian Head of State.  A nod to the failure of the 1999 referendum (which had an indirectly elected President appointed by a two-thirds majority of a joint sitting of Parliament) this one features a somewhat sanitised direct election model in which each state or territory's Parliament may appoint one candidate and the federal Parliament can appoint up to three.  In the event that only one candidate is nominated there is a yes/no vote to approve them, and otherwise there is a preferential election (which if there were 11 candidates could have a very high informal voting rate).  The Head of State's role would be much more minimal than the current Governor-General's - most notably they could not refuse a request for an early election from a PM with confidence of the House, and they also could not sack a PM who had confidence of the House.  (A lot of this seems like Whitlam nostalgia).

The ARM claim to have strong support for their position (73%) via December 2021 polling of yes/no support for various options, which is well ahead of other models.  Many media sources have lapped this claim up and reproduced it without providing any further details on the polling, beyond one AFR report that states it was conducted through Pureprofile.  Pureprofile is a survey research platform and is not a member of the Australian Polling Council.

The most crucial detail lacking is the exact wording of the questions.  A graphic shows "Australian Choice" beating other options but it is not clear how those other options were described to respondents.  In particular was the ARM's preferred model described to respondents as "Australian Choice", which would alone have been likely to prime respondents into voting for it?  

Other models could also be wording-sensitive.  For instance, the ARM seems to be concerned that an open nomination direct election system would see unsuitable populists unsuited to a constitutional role, and also that a direct election system might have too many candidates.  However there are ways around the latter such as requiring a large number of nominators - were such choices put to voters?  How were the respondents recruited and were responses weighted (if so how)?  There is so much that media seem not to have asked about this "polling".

What's especially bad about this is that the ARM extols the virtue of "Rigour and integrity" in having parliaments select nominees but displays no rigour or integrity when it comes to publishing information the public need to evaluate its polling claims.  This is very unfair to volunteers who are being led into supporting this model on the basis that it is "electable" but are not being shown the data based on which there can be a proper debate about whether this is true.  I also note that even if this poll is above board, public reaction to a novel proposition can easily be of a pony-poll style "that sounds nice" form, with no guarantee that the model will still be so popular after public debate.   I asked ARM questions about this polling on their Twitter handle but my questions were ignored. 

More poll-shaped objects may be added later this month, especially surrounding January 26 which is a recurring source of the things.