Ditching Two-Party Preferred Won't Resolve Australia's Polling Problems

Resolve: Coalition 38 Labor 33 Green 12 One Nation 6 IND 8 Others 3.

My 2PP Estimate 50-50

Alternative Estimate 51-49 to ALP

A new federal polling series has been launched and what we have before us is a mixture of the good, the bad and the interesting.  Such is my initial reaction to the launch of Resolve Political Monitor, a new poll by Resolve Strategic for the SMH and The Age.  Many of the reporting decisions around this new poll appear intended to keep it on a similar footing to the Nine stable's former Nielsen and Ipsos polls.  The poll will run monthly, and excitingly will include bi-monthly state polling for NSW and Victoria, something there has been a deplorable lack of in recent years.  

Resolve Strategic is a polling firm run by Jim Reed, formerly of C|T (Crosby-Textor) and Newgate.  It was recently best known through its major and lucrative role in coronavirus-related public attitudes research for the federal government.  Reed's massive experience is mostly in internal and other non-public polling and there is very little on the public record that enables me to predict the accuracy of the new poll.  We'll just have to see how it goes at the election.  Newgate was, however, one of the more accurate polls on the same-sex marriage postal survey.

The new poll is mostly an online panel poll which will have a sample size of 1600, though the inaugural survey also includes 400 live phone poll respondents (mixed landline-mobile).  Respondents are sourced "from reputable panels employing off-line recruitment techniques and incentives", but it is not stated which panels (meaning it is not possible to deliberately opt into the broader panel from which respondents are preselected as is the case for Newspoll).  The reporting notes: 

In all cases, minimum quotas are set for age, sex, area and other demographic or lifestyle attributes, and data weighting is employed where required to ensure accurate representation of the population.

I think this "other demographic or lifestyle attributes" is vague and more detail of what these are should be provided.  In particular, is education included? What about past vote (which is controversial)? However the use of quotas to minimise the use of weighting is something also seen from Newspoll, as the 2019 failure raised concerns about excessive weighting of unrepresentative respondents (especially in robopolls).  

Removing The 2PP

The most notable decision of the new polling series is not publishing a 2PP estimate, although it seems the reporting will continue to drop vague hints about the 2PP.  The following is the major justification excuse for this decision that the SMH/Age has been publishing:

"Our readers told us in the past they did not appreciate the “horse race” nature of the way we reported the results of TPP questions and they wanted something deeper.

It wasn’t just that all the major polls, including the one published by the Herald and The Age, predicted a TPP win for Labor at the last election, although that was clearly a factor in our new approach. Years of leadership contenders using polling as a justification for knifing incumbent prime ministers led, understandably, to the perception the polling was distorting politics, not just examining it."

Let's dig into this history of PM-knifing as it relates to polling more carefully for a moment.  Rudd was knifed while ahead on 2PP polling.  The primary polling-based trigger for his demise was not the mid-2010 public polls (though they had shown him to be a political mere mortal and not the messiah he had been in 2008-9) but rather a suspicious set of ALP internal seat findings that were given away to Andrew Bolt and have still never been released in any examinable form!  Gillard in 2013 was so far behind so close to the election that no-one needed 2PP to know Labor risked a massive loss if it did not remove her.  Turnbull cited Abbott losing 30 2PP Newspolls in a row, but was then allowed to lose 38 himself before being removed.  The catalyst for his downfall later was not polling specifically (though it didn't help) but mediocre performance in the Super Saturday by-elections, followed by internal fighting over climate policy.  So really the case here is all about Turnbull, and does anyone really think that Turnbull putting Abbott's hapless Prime Ministership out of its misery was a political event that was "distorted" by polling and would have otherwise not have occurred?   (I also mention that Hawke was knifed when most polls weren't even publishing 2PPs between elections.)

The other major problem with effacing the 2PP is that it doesn't address the problem: the major polls predicted a 2PP win for Labor in 2019 because their primary vote estimates were wrong.  There was some shift in preference flow, but it played only a minor role in the overall error.  There was also the suspicious way that all pollsters converged on the same 2PP value (both across polls and within their own series) at the end, and this might reflect unconscious herding with the 2PP as the target (because the 2PP is the most important figure).  However, dropping the 2PP won't do anything to address issues caused by over-sampling engaged and under-sampling disengaged voters, if that is happening.  Nor will getting rid of 2PPs stop people from engaging in horse-race commentary - if anything it makes it more likely there will be too much focus on the primary votes, particularly Labor's.  

Of course, publishing primary votes makes it easy for people like me to calculate 2PP estimates off the primaries, and I would actually rather pollsters drop the 2PP and leave it to us than publish 2PPs distorted by respondent preferencing or other weird methods that might then skew to one side.  (The report wants to have it each-way by still saying "The national trend is strong enough to give Labor a small advantage over the Coalition in two-party terms, but the lead is within the margin of error of 2.2 per cent for the poll [..]" but then not actually publishing the figure!)

What I find silliest about the justification for dropping 2PP is that the Age and SMH are continuing to publish Preferred Prime Minister.  Better PM/Preferred PM polls skew massively to incumbents (see here for Newspoll but almost all other polls that report it are the same) and fuel bogus narratives in which even if the parties are very close, the Prime Minister almost always seems to be flogging their hapless opponent.  In a country where incumbent PMs so often win and recover from almost any level of bad 2PP polling, if anything is an obvious suspect for polling distorting political commentary it is the continued use of these metrics without disclaimers about their history of overwhelming skew.  This may disadvantage Oppositions.  (The counter-argument is that Better PM scores capture a truth about government advantage that 2PP polling misses for some reason, but if this was so they should have a stronger relationship with actual election outcomes than they do.)

Some comments by Jim Reed should also be mentioned here;

"Many people aren’t thinking about their vote right now, let alone the people they might give their fifth or sixth preference to, and you can win with less than 50 per cent of the TPP vote anyway.”

The former is true, but it's a reason to avoid respondent preferencing, not a reason to avoid 2PP estimates, especially since last-election preferences usually work, and where they're likely to shift, evidence can be obtained from other sources like state elections.  The latter is also true (it most recently happened in 1969, 1990 and 1998) but I see it more as a reason for being upfront about the limits of 2PP as a predictor than for avoiding using it at all.

Primaries and 2PP estimates

The primary votes in the first run of the new poll are Coalition 38 Labor 33 Green 12 One Nation 6 IND 8 Others 3.  As with the former Ipsos poll, this poll has a high vote for minor parties and independents compared to other polls and recent elections.  

One Nation polled 3.1% in the Reps in 2019 (where they contested only selected seats) but 5.4% in the Senate.  The main reason for the discrepancy is that One Nation do not currently contest every seat.  As they tend to do slightly better in a Reps seat they contest than in the Senate in the same seat, it's quite possible they'd get 6% in the Reps if they contested every seat.  (That is, assuming their vote did not crash like it did in recent state elections.)  Newspoll adjusts for this by only giving them as an option in divisions they've contested.  Not adjusting for it is likely to drag the major party primaries down compared to what would actually happen - unless One Nation started running in every seat.  

Independents polled 3.44% in the Reps in 2019, the highest level since 1943, but 3.44% still isn't 8%.  When Independents are offered as a standalone option in a relatively short list of options, there's some potential for their vote to be overestimated between elections.  The reason is that voters might ideally want to vote for an independent but then find there isn't one on the ballot paper who they have heard of or who is the sort of independent they would like.  Vote-parking in other terms.  This was seen for instance in the 2013 leadup where the Nielsen poll at one stage had independent on 6% during Labor's leadership ructions.  Independents ended up only getting 1.37%.  

The question is whether those voters choosing "independent" are as left-leaning as those who voted independent in 2019 (59.4% to Labor) or not.  Noting that the poll has "others" excluding One Nation and indies at only 3%, but such forces actually polled 8.37% in 2019 (3.43% UAP) it's probable that voters for a range of little parties are picking the Greens, One Nation or Independents because those other parties are not listed.  

There are a few different possible approaches to preferencing here.  One is to treat all the Independent and Others voters as if they were generic Others voters in 2019, and this gives a 2PP estimate of 50-50 (actually 50.1-49.9 to Labor). Another is to treat the Independent voters as actually intending to vote Independent and in this case I get 51-49 to ALP.  I will be using the former approach mainly but also mentioning the latter.  Things can be made still more complex by assuming United Australia has collapsed as a share of Others votes (polls sampling them generally get them around 1%), but given that their voters may have gone over to One Nation in this poll, I'm reluctant about doing that.  

The full report includes age and state breakdowns of voting intention, which I haven't looked at in detail yet.

Other questions

I've already groaned about Preferred PM (in this case 47-25 to Morrison) but there's an interesting approach to approval ratings with the question being, eg "How would you rate Scott Morrison’s performance as Prime Minister in recent weeks?"  The detailed reporting also includes comments from respondents which make it clear that some at least are ignoring the "in recent weeks" part and still referencing events as long ago as the 2019-20 summer bushfires.  I suspect Morrison's supporters are doing the same thing.  Anyway Morrison kicks off with a net +12 (50-38) and Albanese with a net -6 (35-41), not wildly different from other approval polls.  

There are many issue questions that ask "Which party and leader you think would perform best in each of these areas."  I am not sure whether it's more useful to mention the leader or whether that imports some of the skew present in Better PM polling.  A bigger issue I have is that the options given include the major parties and someone else, and that this loses information about how voters compare the major parties on at least some issues (perhaps all).   On handling of the environment, the Coalition is only one point behind Labor (23-24) but 24% prefer "someone else".  

Based on Essential's polling of similar issues, that "someone else" will tend to be The Greens.  In December 2015 Essential included the Greens as an option and found Labor trailed on the environment and climate change by 3 points.  In March 2016 Essential stopped including the Greens and Labor jumped to 8-9 point leads on these questions.  Taking away the Greens also saw the Coalition's lead on asylum seeker issues drop from 15 to 5.  On questions that were not key Green issues, the average change in gap between those polls was just 2.7 points in Labor's favour, resulting from the Turnbull honeymoon abating.

Based on this Essential example, the "someone else" option is likely to be obscuring a significant, but not huge, lead for Labor on green issues over the Coalition in a two-party sense.  Whether that lead is any use for Labor is another question, and on the evidence of 2019 I'd say probably not.

Anyway it is great to have another major poll active.  It's just a shame that again we see a response to the polling failure (dropping the 2PP) that appears to be driven by cliched commentariat and reader concerns, and then is not even consistently applied.  If we are concerned about the potential of polling to interfere with politics, then surely Preferred PM polls need to be first against the wall.