Media Fail Again Over MRP Reporting

Redbridge/Accent MRP model projects Coalition would be likely to have the most seats if the federal election was held now.  

It does not say Coalition would be likely to have a majority.

It does not predict the result of the election.

---------------------------------------------------------

I thought an article was in order to unpack some of what is going on with the recent Redbridge/Accent MRP model and the woeful reporting of it by pretty much every outlet that has so far mentioned it.

MRP models (stands for Multilevel Regression with Poststratification) are not the easiest to explain to lay readers at the best of times, but what we have seen from several media sources reporting on this one goes beyond understandable confusion and into the realms of reckless innumerate false reporting.

What a MRP model does is to build a picture of how certain types of seats are likely to vote based on small samples of all 150 electorates.  Although each seat's sample is uselessly tiny by itself, by assuming that seats that resemble each other in ways that affect voting intention will vote similarly, one can smooth out a lot of the rough edges in the sampling, and samples of a few dozen voters per electorate can build a model that's about as good on a seat-by-seat basis as if those samples were actually a few hundred.  That still isn't very good on a seat-by-seat basis, but on a nationwide basis, the model could capture some general trends about the kinds of seats where each party is likely to be doing well or badly, and about how a party might be going in converting vote share to expected seats.  

Crying Wolf: More Embarrassingly Bad Tabloid Poll Reporting

 "Anthony Albanese could be on track to being a one-term Prime Minister, with a new poll showing Labor's primary vote crashing in three major states.  The federal government is in serious trouble in the eastern states - where most of the seats are - with Labor down to 24 per cent in Queensland, 28 in Victoria and 32 in NSW."

"Labor’s primary vote has crashed to just 24 per cent in Queensland, 28 per cent in Victoria and 32 per cent in New South Wales, the wolf + smith shows.  But Labor is dominant in South Australia, where its primary vote is 41 per cent, and 60 per cent on a two-party-preferred basis.  The poll – which measured both state and federal voting intention – suggests the government is in dire trouble in the eastern states, with just 43 per cent of the two-party-preferred vote in Queensland and 48 per cent in Victoria."

This is how the Daily Mail and the Murdoch tabloids (Courier Mail/Daily Telegraph etc) respectively wrote up a massive new poll by an initially mysterious outlet wolf + smith.  But this was in fact another example of laughably incompetent poll reporting from these outlets, one that again happened to be in service of the narrative their right-wing readers would want to see.  What the poll in fact found is very different.  The state-level figures these outlets were commenting were state voting intention not federal.  This was made so abundantly clear in the poll report that, among other subtle hints, the whole of page 10 of the poll report is devoted to making it clear that the rest of the report is state not federal.

"Abyss" Piece Is Another Shanashambles

 

There is something badly wrong with the op ed sector of the Australian media.  Many outlets continue to pay for articles that are factually shoddy and that reinforce the prejudices of partisan readers rather than helping their readers to actually understand politics.  It's perhaps a vicious cycle in which some outlets think there is no market for anything better than an echo chamber, and by offering a bad product ensure they'll never find out otherwise.

The latest hopeless offering by Dennis Shanahan (screenshot above) is an example.  To be clear, nobody should deny that the Albanese Government has been shedding public support rapidly in recent months. Anyone in Labor's support base who is not at least a little bit concerned about this is a fool. 

Mid-Term Queensland Polling, And The Misreporting Thereof

Just a reminder before I start this one that tomorrow is Button Press Day in Victoria and the action can be followed on the thread below this one.  It's not really democracy but it can still be fun!  I am working on some wrap-up material for the Victorian lower house too, but am being held up by some unclarities regarding 2PP issues (mainly an apparent error in the seat of Pascoe Vale.)

Yesterday Queensland was treated to not one but two state voting intention polls, but also, alas, to some of the worst poll reporting I have seen.  In the recent Victorian state election the Herald-Sun engaged in absurd poll-spinning and stairs-fall-truthing only to embarrass itself completely as the eight-year old Andrews government returned with a slightly increased majority (the oldest state government to do so since 1986).  The Courier-Mail, which has kept playing the same silly game although the side it barracks for has won just one and a half of the last twelve elections, also seems determined not to learn. 

Old Rope: A New Low In The Australian's Newspoll Commentary

With a quietish day in the South Australian count so far, I just thought I'd take the time to call out an article about Newspoll that was published yesterday in The Weekend Australian.  

The article was entitled "PM wins back women's support" and appeared on page 6 of the Weekend Australian.  It commenced with the following claim:

"Scott Morrison has recovered lost ground with women voters in recent months and has built up a larger lead over Anthony Albanese on the question of preferred prime minister among women than men, despite an onslaught of criticism on various gender ­issues."

Other claims included in the article were:

"Although the Prime Minister lost his lead over the Labor leader in Monday’s fortnightly Newspoll, Mr Morrison consistently outranks the Opposition Leader on satisfaction ratings among women voters, an analysis of quarterly demographic breakdowns reveals."

From this I expected that the Australian was releasing a fresh quarterly breakdown for the January-March period.  But no cigar (there might yet be another poll to come this month and be included in the breakdown anyway).  Here's some more:

"But Newspoll surveys show that, at the end of last year, Mr Morrison had a lower dissatisfaction rate among women than men and the same net satisfaction rate – the difference between voters’ satisfaction and dissatisfaction – among men and women."

"But, on the question of who would be the better prime minister, Mr Morrison has had a bigger lead among women since late 2020 and has always had a better net satisfaction rating among females."

(I take it that the word "he" has been omitted before "has always"  Otherwise it makes no sense as a better net satisfaction rating is not relevant to who is the "better prime minister".)

Poll-Shaped Objects: January 2022

Welcome to a new irregular series for this site.  From time to time I will do a post that covers and comments on all of the poll-shaped objects that come to my attention in a given month.  Where relevant, the article may also discuss real polling on the same issue.  Once I've decided there is enough material and time to put out an issue for a given month, all others that I see in that month will be added to the piece.  However I won't necessarily put out a PSO article every month; usually I will release such articles when there seem to be far too many of the things around and I feel that the proper order of things needs to be restored.  An early version of this idea can be seen in a 2017 article Poll Roundup: Attack Of The Poll-Shaped Objects.

The term poll-shaped object is one that I use to deride the overly credulous reporting of unsound or insufficiently transparent polls in media articles, and also the peddling of such polls by pressure groups, so-called think tanks, parties and other groups that cling to the foolish belief that you can influence public opinion by telling fibs about it.   A PSO can, for instance, be any of (i) something that is claimed to be a poll but isn't (ii) an especially unsound poll or (iii) a possibly sound poll for which the level of reported detail has been woefully lacking.  (The term is by analogy with piano-shaped object, something that appears to be a piano but sounds horrible or is impossible to play - whether by reason of shoddy construction, mistreatment or decay.)

The Major Parties Are Not "Neck And Neck" In Victoria

RESOLVE PM (Victoria state) Labor 37 Coalition 36 Green 9 IND 12 Others 5

2PP Estimate 54-46 to Labor.  If numbers repeated at election, Labor would win easily (c. 50 seats)

"Independent" vote very likely to be overstated

-------------------------------------------------------------

I've had too little time for writing here in recent weeks, largely because of a backlog of contract work that I had to clear after it built up during the Tasmanian snap election.  There are a few pieces I have been working on that I do hope to finish some time but they will be well behind the news cycle should they actually appear.  However, I wanted to make some comments about reporting of today's Resolve Political Monitor poll of Victorian state election voting.  This furthers a concern I have had about some responses to the 2019 federal polling failure - that some media sources that commission or work with polls have responded with trendy solutions that lose information and then lead to worse reporting of what polls are actually claiming to show.  

In my initial coverage of Resolve's entry to the major polling markets I noted that fingering two-party preferred figures as a major culprit in the 2019 federal polling failure was simply incorrect: polls were wrong overwhelmingly because their primary votes were wrong, with errors in preference estimation making only a small contribution to the failure.  I doubted the claims that this would deliver readers "something deeper" than "the “horse race” nature of the way we reported the results of TPP questions" and suggested that what this would actually lead to was journalists reporting the horse race off primary votes.  In the case of the reporting of this specific Victorian poll, the horse race commentary hasn't gone away, it's just got worse.  It's like being told just the relative positions of the horses near the end of a race without being told one of them is flagging and the other is charging home strongly.  

AsiNine: NSW Labor Not At Hundred-Year Low

With much attention on the Western Australian election and also a rather interesting Newspoll this weekend, unfortunately there was a third story that saw some of the most lamentable poll reporting that has been seen from a major network for a while.  I've put the other matters aside for an hour or two now, because what has happened here needs to be strongly condemned.

The first I saw of this poll was a news item tweeted by the Nine News Sydney account, concerning a disaster poll for NSW Labor, one supposedly finding that the party's primary vote was lower than at any election since 1904, and worse even than the 2011 debacle.  The Twitter video of the news report was three minutes and 18 seconds long.  I watched it three times in disbelief that a report of such a momentous poll did not even name the pollster involved, let alone state who had commissioned the poll.  Then I ranted about it on Twitter.  Then I watched it a fourth time just to make sure that I hadn't just made a real fool of myself.  But it was real - 198 seconds complete with interviews with ex-Premier Morris Iemma, but no mention of the pollster's name.  This alone was quite ridiculous.

(I've been told there is a longer version that does name the pollster, but haven't seen it, and don't know how widely versions with and without the pollster named were broadcast.)

White Goes First, Right Goes Beatup: The ABC Did Not Attempt To Cancel Chess

In recent days I've been involved in a media and social media flurry sparked by the ABC's decision to explore the subject of whether White moving first in a game of chess was in any way connected to race issues.  This claim was once most commonly seen as a spoof of anti-racism campaigns, but these days, a small number of people seem to be actually fearing chess might be symbolically racist.

I appeared on ABC radio and gave an interview that outlined that there is no evidence this is the case.  The host did not try to argue that there was, just mentioned that people on social media have held concerns about the issue.  The mere existence of that interview has triggered a massive backlash from right-wing culture warriors, which had already started before the interview aired.  The thing is, it is unclear that the enemy they're tilting at exists!  The ABC may be guilty of filling up its programs with offbeat fluff on the slender pretext of a few tweets, but that does not mean it was trying to have chess cancelled.

UK 2019: Win For Polls And Tories, A Shocker For The Left And Centre

After spending yesterday commenting on the UK election on Twitter, I think it's time to put some longer-form comments down about the result in light of the sorts of themes that get covered here.

The result was of course a resounding win for Boris Johnson's Conservatives, with 365 of the 650 seats.  The major seat movements were in England, where the Tories took nearly 50 seats from Labour.  There were also movements in Scotland (where the Scottish Nationalist Party took seats from both major parties), Wales (where Labour lost more seats to the Tories) and Northern Ireland (where the Democratic Unionist Party lost two seats to nationalist parties).  The Tory vote didn't increase greatly, but the Labour vote collapsed.  

There will be plenty of detailed accounts available of the characters of areas that broke to one party or another.  As with Australian analysis, when reading these always beware of the "ecological fallacy" - an electorate with a lot of voters of type X swinging to party Y does not always mean that voters of type X themselves swung to party Y.  The obvious hook (and one not so subject to this problem as some) is Brexit, and unsurprisingly strong Leave areas swung to Conservatives and strongly against Labour, while strong Remain areas swung more modestly against Labour and weakly against the Conservatives.  However the relationship between Brexit position and swing was messier in Labour's case.  Correlation-hunting has unearthed this remarkably strong link between swing and blue collar jobs, so it will be interesting to see where that debate goes.  

Braddon And Bass 2019: Another Rec Fishers Preferences Beatup

One of the eternal tropes of Australian media electoral reporting is the breathless expose of how the preferencing behaviour of some obscure party or candidate could swing or did swing an important contest or in cases an entire election.  And, on a day when there was quite enough going on for election buffs to look at in the Chisholm and Kooyong signs challenge (see my updated coverage of that on the link) it has unfortunately broken out again with the sensational headline "CFMMEU-funded independents helped Liberals steal two key seats".  (CFMMEU = Construction Forestry Maritime Mining and Energy Union).  The article goes on:

"The CFMMEU helped the Coalition win two key seats back from Labor at the May election by funding the campaigns of two independents who sent 1757 votes between them to the Liberal Party."  It then refers to independent recreational fishers Todd Lambert (Bass) and Brett Smith (Braddon).

The piece has been widely criticised on the Twitter psephosphere, but not everyone uses Twitter, and especially as the seats in question are Tasmanian I think it's worth posting a detailed explanation here of why this piece is incorrect.  I note that no psephologist was interviewed for the article.

Migrants Voting For One Nation and UAP? (Plus Some Polling Comments)

A Few Pointed Words About Polling

Before I start this article, a quick four paragraphs regarding polls, in lieu of a formal roundup.  Firstly and mainly for Tasmanian readers, there has been some (disappointingly uncritical in some cases) media coverage of a poll commissioned by The Australia Institute (Tas) concerning proposals for a Tarkine National Park.  Unfortunately the poll is simply totally unsound.  It uses a loaded preamble that gives arguments for one side of the debate, quantifying a claimed total clearance area while not quantifying how much of the area included is really old growth or rainforest, and further leading the respondent with a comment about claimed community and business support for a National Park.

Having been presented with just one side of the argument, respondents may well be led to give the answer that suits the sponsor, or may well be driven to just hang up if they don't agree with the statements made.  The poll report also provides no data whatsoever on other questions asked, disconnection rates or on the methods and extent of any weighting used to obtain the final results.  The question design also fails to establish whether any support for a National Park would be in addition to some logging activity or as an alternative to it.  Maybe voters really do support a Tarkine National Park of some kind, maybe they don't (the risible voter support for the Greens in Braddon in recent years is not the most promising sign)

I have been trying to write about polling more generally but it is very difficult to get the job done with any motivation when leading pollsters, with the sole exception of YouGov's Queensland polling, have thus far done virtually nothing about the pressing need for a major improvement in polling transparency following the 2019 Australian polling failure.  As such there is no basis for confidence that Newspoll's current picture of a close federal race is in any way accurate (the Coalition's 51-49 leads might really be 54-46 or more, or alternatively Labor might be in front, though that is much less likely.)  And since Essential keeps suppressing its voting intention figures although its unsatisfactory reason for doing so long ago expired, there is no way to benchmark any of its leadership polling, and its issues polls are often problematic.  

Media coverage of commissioned polling also continues to be as awful as before.  Some recent amusing nadirs were rival YouGov poll results being cited and uncritically reported by friendly media on both sides of the NSW abortion debate, and also the Your Right To Know campaign claiming to have Colmar Brunton polling supporting their position, but failing to publish the details of the polling.  If you want to scrutinise it, you can't - you just don't have the right to know.  Media are rightly, if in some cases hypocritically, concerned about laws that can unduly limit what public interest information they are allowed to report. But the claim of media outlets to be servants of the public in reporting public interest information is undermined when they so frequently fail to report relevant information or cautions about their stories when they could and should, largely for reasons of laziness and the back-patting of sources who have fed them material for easy articles.

On to the main course ...

There has been quite an amount of interest in an ABC article by Stephanie Dalzell that claims that migrant voters are increasingly voting for populist right outfits like One Nation and the United Australia Party. Of course, some migrants will vote for these parties, but the article is not a useful contribution to establishing how many.  

How Can Australian Polling Disclosure And Reporting Be Improved?

Australian national opinion polling has just suffered its worst failure in result terms since 1980 and its worst failure in margin terms since 1984.  This was not just an "average polling error", at least not by the standards of the last 30+ years.  The questions remain: what caused it and what can be done (if anything) to stop it happening again.

A major problem with answering these questions is that Australian pollsters have not been telling us nearly enough about what they do.  As Murray Goot has noted, this has been a very long-standing problem.

In general, Australian pollsters have taken an approach that is secretive, poorly documented, and contrary to scientific method.   One notable example of this was Galaxy (it looks like correctly) changing the preference allocation for One Nation in late 2017, and not revealing they had done this for five months (in which time The Australian kept wrongly telling its readers Newspoll preferences were based on the 2016 election.)  But more generally, even very basic details about how pollsters do their work are elusive unless you are on very good terms with the right people.  Some polls also have statistically unlikely properties (such as not bouncing around as much as their sample size suggests they should, either in poll to poll swing terms or in seat-polling swing terms) that they have never explained. 

Threatened-Listed Species And The Proposed Cable Car

The debate about the proposed cable car on kunanyi/Mt Wellington has already been a dismal spectacle of false claims and questionable standards on both sides.  On this site I have already dealt with claims that concern polling or other poll-shaped objects that claim to mention public opinion (see the rolling Polling on the Mt Wellington Cable Car Proposal article and the earlier Public Opinion and the Mt Wellington Cable Car.) 

Now it is time for me to post a new article covering the already suspect claims in the area of threatened species impacts, in the hope of deterring any more threatened species nonsense and encouraging everybody involved to actually do some research.  While this is mainly a psephology site, most of my professional income comes from working on invertebrates of the non-political kind, and I am Tasmania's only living expert on native land snails.  I also have a keen amateur interest in native orchids, and have worked on or surveyed for a range of threatened species of various kinds.

The catalyst for the current burst of threatened species claims is a proposal by the cable car proponent, the Mt Wellington Cableway Co, to have its proposed cable car depart from a site on Main Fire Trail.  This proposal includes a new road from McRobies Gully (see route maps here and here) in an area of bushland that includes extensive areas of a state-listed threatened vegetation community (Eucalyptus tenuiramis on sediments.)

"Margin Of Error" Polling Myths

A lot is said about "margin of error" in public discussion of opinion polls, and nearly all of it is wrong.

This is another piece I've been meaning to write for quite a while.  There are many other articles about this on the web, but I'm not aware of any that make all the points I'd like to make and make them in an Australian context.

The concept of "margin of error" is one that is commonly talked about in discussion of polls.  It is often used to (rightly) deflate breathless media babble about movements in polls that are supposedly caused by whatever the media pundit wants to see as the cause, but in practice are often nothing more than random variation from sample to sample.  Possum's Trends, The Horserace and Random Numbers (2012) was a classic debunking of that sort of commentariat nonsense, written in the days of the older, much bouncier Newspoll.

Unfortunately, of all the things that get talked about in polling, margin of error is probably the concept that best shows that "a little learning is a dangerous thing".  People grasp the basic principle and misapply it constantly - and are assisted by pollsters, the media and even the Australian Press Council's otherwise good (and too often disregarded) poll-reporting guidelines in doing so.

Poll Roundup: What Is Going On With Newspoll Preferences?

2PP Aggregate (Last-election preferences): 52.4 to Labor (-0.4 since last week,  -1.2 points in five weeks)
With One Nation adjustment, 51.8 to Labor 
Closest position since May last year
Labor would very probably win election held "right now", but would probably have a small to moderate majority

[Updated on 26 April at bottom of post]

------------------------

Normally I issue Poll Roundups every second Newspoll. Since the last one there's been the slight distraction of the Coalition's 30th consecutive 2PP Newspoll loss. This week's Newspoll was number 31 in a row, just two shy of equalling the longest losing streak held by Labor under Julia Gillard, but it was more significant for the discussion it has sparked about (i) Newspoll's preferencing methods (ii) the prospect of a Coalition recovery.  So firstly, a detailed discussion of the preferencing issue, and then a slightly shorter discussion of where things now stand in federal polling.

Newspoll as a brand name linked to a standard set of questions has a 33-year history in Australia.  However, in 2015 the company previously doing Newspoll for the Australian was wound up.  The Australian retained control over the brand, but its operation was contracted out to Galaxy Research (which in turn was recently acquired by YouGov).  While the questions didn't change when Galaxy took over, the methods did, with a mixture of robopolling and online polling replacing the old Newspoll's landline-only live phone polling, and an increase in average sample size.

Recent Polling On The Same-Sex Marriage Postal Survey

The national ABS postal "survey" on whether the law should be changed to allow same-sex marriage in Australia is now in its second week.  A number of pieces of polling have been published or alluded to since my last general polling update, but what do they really tell us about the outcome and how reliable are they?  At this stage there is still much that we do not know.  It is too early to be certain Yes has it in the bag, but the widespread narrative that support for Yes is crashing rapidly and that this is another Trump or Brexit coming is so far not that well supported by the evidence.

Public Polling: Ipsos

Firstly, the public polling.  Last week saw an Ipsos poll which buoyed some worried Yes supporters with a 70-26 Yes response, one of the highest Yes votes ever recorded in a poll in Australia.  Indeed, as far as I'm aware, this score has only been exceeded in a few commissioned polls and one Morgan-SMS (a suspect polling method) which did not use an undecided option.  The Ipsos also found a 70% Yes response among the 65% of voters who rated themselves as certain to vote, and found a gender gap with 72% of women and 59% of men saying they were certain to vote.

The UK And Australian Elections Weren't That Similar

Last week the UK had its second straight surprising election result.  In 2015 an expected cliffhanger turned into an easy win for the Conservatives while in 2017 an expected landslide turned into a cliffhanger.  The government went to the polls three years early (that's a whole term over here), supposedly in search of a strong mandate for its position on Brexit, yet came away with fewer seats than it went in with.  The real motive seemed to be to turn a big lead in the polls into a bigger majority, and if that was the aim then it backfired spectacularly.

In the wake of this result the Australian commentariat have put out several articles that seek to stress parallels with Australian politics.  The primary themes of these articles are as follows: that Malcolm Turnbull is Theresa May and that Anthony Albanese is Jeremy Corbyn.

Let's start with the Turnbull-May comparison.  Turnbull has no hope of winning the battle of perceptions on this one, because it's the kind of analogy many of those who consume political chatter will congratulate themselves on having thought of first.  But on a factual basis, the comparison is twaddle.

Poll Roundup: Are Budget Bounces Ever Real?

Aggregate: 52.7 to Labor (+0.2 in two weeks since last completely pre-Budget reading)

Labor would easily win election "held now"

Voter budget ratings are historically about average

Commentary around the 2017 federal budget has been even more focused than normal on one of the Australian beltway polling obsessions, the idea of a "budget bounce".  The Coalition's moves on the National Disability Insurance Scheme and the announcement of a bank levy, in particular, were seen as bold moves to seize the middle ground from Labor in an attempt to make the Government popular again.  Some even saw this as a threat to Bill Shorten's leadership of the Labor Opposition.

That at least a temporary bounce was widely anticipated (and also widely seen as the aim of the game) is one thing, but another has been the common blight of house-poll myopia in which commentators from particular stables obsess over the single-poll-to-single-poll changes in the poll their network commissions, while ignoring both the aggregated cross-poll trend and even the longer history of their own poll.  In the case of Newspoll, it didn't help the Government that their previous result (48-52) was a shade on the friendly side of the recent run of polling, thus setting a pretty high benchmark for any budget bounce to be assessed by News Ltd scribes from.

Poll Roundup: Attack Of The Poll-Shaped Objects

2PP Aggregate: 52.8 to Labor (-0.8 since last week)
Labor would comfortably win election "held right now"

This week has been a confusing week for many poll-watchers, and an amusing week for those of us who watch the antics of the poll-watching partisans.  Newspoll was eagerly anticipated following its 55:45 to Labor three weeks ago, and widely expected to come out with more of the same if not then some, but pulled up at only 52:48.  Oh well, the line went, maybe it was never 55:45 to begin with, after all Essential had only got out to 53s and the odd 54.  But then Essential jumped to 55:45, so two polls not usually noted for volatility had delivered it not just in spades but also in opposite directions.  At the moment we don't have a third opinion, since the others are very inactive lately.

In trying to decide between these competing figures, it is worth bearing in mind that Newspoll is now administered by Galaxy.  Galaxy was the best pollster of the 2013 election, and the Galaxy/Newspoll stable triumphed again in 2016.  Essential was poor in 2013, and while its final poll in 2016 was very good, its tracking performance suggests it was probably lucky or herding.  So my aggregate comes down more on the side of Newspoll, crediting Labor with 52.8% 2PP.

Anyway, that is a recovery of sorts, but we have seen false dawns before.  Here's the smoothed tracking graph:

For anyone with an interest in my input values, this Newspoll went in at 45.1% for the Coalition after considering the primaries, and the last three Essentials have been 46.9 (published 47), 46.7 (47) and 45.2 (45).