Media Fail Again Over MRP Reporting

Redbridge/Accent MRP model projects Coalition would be likely to have the most seats if the federal election was held now.  

It does not say Coalition would be likely to have a majority.

It does not predict the result of the election.

---------------------------------------------------------

I thought an article was in order to unpack some of what is going on with the recent Redbridge/Accent MRP model and the woeful reporting of it by pretty much every outlet that has so far mentioned it.

MRP models (stands for Multilevel Regression with Poststratification) are not the easiest to explain to lay readers at the best of times, but what we have seen from several media sources reporting on this one goes beyond understandable confusion and into the realms of reckless innumerate false reporting.

What a MRP model does is to build a picture of how certain types of seats are likely to vote based on small samples of all 150 electorates.  Although each seat's sample is uselessly tiny by itself, by assuming that seats that resemble each other in ways that affect voting intention will vote similarly, one can smooth out a lot of the rough edges in the sampling, and samples of a few dozen voters per electorate can build a model that's about as good on a seat-by-seat basis as if those samples were actually a few hundred.  That still isn't very good on a seat-by-seat basis, but on a nationwide basis, the model could capture some general trends about the kinds of seats where each party is likely to be doing well or badly, and about how a party might be going in converting vote share to expected seats.  

A standard issue with the Redbridge/Accent MRPs has been that they report probabilities of various outcomes, but those probabilities only apply if an election was held right now, and yet these statements are confused for predictions - something I think the pollsters involved could do more to ward off in their reporting and publicity.  A model that finds that in an election held right now there would be a 98% chance of a no-majority parliament does not mean the chance of a no-majority parliament when the election is held is anywhere near that high - it's a nowcast, not a forecast.  Media misreporting of it as a forecast gives the false impression that the outcome is already more or less locked in, when in fact five and a half months (if the election is in May) is still plenty of time for things to change a great deal.  (Also, this is only one model, albeit a large one.  In theory, a different pollster using a different panel and weightings but doing the same exercise might get a more substantial but probably still low chance of majority government now for one side or both.)

Again assuming a May election, historically on average polls taken now have been about 3% 2PP away from the actual result, and limiting things to more recent elections and higher quality polls doesn't change a lot there.  (The average difference between results and Newspolls taken five and a half months out has been just under 2.5%, with governments on average improving by nearly 1% in that time.)  That said, I find with Newspoll at least that polls taken this far out tend to be closer to the result when the polls themselves are close.  Most federal elections these days are pretty close on the 2PP front, with only two of the last 13 outside 53-47.  So it's more likely to see a massive lead shrink than to see a close race blow out into a landslide.  Even so, if I halve the average error and say there's historically a 50% chance of polls taken now being within 1.5% of the final result, there's still plenty of room for one side or the other to win outright.

In terms of what this MRP is saying now, exact voting intention figures have not been published yet (there is a graph), but the report includes primary and 2PP figures for every seat.  After weighting these by October 2024 division enrolement multiplied by 2022 division turnout, I estimate the national primaries in this model as ALP 31.6 Coalition 38.9 Green 11.4 others 18, and the 2PP as 50.8-49.2 to Coalition.  (This is by last-election preferences which might be over-generous to Labor, but the Coalition seem to have done rather well on those for whatever reason - could include high One Nation or low IND votes, for example - as I get 50.2 as the expected 2PP for those primaries.)

Off such primaries after taking into account personal votes and allowing for otherwise random variation in seat swings I would expect something like 69 of the 135 current major party seats to go to Labor and 66 to the Coalition.  However, the model's median result (from a fairly wide range of possible outcomes) is 65 Labor seats and 71 Coalition.  The main difference here is that in the Redbridge/Accent model, the swing is significantly non-uniform; it falls more strongly in outer suburban seats hitting Labor in more marginals.  (MRP models also don't cater for personal vote effects to any real degree, so this might be a seat or so pessimistic even on that basis).  The MRP model also expects the Coalition to recover on balance one teal seat.  

The median parliament in this model would be fascinating.  Contrary to a fairly common myth, the Coalition wouldn't automatically form government or even necessarily get the first go at trying to just because it won the most seats.  Indeed, if the incumbent PM chooses to dig in and make those wanting to support an alternative government put their votes to it on the floor, Anthony Albanese could in this situation choose to "meet the parliament" and force the Coalition to find 75 votes for a no confidence motion against him if it wanted to govern.  Katter (but he may name a crazy price), Sharkie (unless she finds out what a 2PP count is), Dai Le (perhaps), Allegra Spender (maybe),  ... it's likely but it isn't straightforward.  Some of the teals and other independents might use the fact that the Coalition had won the most seats and the 2PP to justify supporting it, or might do so simply out of fear of oblivion at the following election.  Or they might not.  (Although their seats are just not that conservative anymore on a 2PP basis, the conservative ends of the teals' support bases may be more likely to turn on them for backing Labor than the left-wing ends would be for backing the Coalition.)

This is a similar picture to the previous Redbridge/Accent MRP, which also projected the government to be a few seats worse off than voting intentions implied, and for the same reason (outer suburban seats).  That's important because in 2022 Labor did very well with the 2PP distribution in the Labor vs Coalition seats, which means that with an even swing back the Coalition needs something like 51.3% to win the most seats.  But if the Coalition does better where it matters this time, that 51.3 can come down, and things can then get very messy for Labor if they lose the 2PP by any more than a whisker.  I've been of the view for some time that while a 50-50 2PP at the election would probably not be too bad for Labor, 49-51 is a rock they should not want to turn over.  

But still, the model's 71 is far from a majority.  Indeed the report finds that:

"The probability that the Coalition would have a majority in the House of Representatives is currently less than two per cent, and for Labor, essentially zero. If an election were held now, there is a greater than 98 per cent probability of a minority government."  (my bold)

So, you're telling Sky there's a chance ...

We have here a model that finds that the chance of anyone winning a majority if the election was held now is negligible.  But the media reporting on it didn't report it that way.  I am not linking to any of the media pieces in question here because they do not deserve any links from which they might obtain revenue.  Indeed some of the worst offenders should be banned from journalism until they have retaken the entirity of primary school.  

At the less severe but still bad end was the original Daily Telegraph version with a headline "Shock poll reveals Peter Dutton on track for major win at next Federal Election".  What on earth is a "major win"?  Major gain (in seat or swing terms) would have been a fair description, but a scraped minority government (and not certainly even that) is not a "major win" in the context of landslides or majorities, though it would be great going for a first-term Opposition.  The article started with "Peter Dutton is now a chance to win an outright majority in his own right at the next election [..]" before noting nearly 400 words later that that chance was under 2% in the model.  This is a good example of how the nowcast/forecast confusion leads to bad reporting - there has always been some chance of a Coalition majority at the next election simply because nobody really knows that it can't happen.  All that has happened here is that for the first time one of these MRPs has found that that chance would be non-zero in an election held now.  

Sky "News" was much, much worse:

"Opposition Leader Peter Dutton is on track to win a majority at the federal election next year, while PM Anthony Albanese is set to lose several seats, according to a new poll.

Accent Research and Redbridge Group conducted a survey of almost 5,000 people and mapped out the findings to predict the electoral map ahead of Australians voting.

It found the Dutton-led Coalition is on track to pick up at least nine seats and win an outright majority, pointing to a potential defeat for the Albanese government."

Again, the model is not a prediction, it's a claim about what would happen if the election was held now.  What the report does say about future tracking based on the "linear trend" is actually the opposite of Sky's commentary:

"The trends observed for the number of seats won by the Coalition parties and Labor are essentially linear, and now means that a majority government is unlikely."

There is absolutely nothing in the report that justifies Sky's claim.  Anyone with the slightest ability to add and subtract numbers knows the Coalition needs more than nine seats for a majority.

The Nightly, whatever that is, was equally useless:

"A new poll has revealed Opposition Leader Peter Dutton looks likely to win the yet-to-be-announced Australian election with a clear majority."

(Neither of these outlets in the report quoted from mentioned that the model's probability of a Coalition win now was less than 2%).  

The Daily Mail (probably saving its effort from greater embarrassment by not having Peter van Onselen write it) sub-headlined its article "Federal Labor heading for defeat" and carried on in such a vein when the report actually said nothing whatsoever about who would form government with what probability even now.  The model says only that the Coalition currently would have an 82% chance of being the largest bloc (which is not the same thing.)

Seven News managed to sanitise the less than 2% nowcast chance as merely "slim", and the poll was also mentioned on breakfast TV but one never expects to find anything with a brain there.  

There is far more that could be discussed in technical terms about MRP models, how accurate they are, how good this particular one is and so on.  (This model is the heir to the YouGov model which performed remarkably well in classic 2PP seats but less well in crossbench seats at the last election, but it is also a smaller sample.)  The main theoretical debate I have seen is over whether a sample of around 5000 with an average of 33 voters per electorate is really enough for the method to work its magic or not.  But in terms of public understanding, before we can even get to that sort of debate we need to get past the stage where incompetent, lazy and biased media companies churn out complete misrepresentations of what a poll is actually saying.  And we're not there yet.  Where we are instead is at the point where I dish out these.

image source

Wirrah Award For Fishy Poll Reporting

Awarded to Daily Telegraph and Daily Mail

Porcupine Fish Award For Ultra-Fishy Poll Reporting

Awarded to Sky News and The Nightly  (image credit)