2016 Ehrlich Awards For Wrong Predictions

It's time for the fifth annual giving of the Ehrlich Awards, which round the start of each year go to the most amusingly or staggeringly wrong predictions I observe in any field of interest relating to the previous twelve months.  The Ehrlichs are named for Paul Ehrlich, the evangelist of ecological end-times who put me on a path to a lifetime of health scepticism of dark green gloomery when he not just lost a famous bet with economist Julian Simon but also gave poor excuses for the defeat. For the past editions click the Ehrlich Awards tab at the bottom, and for the ground rules see the 2012 edition.

As usual I should note my own predictive efforts are hardly perfect, but I had a pretty good year in 2016, missing by only two on the Coalition's national Reps seat tally (for example).  I did wrongly predict two Reps seats in my own home state, which was embarrassing, though I did indicate those were quite uncertain.

Two areas of widespread predictive failure that will dominate these Awards were the US election and Senate reform.  They weren't the only ones I noticed.  For instance the late Bob Ellis made a bold bid for posthumous glory with "It is likely, though not certain, that Malcolm Turnbull will lose his seat" - Turnbull won his seat 68:32 with a trivial 1.2% swing against him.  But I think the two I've mentioned are the most interesting ones and I've decided to make the US election case the dishonourable mentions, saving Senate reform for the medals.



In the US election, poll aggregation methods systematically failed to predict the winner.  They did so because while they had the national vote picture broadly correct, polling in a few swing states was wrong and Trump won these states by narrow margins.  It isn't clear how any analysis of opinion poll data, whatever adjustments it applied from past results, could have got it right - some elections are difficult to predict and sometimes an underdog wins, seemingly no matter what you do.  Had the national polls been wrong by a similar average amount in Australia's federal election, we would have had a hung parliament as in 2010, and we would now be having much the same conversation here, especially if Labor formed government.

There is hence a lot of hand-wringing in the US about whether poll analysis is any use at all, with one takedown of Nate Silver (language warning on that link) insisting polling analysts are "useless in a crisis. They don’t understand anything that’s going on around them, and they’re powerless to predict what’s about to happen next."  In fact, it's not that clear anyone else really knew what would happen either.  Those who "predicted" Trump's win often did so using overfitted (and sometimes retrofitted) models or strange polls that actually said Trump would win the popular vote (which he didn't.)  Successful predictions for the right reason using repeatable models that had worked before were very rare, if there were even any at all.  And the way Trump won - from where he was in the final polls - was nothing so exotic that it had to be attributed to the strange nature of Trump (or Clinton) as a candidate.  It was the kind of polling and modelling error that could have happened with any two candidates in a fairly close race.

If there's a lesson for polling analysis in the Trump result it's mainly that local polling is very untrustworthy, and we need to measure how a campaign is resonating demographically and use that in connection with national polling to try to predict local variations better.  In Australia, this would have given us pretty much the same result (and poll modelling generally had the Coalition only slightly above the seat tally they actually won) but could well have led to a much better seat-by-seat prediction.  In the US it might well have made all the difference.

But giving the eventual narrow winner a 30% chance isn't a major predictive failure.  Giving them a 2% chance of winning their party's nomination while they're leading in the polls is, and while Nate Silver's mea culpa addressed a lot of reasons why this was a silly thing to say, I think it overlooked an important one.  The chances of Trump getting through each of Silver's six stages of doom were nowhere near independent of each other. If attention fails to focus on other candidates in Stage 1, for instance, it very probably won't do so in Stage 4 either.  I really enjoy a lot of what FiveThirtyEight does (especially their group chats - can we have something like that here for Australian elections?) but this was a dire year for the site's credibility.

Many others were worse than FiveThirtyEight in the general election, and I'm not entirely sure what Sam Wang did wrong to give Clinton a 99% win probability but it should be obvious that in a close-ish race, if your model doesn't sometimes throw up a batch of states that buck the trend and spoil the party based on locally corellated errors, then your model is unrealistic.  Wang topped off this blunder by not just backing his model in (the probability the model is just rubbish is always going to exceed 1% in any kind of unusual race) but telling people they should ignore the POTUS race and focus on other contests.  This kind of complacent thinking seems to have been widespread and fatal in Democratic ranks.

Another example of the same was the dreadful effort from Huffington Post's Ryan Grim, who tried an advance takedown of Silver on the grounds that poll adjustments (in fact based on empirical evidence of past performance) were "just guessing". What was especially Erlich-esque in Grim's piece was that he tried to make his attack on Silver unfalsifiable by claiming his critique was right no matter what the result: "If he’s right, though, it was just a good guess ― a fortunate “trend line adjustment” ― not a mathematical forecast. If you want to put your faith in the numbers, you can relax. She’s got this."  In fact, fivethirtyeight's final model was closer to the truth in its assessment of Trump's chances because its dealings with uncertainty were based on empirical evidence about it from past contests.

Senate Reform

I've already written quite a lot on this (click the Senate reform tab and especially see performance review part 1, part 2 and my 2016 JSCEM sub (PDF download)).

According to predictions from opponents, Senate reform was going to cause the Coalition to win at least a blocking majority, eliminate all parties other than the Coalition, Labor, Greens and NXT from the Senate, "disenfranchise" up to a quarter of Australians by causing their votes to exhaust (this is not, in fact, what "disenfranchise" means), cause a massive increase in informal voting, see nearly everyone just vote 1 above the line, advantage extremists specifically because more moderate candidates would not catch them fast enough on preferences, and be effectively first-past-the-post.

None of this happened.  The Coalition's seat tally went backwards and they would not have got near a blocking majority at a half-Senate election.  Three existing non-Green crossbenchers retained office and they were joined by four One Nation Senators and Derryn Hinch.  (Lambie, Hanson and Hinch would have won at a half-Senate election too based on the votes cast).  Informal voting went up a point in the Senate, but was still less than on the first run of the previous system, and it actually declined in the Reps.  Effective exhaust was around 5% (not a quarter as claimed), but even that 5% included parts of votes that had already helped elect someone - so not even 5% of all votes exhausted at full value.  The just-vote-1 rate was just 4.7% in NSW and below 2.5% everywhere else.  One Nation won seats not despite weak preference flows, but on primaries or because of good preference flows.  And two candidates won by overtaking others on preferences, in one case several others.

Before this happened the Australian Labor Party for some still murky reason left the tripartisan tent on the need for Senate reform (though internally this was far from unanimous).  I do know that some influential figures in the party were genuinely concerned about the Coalition getting a "blocking majority" and remained so (scars of 1975 perhaps) no matter what evidence was produced.  Australia was treated to the unedifying spectacle of what should have been an three-party process being obstructed for many hours by some very silly claims indeed.  Among the worst of these was the claim that if a person's vote did not reach anyone who was elected, then that person had been "disenfranchised".  In fact, disenfranchisement occurs when someone is denied the right to vote, not when the vote they choose doesn't actually reach anybody who wins.  Many voters who detest the major parties would be very happy about their vote not reaching either Labor or Liberal!

Anyway I have gone through the whole Senate debate - rather hurridly so I may have missed some juicy quotes - and assembled a list of predictions and implied predictions made by Labor Senators and that I consider to have been falsified by the results of the election.  I've left out those claims that merely alleged the Coalition was trying to create a three-party monopoly or a blocking majority or something else equally unlikely, as I cannot judge the Government's intentions.  I've also mostly left out those that were predicated on a cycle of many elections and have not yet proven false, though I am certain a lot of them will be.  Even with this strict culling, at least 14 of Labor's 25 Senators in the 44th Parliament issued at least one false prediction.  The silver medal for 2016 therefore goes to this team of 44th Parliament Senators (in alphabetical order) Bilyk, Cameron, Carr, Collins, Conroy (captain), Dastyari, Ludwig, McEwen, Gallagher, O'Neill, Peris, Polley, Sterle and Wong and a sample of their efforts can be downloaded here:

False Labor Senate Predictions (6 pp)

The bronze medal goes to Malcolm Mackerras.  Mackerras has been quite snarky towards some of his fellow psephologists through this process, so I'm sure he will not mind being singled out.  Unfortunately Mackerras was one of reform opponents' very few go-to psephs as a source of quotes during the farcical Senate debate, and why not with such juicy (and wrong) claims as:

"'It is not about fairness what is going on here. It is about the reshaping of our party system. South Australia is to have a four-party system, Liberal on the right, Xenophon in the centre and Labor and Greens on the left. The rest of Australia is to have a three-party system—coalition, Labor and Greens. There will be no independent senators, unless Jacqui Lambie can get a short-term at a 2016 double dissolution election."

(It is true that this prediction was made before the reform to below-the-line voting, but that reform had nothing to do with the success of those crossbench winners who did win).

Lambie in fact got a long term, and Hinch was elected as a Senator as independent (or not) as Lambie, not to mention the 4 PHON winners and the wins by Leyonhjelm and Day.  But it wasn't just that.  Mackerras predicted so many wrong things about the reform and election cycle.  He said repeatedly that there would not be an early election (there was) and on and off that there wouldn't be a DD (which there was).  He predicted Senate reform would be BTL only, then when it wasn't he dropped rather strong hints that the High Court might throw it out without below-the-line reform (it didn't go anywhere near doing so and clearly wouldn't have even had BTL voting not been reformed).  His Senate predictions were mostly reasonable, but he wrongly tipped Peter Madden to win in Tasmania, apparently solely based on Madden having opposed Senate reforms.   (Oh and he also thought Clinton had a 99% chance of winning.)  Not all Mackerras' predictions in the year were bad by any means, but what was striking to me was that when it came to the Senate reform process especially, he was "predicting" with the heart and not the head, more or less every time.

But our gold medallist had too much class for this lot and has created a work of art in the field of false prediction that will endure forever.  Politicians often predict they will win then falsify it by losing, but our winner did the opposite.  Many complained about the new system and what it would supposedly do, but only he (and a few obscure colleagues) took the government to court and told the High Court all about how minor party voters would be "disenfranchised" by the new system (using what appears to have been money he didn't have to do so).  Yes he even told them that:

"The practical operation of the challenged provisions is that approximately 75% of the electors in each State are enabled to elect 100% of the Senators for such state and in so doing disenfranchise 25% of the total number of electors voting in Senate election"

Having received the appropriate raspberry, our hero not only proved his claims before the highest court in the land wrong by winning his seat back, but did so in exactly the way his claims most implied would not happen - by polling the lowest successful party vote in the nation (2.87%), then overtaking a candidate - and from a major party at that - on preferences, including those from many other minor parties.

And our artist was not finished there!  He has since rendered the victory that disproved his own prediction (after all that effort) pointless by quitting the Senate within months, and it may even be that he was not eligible to be elected in the first place, for entirely unrelated reasons.  Perhaps even his decision to predict his own defeat before the High Court helped give him the publicity needed to prove his own concerns wrong, and perhaps even had he not so absurdly and prominently predicted he would lose, he would have done so.  I can only recognise this canonical masterpiece at the heart of the sheer absurdity of the whole vast anti-Senate-reform circus by declaring Ex-Senator Bob Day the winner of the 2016 Ehrlich Award for Wrong Predictions.