A few days ago, Nate Silver of fivethirtyeight was the target of a terrible Huffington Post article and an argument broke out about whether it was more accurate to say Donald Trump had about a one in three chance of becoming President, or virtually no chance at all. HuffPo was to double down with this rather pretentious piece by a stats prof accusing Silver of overstimating Trump's chances - a piece that has proved to have an exceedingly short shelf life indeed. Silver's model might not look crash hot in the wake of what has happened, but it still looks a great deal better than those that were saying Trump had only a 1% chance of winning.
Despite a few requests from readers, I didn't have time to attempt a model of the US election. To do it justice would have taken months, and I don't see that I could have done any better from afar than the more credible of those who did in fact devote such time. But I do want to add some comments about what went wrong with even the good forecast attempts (most of them anyway), as there is a lot of curiosity in my (mostly Australian) audience about this. (Australian national polls in fact got it right at another federal election this year, but we seem to be about the last place left where this still happens.)
Firstly, of course, the popular vote polls nationwide for POTUS were wrong, but not by a very great amount. Final polls and aggregators had the margin for Clinton over Trump at around 3.5 points. In the current primary count Clinton has a tiny popular-vote margin which is currently at around 0.2 points and which may finish up around 0.6 points.
(Note added: Some sources are now projecting Clinton's popular vote margin as over 1 point and perhaps even closer to 2, which if true further underlines the point that national polling error was not that large by historic standards.)
FiveThirtyEight argued repeatedly that this was quite a probable scenario, having Clinton winning the popular vote but losing the Electoral College at a 10.5% chance in their final model, compared to just 0.5% for the reverse. The basis for this was that the Electoral College map was slightly unfavourable for Clinton.
Why the national polls went wrong will be the subject of much enquiry in the months to come. People have put to me the idea that it was "shy Trump voters" that did it, and there is lots of ammunition in the difference between live phone polling and other methods for that conclusion. But it could well also be that polls overestimated turnout for Clinton and didn't realise how many left-wing voters, still deflated by Clinton's use of the party machine to just overcome Bernie Sanders, might have said they were voting Clinton but then simply not shown up.
FiveThirtyEight noted that a national three-point polling error in Trump's favour (about what happened) should lead to a more or less tossup race. Had there been a uniform swing in the popular vote margin of the size that we have seen, Clinton would have lost Nevada but saved Wisconson, Michigan, Pennsylvania and Maine-2, and scraped home with a 273-265 Electoral College margin.
Swings are never uniform, so I tried simulating a normally distributed 2.9 point polling error with a randomised swing with the same standard deviation by state as the one that actually happened. (In calculating the SD for this purpose I ignored one massive outlier, Utah). When I ran Monte Carlo simulations of this (10,000 runs) Clinton still won the election 63.3% of the time, and only lost the EC as badly as she seems to have done (232-306) or worse 2.6% of the time. With a more "fat-tailed" distribution I would have got slightly different results but still the picture that this was not a likely result given the national popular vote and without something unusual happening would remain.
The discrepancy isn't explained by swing states being swingier (they weren't). The best description for the cause of Clinton's loss is that she was undone by the regional unevenness of the swing that affected the three key states described above. These were seats Clinton was sometimes viewed as taking for granted while Trump's tilting at them appeared hopeless. However in strategic terms Trump had no reason on earth to care what margin he might lose by, while for Clinton it was important to not just win but win big so that her success would carry through to down-ticket races. With polling showing Clinton winning Wisconsin especially by several points it may be that the Clinton team thought key swing states were in the bag.
Apropos of nothing much but hopefully interesting, this is what FiveThirtyEight projected about how the 2012 results would relate to shifts in the popular vote margin in particular states:
This is either a sign that his campaigning in the key states was better than Clinton's, or that the key states were always going to deliver for him. I lean more towards the former. There is evidence that especially in Wisconsin, which had the biggest polling failure of these states, late deciders went very heavily for Trump - though this does not alone explain the polling failure.
There's a question here about the extent to which bad modelling played a role in convincing left-wing voters that the outcome was a done deal, fuelling complacency and hubris among Clinton supporters. Nate Silver was right when he said Grim's article was "irresponsible". I am not convinced that there was actually a way for objective polling-based modellers to get this right given the problems with the polling in swing states. But giving Trump a 1% or even 10% chance instead of at least a 20% chance was completely inexcusable stuff from those who did it.
"Senator, you are no Donald Trump"
Lastly I would like to comment on some remarks by Tasmanian Liberal Senator Eric Abetz, in the knowledge that by his own standards he and anyone who takes them seriously will ignore me anyway. Speaking on radio this morning, Abetz claimed that the result showed that the public respond to conservative values and that right-wing politicians should ignore commentators and continue banging on against same-sex marriage, section 18C of the Racial Discrimination Act and so on.
He might have a point had the US elected a President Huckabee or Santorum. Instead, Abetz's brand of religious political culture-warring was almost nowhere to be seen in the final campaign, its adherents having been sent packing with no support at a very early stage of the Republican primaries. While Donald Trump did throw red meat to the base on issues like SCOTUS appointments and abortion, and while he did pick a religious conservative as a running mate, same-sex marriage simply wasn't an election issue.
For the most part Trump campaigned less like an Abetz-style Christian conservative and more like an avatar for most if not all of the seven so-called deadly sins. Genuine "Christian conservatives" were initially much taken aback by this, many saying they couldn't possibly support him, but they tended to come back to the partisan fold over time and argue that however crass Trump was, Clinton was worse.
The result did show that US voters were prepared - at least when the alternative was yet another dodgy political-class dynasty - to embrace a candidate who said what he would, no matter who he offended. That's a different thing to embracing candidates who continually bang on about the right to be offensive instead of focusing on issues voters actually care about. It might be argued that the Left has paid a price for too much culture-warring as an alternative to policy focus in this case, but the Right is far from immune from the same.
One thing that rarely gets mentioned in these post mortems is that it impossible to know what went wrong with the polls because they never release the raw data, who knows what corrections were made and why. I did hear of one pundit who predicted Trump's win by completely disregarding the poll data and using data from the primaries etc instead. I've no idea if this was just random chance or if there is something to his methodology.
ReplyDeleteI think this puts the theory of the shy conservative beyond doubt and will put a lot of attention on the polls for the French Presidential election, I see the odds for Le Pen shortened dramatically in the last few days.
Looking at a bunch of reports from the final US polls they are a little less generous in detail than UK polls I have looked at, but a lot more generous than Australia's. Still, raw data are generally not released. In the UK a lot of the pollsters would have disclosed the decisions they were making during the polling review, but I am unsure whether anything like that will even be attempted in the USA.
ReplyDeleteMy question with any unusual method claimed to have predicted Trump's win would be how many times has it been used successfully before (with predictions published before the election and not retro-fitted or fudged afterwards). If it is a method with a track record that is not based on polling then there may be something in it. If it is something that has just been done once it is probably a lucky hit.
It seems pretty clear that a lot of Clinton votes were "wasted"....either in safe Democratic states who would never vote Trump anyway, or in safer Republican states that she targeted early on (Texas, Georgia, Arizona).
ReplyDeleteOn the other hand, Trump got his absolute dream scenario. He got votes exactly where and when he needed them, and eeked out narrow wins in critical swing states.
It's basically the US version of the Australian 1998 election.
The way I look at it, this was an accident waiting to happen. I had made some input into fivethirtyeight late last week about the “failure” of polling here in Australia over the 1996-2016 period. There is a random scatter of error about the “correct” prediction. Poll aggregation does not improve this error. This was a bit dispiriting to me – especially as the first time I tried it (1996) I was spot on the money. That was pure luck – I had to wait 20 years for a repeat.
ReplyDeleteSilver ran a piece on Monday night, pointing out where everything could go wrong and, despite him being on the money in 2008 and 2012, he said that an average miss by 2 to 3 %age points is anything but uncommon.
I did a bit of crunching using the numbers from several websites and could faintly discern the following.
As has been reported, if 107,000 voters had gone the other way in Michigan, Wisconsin and Pennsylvania (combined turn-out of 13.6 million voters), Trump would not have won.
Turn-out was highest in the Swing States (response to attention?)
Turn-out was down everywhere except in several of Trump’s critical swing states (Trump recruitment of previous non-voters?)
Turn-out was especially down in Clinton’s strong states (“none of the above” effect?).
As a colleague in Pennsylvania says – the Electoral College has a magnifying effect on minor shifts in voting. But, although “everybody hates the EC, don’t expect changes anytime soon.”
None of the Presidential Swing State results were outside the 80% confidence intervals from the 538 model, so the 538 model did not fail. And the fact that the mean Clinton Trump difference in the polls for the Midwest blue states which fell was about 5 percentage points in error is not really a failure of the polls either, as in the US where you do not have compulsory voting, it is much harder to estimate the actual result from the polls. The Michigan polls were predicting 5% for Johnson, and he only got 3%, so if that 2% went to Trump, you have straightaway explained half the 'error' in the polls.
ReplyDeleteI agree that the 538 model didn't really fail as such and was just unlucky in not predicting the outcome. It was the exception as far as models went though; many others were overconfident. As concerns the others, if one poll is wrong by 5 points on the margin in a state that can be just margin of error stuff given the problems you mention. But if all of them are wrong by about that much in the same direction, then there is a problem.
ReplyDeletePart of the problem parties have is the quality of candidates. It's easy to separate parties vs candidates but candidate selection and management is part of basic competence in running parties.
ReplyDeletePart of the reason why minor parties stay minor or don't survive is because they live or die on the candidates they select. Major parties can claim "no individual is bigger than the party" and not only have more/better candidates to choose from, but are often smarter about who they put up.
One Nation will need to put up attractive candidates to win in Lyons, Braddon or wherever else they might contest. Equally importantly, they will have to avoid selecting the sorts of ding-dongs they chose in Queensland in 1998 (turning 11 seats into a one-term aberration), or Culleton more recently. Major parties make selection errors too (eg Labor's Peter Knott in 1993 or Liberal Jaymes Diaz in 2013) but their reputations and institutional base carry them past these aberrations. One Nation and other minors lack that buffer, ensuring that any candidates of quality and potential have more to fear from loose units in their own ranks than from their clearly identifiable opponents.
Psephology doesn't really assess candidate quality (except perhaps stunts like putting popular candidates down the ticket for multi-candidate elections), and takes whomever parties put up as given. There is some scope to model the impact of particularly popular candidates. Only in retrospect can you assess the impact of an undisclosed bankruptcy/conviction or other voter-repellent behaviour by a candidate, which reflects upon (if not sinks) their party. This was certainly a problem with assessments of Trump, where his voter-repellent behaviour did not appear to have repelled actual voters at the election.