Poll Roundup and Seat Betting Watch: Ghost of '75 Edition (20 August - Updates Added)

TASMANIAN POLLING COMING: Check here on Saturday and Sunday mornings from 9 am and there will be extensive commentary on state and federal results.  ReachTEL are in the field tonight and I was even polled! Questions are federal voting intention, preferred PM, issue importance, state voting intention, and rate the performance of the state government.

(Apologies to Tas Times readers looking for that stuff; I send the wrong link!  Correct link goes here)

-------------------------------------------------------------------------------------------------------------

2PP Aggregate (Tuesday 20 August): 52.0 TO COALITION (+0.5 since last week)
Individual Seat Betting: Labor favourites in 61 seats (-2, Forde and Moreton)
Seat Total Market: Labor 63 seats (-2) (This figure is probably slightly skewed by longshot bias.)

In this issue:
* Newspoll to a place no-one has won from.
* Is Essential just a Poll-Shaped Object?
* Robopolls: confessional for homophobes #
* Why Rudd Is Not The Electorate's Dumped Ex-Girlfriend
* Non-zero estimate of Labor's chances!
* Ghosts of 93 ... and 75.
* ALP not favourites in any Coalition seat.
* Betting Market Debate: Do Punters Just Dawdle Like Sheep?



(This article is very long.  I've tried to break it up into sections so feel very free to just browse the bits that interest you.  Updates will almost certainly follow.)

This is week eight in a regular weekly series in the leadup to the federal election.  Week seven was here and through it you can click back to the previous weeks.  Or just click the "betting" label at the bottom.  As stated before, the aim of this exercise is not to claim that seat betting markets have predictive value, but to test whether they do, and to see which of the markets and the aggregated polls see the ultimate outcome of the election first.

This week's polls

This weekend saw a third consecutive round of mostly worsening polls for Labor.  Galaxy kicked off with 52:48 to the Coalition, which was followed by 54:46 from Newspoll, 52:48 from Morgan multi-mode (by last election preferences) and, surprisingly, 50-50 from Essential.  On average, these four polls moved almost a point to the Coalition.  Among the lesser known pollsters there was also a 52-48 from Lonergan, their second national poll, which I have included (though the Green vote of 12 is on the high side).  And there was a 50-50 from AMR, an online panel poll that I suspect may have a slight pro-Labor house effect (and am applying a one-point correction on that basis.)  My aggregate, which has moved into a new phase in which very recent data is weighted more heavily, is now at 52.0% for the Coalition.  The Coalition would win an election "held now", probably by about 15-20 seats.

There are many dark mutterings about Essential following the 50:50 and how often it behaves in a completely different way to the other polls.  Some modellers are seriously doubting whether Essential is a poll or just a Poll Shaped Object* and considering ignoring it completely.  It's especially perverse that Essential couldn't get to 50-50 with the Rudd recovery at full steam ahead and now makes it there when everybody else who matters has it heading south at varying speeds. I persist with including it but am glad that it is not eligible for my last-three-weeks upweighting for the most recent polls.  However, the case for the benign readings for Essential and AMR is that there may well be equipment biases between online and phone polls that no amount of weighting can eliminate.  Online poll respondents are more likely to be heavy internet users and may be reacting negatively to the Coalition's broadband policies, for example.

The Newspoll showed Prime Minister Rudd with, by his standards, very bad personal ratings.  Even including his previous tenure, Rudd's satisfaction rating of 35 is his lowest ever, and his net satisfaction of -19 is exactly as bad as it was when he was removed in 2010.  Furthermore, his Preferred Prime Minister "lead" (not that it counts as such given the way PPM ratings skew to the incumbent) sits at a mere two points (43:41).  By way of contrast, when Rudd was removed in 2010 he still had a nine-point "lead".  More tellingly, when Rudd was Opposition Leader, he led John Howard by more than this in 22 of 26 Newspolls - even despite the circa 16-point handicap that being Opposition Leader creates in Newspoll 2PP polling.  Rudd only "trailed" Howard three times - those being his first three while still becoming established in the role.

Peter van Onselen in the Australian today quotes an unnamed "Labor MP" as saying the Rudd souring is "like when you get back together with an ex-girlfriend. It seems like a good idea at the time, but it doesn't take long before you remember, 'Ahh, that's why we broke up.'"  There's just one problem with that analogy.  The Australian voters never broke up with Rudd in the first place; the Australian Labor Party did.  Rudd was dumped in an election-winning position, with ratings that were bad but not as bad as those of several PMs whose relationships with the electorate survived and later prospered.  A better analogy might be attempting to rekindle a relationship that was going through a bad patch when it was disrupted by an interfering parent, with the estranged partners embittered by the experience. 

Meanwhile Tony Abbott's satisfaction level jumped to 41 and his netsat reached -10, both of these being his best single Newspoll results since July 2011.  Given that this Newspoll shows a higher 2PP than the average of other polls, it is likely that these figures for Rudd and Abbott aren't entirely representative.  But the central point is that Labor's strategy of running a presidential campaign around a leader who is much more popular than his opponent has failed as its underlying premise is no longer true.  The premise that "hypotheticals" polling only acts as a preview of the initial bounce for a leadership change, and doesn't tell you where the new leader or the party will rate over time, has been confirmed again.

No government has ever won re-election after polling a 2PP as low as 46 in any Newspoll in the last five weeks.  Paul Keating did, however, poll a 46.5 in 1993, and Malcolm Fraser had a Morgan Gallup reading during the pre-Newspoll 1980 campaign that comes out to about 46.2.  My Newspoll rolling average scores tell a similar story with Labor currently on 47.2; governments since the mid-80s have won from Newspoll rolling averages in the high 47s, but no lower.

Attribute findings by the major pollsters this week have had predictable results, eg see Newspoll here.  Essential's party attribute comparisons here seem rosy for the ALP, with the Coalition perceived as slightly more likely to keep promises, slightly clearer on what it stands for and much less divided, but Labor otherwise level-pegging or ahead.  However that too should be seen in the context of the unusually friendly 2PP result in this Essential.

I listed most of the late-week seat polls in last week's edition.   Since then we have had:

* ReachTEL of Deakin with a small Liberal lead, 53-47.
* ReachTEL of Corangamite with a Liberal lead of 56-44.
* ReachTEL of Melbourne with Labor ahead of the Greens 33.8:33.5 on primaries, which would translate to a comfortable win (probably about 55:45) with Liberal preferences (4.7% undecided included in primary tally.)
* ReachTEL of Indi with incumbent Sophie Mirabella (Lib) ahead of independent Cathy McGowan 43.5:23.3 on primaries.  That sounds like Mirabella could lose (most of the rest being Labor and Green), except that 7.1% of the primaries are undecided voters.  If these are redistributed proportionally, Mirabella jumps to 46.8% and would retain.  However, it's enough to indicate that Indi is worth watching and a boilover is possible.

We have also had a comment by Michelle Levine (Morgan) that polling in Eden-Monaro is running at 52-48 to Labor, though the sample size of this observation isn't stated.

Eight ReachTELs have been published on their website with detailed results that, because of the amount of stuff going on at once presently, will not get quite as much detailed attention from me as they otherwise would.  These include the four above plus MacMahon, Kingsford-Smith, Blaxland and Bennelong.  In summary:

* Abbott is Preferred Prime Minister by ReachTEL's measure (which gives little if any incumbent advantage) in all the seats bar Deakin and Melbourne.  In the six electorates with 2PP scores, he tracks an average of 2.5 points behind it. This is quite normal for ReachTEL nationally.

* Voters in six of the eight electorates are more likely to support than oppose same-sex marriage.  (I do suspect that robopolling gets somewhat lower pro-SSM figures than other methods, along the lines that voters may be less likely to confess to anti-SSM views when talking to another human being).  In every electorate the Liberal voters tend to oppose, Labor voters tend to support (though in Blaxland it's close, 44.3:36.6) and the Green voters overwhelmingly support.

The variation between electorates isn't just down to which party is winning.  It turns out that in an electorate where Labor voters tend to support SSM, Liberal voters are more likely to support SSM, and vice versa.  This is quite a strong relationship, explaining 71% of variation between the results in the different electorates for supporters of each party. That's over quite a range of variation, with Labor supporters ranging from 44.3% support (Blaxland) to 79% (Melbourne) and Liberal supporters ranging from 21.1% support (MacMahon) to 36.2% (Deakin).

* In all electorates most Labor supporters support the decision to replace Julia Gillard with Kevin Rudd (about 20% disagree), but a minority of Liberal supporters (about 20% again) agree with them.  So while most Liberal supporters wanted Gillard to be replaced when she was in office, while ALP supporters were split down the middle at the time, now Labor supporters are generally behind Rudd in retrospect while Liberal supporters are not happy about getting what they wished for.  Voters in Melbourne are most immune to this partisan nonsense.  The attitudes of Green voters vary from seat to seat.

* In general, supporters of each party overwhelmingly think their leader has the best asylum-seeker policies.  In the NSW seats, a substantial minority of Labor voters prefer Abbott's, but in the Victorian seats this isn't the case.

(* By analogy with "Piano Shaped Object", denoting a piano that may look the part but plays badly, eg because it is out of tune, damaged, mistreated or was no good in the first place.)

Are We There Yet?

Declarations that Labor has more or less certainly lost the election have been growing in frequency as polling and betting odds continue to blow out.  Not only is Labor's polling seen as hopeless, but Labor's campaign is being seen as a thematically garbled, disorganised and impotent mess, and Labor is well behind in issues polling on the issues supposed to most matter.

In terms of polling history, there's still not nearly enough evidence Labor is gone!  In fact, looking at Newspoll history, government polling histories three weeks out have been even less reliably predictive than at five weeks out, perhaps because of the small sample size from past elections.  The crude Newspoll-based model of this kind I ran two weeks ago now still gives Labor a 29% chance of getting a 2PP above 50%, and projects a Labor 2PP that recovers to 49%. An alternative version which considers results by party rather than by government/opposition status (and I think this may be a better model on the whole, as it explains more variation) gives Labor 20% and projects 48.6%.

However, Labor's polling in the last three weeks looks a lot like what happens when a large leadership transition bounce starts to wash out of the system - something not seen in the previous elections because the Rudd-to-Gillard bounce was rather small.  Parties don't normally shed over half a 2PP point in the same direction three weeks in a row by chance. If the washout stops right now, Labor remains competitive and polling should tighten, although Labor will still probably lose.  If the washout effect continues through to election day, even without accelerating, Labor should lose very heavily.

I think the average of these two scenarios and the various points in between is about 47.0% 2PP for Labor, and I'm keeping last week's very tentative forecast of 61 Labor seats for now.  At the moment, I think analysis of past polling data, with an adjustment for the Rudd bounce issue, gives Labor about a 12% chance of pulling this one out of the fire.  A lot can be argued either way on this - for instance, the threshhold for success is a 50% 2PP vote, but probably the break-even point for this Government is slightly higher than that.

Ghosts of 1993 are haunting the online corridors at the moment.  If you've got a spare half hour, this election eve Lateline from 1993 is a classic, and so much of it sounds familiar: planless government campaign, chaotic co-ordination between leader and party, even the familiar complaints about the abundance of polls and whether the media have paid too much attention to them!  It's also notable that Paul Keating had much worse personal ratings during the 1993 campaign than John Hewson, and even trailed as Preferred Prime Minister.

But I think the 1975 edition spook is making a few funny noises too.  In 1975, Labor was being whacked horrendously in the polls all year.  Then a remarkable situation resulted in a political crisis and a rapid Labor polling recovery.  Labor started the campaign in a competitive position, with a leader who rated acceptably, against an opponent seen as negative, destructive and unpopular.  But through the campaign Labor's vote went down and down and down, the leadership ratings reversed, and they finished with just 44.3% of the two-party vote.  It was nearly three weeks into the campaign before polling made it clear how badly they were likely to be thrashed.  Of course, it's not an exact analogy - Malcolm Fraser was only fleetingly unpopular, while Tony Abbott has rated poorly for a very long time, for example.

That is why just as I don't yet agree that this one is all over, I also don't agree that it will necessarily be close should the Coalition win.  A blowout to 54:46 or even 55:45 is still a very real prospect that cannot be ruled out.  But there's much better objective evidence now than a few weeks ago that a middling Coalition victory should be the midpoint of expectations.  And indeed it generally is.

Markets Slip Away Again

Labor's standing in betting markets was so low already that even the 54:46 Newspoll was little more than a thumbtack in the coffin.  Labor has now gone above $7 on the headline rate.  [Update a few hours later: blowing to $8.50 now!] The Correct Result market is implying 63 Labor seats (a slight overestimate because of longshot bias as discussed last week.)  The moveable handicap market (which I trust I've interpreted correctly this week) gives Labor 65 or 66. The lowest point of the exact result market gives the Coalition 87 (implying Labor 61).

Here is this week's seat graphic:

The colour-coding again:

Medium blue: A seat in which the Coalition is favoured in all betting markets.
Pale blue: (none this week) Coalition favoured in some markets, level in others.
Grey: (none this week) All markets tied or both parties ahead in some markets and behind in others.
Orange: Labor favoured in some markets, level in others.
White: Labor favoured in all markets.

Bold shows a seat that has changed in colour in the last week. There are four of these.  Moreton and Forde have switched sides, Eden-Monaro is tied on one market, and Brisbane is no longer tied on any market.  The rise and fall in the markets of one Labor candidate for Forde, P Beattie Esq, has been remarkable.  A couple of 40-60 robopolls have thwacked him all the way from $1.40-ish to $4-ish as markets realise this desperate move doesn't seem to be working.  (Simon Jackman today listed Eden-Monaro as already crossing but I believe this is a result of longshot bias issues as discussed here.)

For the first time since Brisbane flipped in mid-July, Labor are not favourites in any Coalition seat. Indeed, Cathy McGowan (Ind, Indi) is rapidly flying up the list of non-Coalition candidates most likely to knock off a Coalition MP.  She is currently at $4.25 at Sportingbet.

On to the trend tracker:

The seat betting favourites picture is now almost exactly where it was on 10 July.

The close seats picture suggests an even grimmer view.  Only seven (-7) seats in which the Coalition are favourite are considered close by the markets (Labor inside $3 on at least one market).  That is despite one new close seat crossing from Labor in the last week.  (Forde is not even now considered close.) Meanwhile 14 Labor-held seats where Labor is still favourite are considered close.  This is down four on last week, but two of those went to the Coalition pile and a third was Melbourne, in which Adam Bandt has gone outside $3 to retain.  A new entry on the close seats watch for Labor is Lilley, held by ex-Treasurer Wayne Swan, and now almost a tossup on some markets.  Worse, there are only two Coalition held seats (Brisbane and Dawson) that Labor is still considered to be in with a serious chance of snagging.  Labor's hopes rest on a national swing back that would be sufficient to recover or win many seats where it is now at prices like $5.  The disparities in the close-seat analysis suggest that the seat betting markets actually reckon 61 seats is a bit generous; 59-60 might be the most accurate read.

One question I haven't covered is what seat markets think about the size of swing to be expected in different electorates.  David Barry has a rather striking model of this question here.   The seats where swings to Labor seem to be implied by current odds (there's precious few of them) are mostly actually seats where an IND or KAP or PUP is disturbing the betting picture.  Beyond that down the bottom end we have ex-PM Gillard's electorate of Lalor and the four Tasmanian seats of Bass, Braddon, Lyons and Franklin.  Then Dobell and a couple of suspected hotspots on good margins, Newcastle and McEwen. 

Nerdy Long PS: Betting Markets: Predictive Or Just Sheep?

Warning: this section carries a Wonk Alert, Level Four.  It's the end of the article, so if that stuff scares you, you can safely skip it.

There has been a degree of Twitter debate about the predictiveness of betting markets in recent weeks, particularly involving the AFR and Peter Brent.  You can see the case for the defence at electionlab here and the case for the prosecution scattered through numerous Brent articles including this one (2010), this one (Gillard vintage) and also this one (handy review of past headline betting).

What to make of all this?  Well, for starters, I think the seat betting markets last time were actually expecting more like 76 seats than 74; the 74 figure was a Jackman model that I believe did not adjust for longshot bias.  But still, pretty close, and closer than any straight reading of the combined late polls or even the late judgement of most experts.  Also, the exact seat total markets were sniffing around figures like 73, 74, 75 for a long time, even while the headline figures held that a Labor win was very likely.

Both sides agree that betting markets get it wrong sometimes (as they did in 1993 and in the state elections in WA 2009, Victoria 2010 and Tasmania 2006).  Both sides agree that betting markets usually get it right.

Peter Brent has noted that betting markets do not follow polls closely but "dawdle" after them - requiring a great degree of evidence to shift a previously strong conviction that a given side will win.  However, when we look at all the times Oppositions have led for several months but failed (80, 93, 98, 01, 04) is this really such a bad approach to take?  A model that installs an Opposition as favourite just because it polls well for a few months is most likely a bad model, given that most governments get re-elected, although the average government will be about seven 2PP points behind as a rolling average at some stage of its term.

Given that polls are not forecasts, and given that no predictive method will be perfect, I don't think that the imperfect record of betting odds and examples of them failing prove much in this debate.  After all, the cases in which betting odds got it wrong are often cases in which the final polls also got it wrong, and in which many experts were wrong too.  And particular experts who got those late-plunge situations right (through whatever arcane means) will have made other predictions that were wrong.

If Australian psephologists are to knock the challenge of market-based prediction over once and for all, there are two things we can do.  The first is to establish a documented record of individual judgement, based on all our clearly stated and falsifiable predictions, that is superior to that of the markets on the same questions.  The second is to build predictive models (not just poll-trackers) that do not use betting data at all but that still outperform it.

The third approach, as seen in the Pottinger model, is to assume that markets do have a part of the story that we won't capture by any other ready objective means, and to use them as one input alongside polling data.  Now that the Rudd bounce has gone down, this model appears to be tracking much better than any purely polling-based model that did not incorporate either a leadership bounce or else some kind of long-term assessment would have done a few weeks ago.

Last week's Pottinger post noted that that model seemed overconfident of the end result in using current data to imply that Labor has no chance at all.  It canvassed various possible reasons for this, but I'd suggest a different one: that we have too little information on the performance of governments at very long odds to be confident of any read of betting market data that implies their chances are really virtually zero.  The headline rate for Labor may well contain significant longshot bias, but it shouldn't be that much.  Indeed, if markets are very efficient, and the markets alone really think Labor's chance is less than about seven percent, then the Coalition's headline price should be lower.

I also suggest a rule of thumb for analysing betting markets generally:

If an indirect assessment of the market's view of the chances of event X implies a radically different probability to an existing direct market on event X, then the indirect assessment is incorrect.

A lot of the action in the betting market debate generally arises not from people reporting what the markets think, but from models that think the markets say things that in my view they actually don't.

# Yes, I know not all opponents of same-sex marriage are homophobes.  My deepest apologies to those who are merely illiberal, discriminatory, paranoid and wrong.

====================================================================

Update 22 Aug: Rudd Losing Griffith? Whaaaaaaaaat?

Oh dear, here it comes again ...

 (NB 24 Aug: There was a Wirrah here.  It's been replaced by a Reverse Ferret.)

A Lonergan poll for the Guardian has provoked widespread derision by showing Kevin Rudd trailing 48:52 in his home seat of Griffith, a swing of 10.5% (margin of error 4 points).  Although most polling is no longer showing a large swing to Labor in Queensland (Bludgertrack, for instance, currently has the Queensland swing at zero) this is yet another dire "result" for Labor in seat robo-polling that is not reflected by the state and national mood.  While it is true that seats to target are being cherry-picked by the media and pollsters, there are now far more seats supposed to be showing monster swings than is credible.  Seat swings nationwide should be normally distributed with a standard deviation of about 3.2 points (see here).  That means that for a base swing of two to three points, the current national estimate range of various aggregators, there should be only about three seats in the country extreme enough to swing by more than, say, nine points.

This comes with a caveat that now and then you will get a regional cluster of seats with larger swings - Tasmania being a prime candidate.  Also, some elections are more volatile from seat to seat than others.  But the idea that Labor is trailing by a few points nationwide and yet facing swings of 8-10 points or more all over the place is just not credible.  Of nineteen recent robopolls at seat level by JWS, Lonergan and ReachTEL - excluding Tasmania - swings of 8 points or more occur in nine polls covering seven different seats.  (Yes, some will say this election is different and showing volatility on a level never seen before.  They said that every other election too.)

Lonergan appears to be the worst offender (albeit from a small sample size) and the apparent chink in its armour is that it uses the 2010 results to scale its figures.  This is a flawed method because respondents are not always honest about who they voted for last time and tend to over-report voting for the winner.  Scaling in this way may then produce a result that favours the side that did not win.  I have removed Lonergan from my national aggregate until it carries clear primary documentation of its national methods on its site.

Looking at the individual recent ReachTEL seat polls that are available they seem largely explicable by a combination of a slight conservative skew and the deliberate targeting of trouble-spots such as western Sydney and Tasmania.  A similar story may apply with JWS.

OMG! Established Pollster Seatpolls! (23 Aug)

The following seat polls have just appeared.  Details are sketchy at this stage.

Galaxy: (sample sizes about 550-600)

Corangamite (Vic) Coalition 56-44 (identical to ReachTEL, 3 points above JWS)
LaTrobe (Vic) Coalition 51-49
Chisholm (Vic) ALP 52-48 (3-point swing to Coalition cf. ReachTEL 27 June, but that was at a time of higher Labor 2PP)
Lindsay (NSW) Coalition 54-46 (6.7 points under JWS, 10 points under Lonergan)
Banks (NSW) Coalition 52-48 (0.8 points under JWS)
Barton (NSW) ALP 52-48
Parramatta (NSW) 50-50
Greenway (NSW) Coalition 51-49 (2 points over JWS)
Reid (NSW) Coalition 53-47
Werriwa (NSW) Coalition 52-48

All these seats are currently Labor-held, and they're a mix of seats being treated as probably lost and seats where the Coalition is competitive.

Newspoll:

Forde Coalition 54-46 (identical to ReachTEL albeit on day of announcement, 6 points under JWS and Lonergan)

Forde is the Coalition-held seat Peter Beattie is attempting to win.  Abbott has a slightly better netsat in it than Rudd (-10 vs -13) and the PPM is tied.  For a 54-46 result, no surprises there.

It's only a small sample size but the three results comparable to Rudd-return ReachTELs are more or less identical to them, which will be a pleasing outcome for ReachTEL.  The mainstream pollsters are on average about 2.5 points lighter on the Coalition than the five JWS seat polls (but with a fair variation range), and 8 points lighter than the two Lonergans. 

The other thing to note is that while some of the swings the mainstream pollsters have recorded in these cherry-picked danger seats are high, and the sample sizes in the Galaxy polls may not be big, there are no double-digit swings.  The worst is 9 points in Werriwa, a seat that has been no certainty for Labor for quite a while.

Many of the leads shown will be well within each seat poll's margin of error, and when so many seat polls are released at once there's about a 40% chance of at least one rogue result that is a long way out.

Update 23 Aug: Simon Benson: Familiar Suspect: The reporting here claiming that Barton was "the only seat among the seven that Labor would have a chance of retaining" is, as John Hewson would have it, "innumerate", and it's no surprise that Simon Benson, a familiar suspect, is involved.  On the figures in question, chances of retention in the seven NSW seats in an election "held now" would range from about 3% in Lindsay to 83% in Barton.  In Greenway, the poll taken alone implies a retention chance of 31%.  To say a party would have no chance of retaining a seat based on a poll is bad enough, but to imply it even of a seat where the polling is actually 50-50 takes the cake for atrocious poll reporting.  Yes, even by Torygraph standards, or lack thereof.

Galaxy Polls Automated: It has emerged that the Galaxy polls were also robopolls, albeit by an established pollster rather than an inexperienced one.  I have changed some comments above for that reason. There remains a dearth of seat polls conducted by conventional means.

Even Stranger: Newspoll Agrees With Lonergan (Aug 24): The Lonergan finding in Griffith (48-52), which I rubbished yesterday, has now been repeated by Newspoll!  Newspoll also found a 40-60 2PP in a selection of Queensland "marginals" (Brisbane, Forde, Flynn, Longman, Herbert, Dawson, Fisher and Bonner, though personally I would have left Fisher out), for about a 7.5-point average swing. 

It's looking like there is a serious discrepancy issue between national polling (even when broken down into states) and state polling.  It's not just about national pollsters vs specific robopollsters, or old-style polling vs robopolling.  It may even be, as in the US for the Republicans at last year's Presidential election, that something is causing all the national polls to be too generous to the ALP this time around.