Poll Roundup and Seat Betting Watch: New Dawn Fades Edition (August 27)

2PP Aggregate (Tuesday 27 August): 52.3 TO COALITION (+0.3 since last week)
Individual Seat Betting: Labor favourites in 59.5 seats (-1.5, Eden-Monaro, Brand becomes tossup)
Seat Total Market: Labor 61 seats (-2) (This figure is probably slightly skewed by longshot bias.)

This is week nine in a regular weekly series in the leadup to the federal election.  Week eight was here and through it you can click back to the previous weeks.  Or just click the "betting" label at the bottom.  As stated before, the aim of this exercise is not to claim that seat betting markets have predictive value, but to test whether they do, and to see which of the markets and the aggregated polls see the ultimate outcome of the election first.

This Week's Polls

It was a relatively quiet weekend for polling compared to the previous week's six-poll deluge.  Nielsen and Newspoll both returned 53-47 to the Coalition.  Roy Morgan returned 52.5-47.5 by last-election preferences.  Essential returned a second consecutive 50-50, but this one was off the back of a two-point fall in the ALP primary and a three-point rise in the Greens primary. My aggregate has gone to 52.3.  At the moment, it has one of the more benign readings for Labor doing the rounds - aggregates that benchmark solely off the last election are about a point worse, and those that handle sample size issues in a more sophisticated way, or weight heavily for accuracy of particular polls, are also showing slightly worse readings for Labor.  An aggregate of all the aggregators listed in Poll Bludger's excellent review of the state of things here comes out to 52.85.  My Newspoll rolling average, which I use heavily in historical modelling, is at 53.



A catastrophic decline in Labor's polling this week was expected in some quarters but hasn't happened - instead the picture is one of the dramatic recovery steadily fading and chances gradually slipping away.  All kinds of tactical explanations can be attached to that but I don't think we need any.  This is what polling bounces usually do.

It's really hard to know what to do with Essential in the current circumstances.  The strange behaviour of the poll has been framed by some as a phone-polls-vs-online-polls issue, but Morgan Multi-Mode, which uses an online panel as one of its sources, has displayed a similar pattern to the phone pollsters (just with the extra point or two for Labor thrown in).  Essential hasn't displayed a consistent house effect relative to the other pollsters, nor the same patterns in a muted or delayed form; rather, it seems to be polling for a different election altogether.  At Mark the Ballot, Essential has been sent to the naughty corner for its strange behaviour.

Here, I've decided I should leave Essential in (since for all we know, there's always the chance it is right and the others are somehow wrong) but that during the election period it should carry a sightly lesser weighting than normal because of the increased sample sizes of Newspoll and Nielsen.  Concerning the overall online-vs live dial-vs robopoll issue, it was notable that online surveys did rather well in the US election.  However, when the party biases of the groups of pollsters using the different methods are averaged out (as opposed to just their average accuracy errors), they didn't come to much difference at all - and US pollsters had the added challenge of identifying who will bother voting.  It's most likely in Australia that house effects in the campaign will have more to do with individual pollsters than with their primary surveying method, and that attempts to generalise across kinds of polling when we only have a few examples of each will be meaningless.

The weekend's polls contained quite a range of leader ratings.  Newspoll finally killed off my prediction (initially given an at most 20% chance of failing) that Tony Abbott wouldn't get his netsat into negative single figures by the election; he's reached the heights of -7, and now has the outright second-highest trough-to-peak netsat gain (29 points) for an Opposition Leader since the late 1960s.  He may even yet reach a positive rating by the election.  Abbott's current satisfaction ratings (42-49) are exactly where they were in July 2011; they were last better at the start of May that year.  Essential, not surprisingly given its generous 2PP, has Rudd with a -4 netsat while Abbott languishes where he was in May this year on -15.  Nielsen splits the difference with Rudd (down nine points in two weeks to -8) and Abbott (down one point to the same amount) tied.  As far as people who answer the telephone are concerned, Tony Abbott's long-term unpopularity is no longer an issue.  He's just another politician.  The online view might still be a bit critical.

Most of the seat polls from late last week were covered in the updates to last week's roundup.  There is one that was not included so far, a 50-50 in Hindmarsh (SA) via automated Galaxy. (Minutes after I posted this article, a 55-45 to Labor in Wakefield (SA) via the same company and presumably the same method also appeared on the radar.) We also saw a Tasmania-wide ReachTEL showing a swing of about 12 points statewide (link goes to my lengthy analysis) and projecting that Labor would currently lose Bass, Braddon and 12.3%-margin Lyons, with 10.8%-margin Franklin on the wire.  These swings are much greater than the six or so percent swing implied by Morgan's state samples and there is some need for an established phone pollster to resolve the difference between the two; if only to deliver me from a fifty-fifty chance of getting Lyons wrong.  The complex seat-poll vs national-poll debate continues to rage; I think I'll wait until it is nourished with more data later this week before having another go at it. 

Lastly, we've had the perennial problem in the last week of Newspoll aggregates of surveys conducted over several weeks being reported in the media as if they were fresh polling. They are not, and the rosy picture painted by such reports for Labor should be ignored.

Ghost Of The Week: 1996 

What we've seen this week so far in the national polls is nothing startling (except for those who didn't cease being startled by Essential doing odd things long ago.)  The slight worsening of Labor's position has neither matched the script of a recovery to a very close fight (a la 1993) nor the script of a continued slide into catastrophe (like 1975).  With last week's visiting ghosts looking less likely this week, but with seat-polling menaces lurking everywhere, a reprise of 1996 is another mooted scenario.  The Coalition could finish up in the mid-53% range, but with Labor losing significantly more than the fourteen seats implied by the pendulum, and finishing up on about 50 seats.

The curious thing about 1996 is that although Labor's defeat seemed heavy in seat terms for the national 2PP, it was more or less exactly "by the pendulum".  Labor went into the election holding 29 seats on less than the eventual swing, and lost 21 of those to the Coalition and 1 to an independent.  Seven were saved and one seat Labor had lost at a by-election was recovered from the Coalition.  On the other hand, seven seats above the swing line were lost to the Coalition, and one to an independent.  So Labor's net seat losses at that election were almost exactly what would have been expected based on the swing, as a result of a concentration of Labor seats on margins below 5%.  If we get a result like, for instance, 97-50 off a 53.6:46.4 2PP this time around, it will be because Labor has suffered much worse in seat terms than such a 2PP result predicts based on the current distribution of seat swings.  Generally in the past, predictions that one party will do much better or worse than the pendulum expectation for a given swing don't have a good predictive record. 

Some Other Ghosts Cleaned Up

Something I should have done long ago is clean up my Newspoll 2PP archive to correct for Newspoll having allocated 2PP results using respondent-allocated preferences in 2004 (Peter Brent covered this here).  When these are converted (somewhat roughly) to 2PPs by the modern method, it makes quite a difference.  The value of the switch to Latham now peaks (in rolling average 2PP terms) at about 4.4 points, with Labor's best run having been a run of 53-54-53 in March-April 2004.  For the last five months of the 2004 election, Labor under Latham were never really ahead by much.  They started the campaign very marginally ahead, were back to level by the two-week-out point, and then fell behind in the last two polls.  Latham was never as far ahead as those watching the Newspoll 2PP thought, and this explains some of the discrepancy between Latham's perceived leads in Newspoll and him trailing for most of that year in betting odds and "who do you think will win?" type questions.  While the discrepancy between polling at this stage and the final result in 2004 was still a large one, it was more like 2.7 points than four points.

I've also unearthed some reporting about pollsters other than Morgan for 1980, that reveals that Morgan's final poll that showed Labor well ahead was a bit of an outlier; other polls in the field at the time had it more or less level (which is where it ended up).  An average at the time would probably have had Fraser's government behind, but not by any great margin.  Say, 49:51, or maybe a little bit worse.

Modelling Labor's Chances Using Past Data

Last week I discussed how a few simple linear regressions could be used to estimate Labor as having a 29% chance of breaking 50-50 based on the record of past governments, or a 20% chance based on the past record of the parties (I prefer the latter method).  However that was if one ignored the Rudd leadership bounce issue, which I argued one shouldn't do.  I came up with a past-data based estimate of 12% for Labor's chances. Because of the 2004 preferences issue mentioned above, those models were a little more optimistic for Labor than they should have been, but not much.  About 10% was probably a better estimate then.

This week, ignoring the Rudd-return bounce issue, modelling based on the fate of the government at the time gives Labor an expected 48.8% 2PP with a 22% chance of winning.  Modelling based on fate by party (which is more predictive and in my view better) gives Labor an expected 48.3% 2PP with a 13% chance of winning using just Newspoll data, or 48.5% and 17% if I use my current aggregate and patterns from past Newspolls. 

There is a third way of modelling, which is modelling based on the fate of the underdog.  However, from a data set of nine elections, which way you assign "underdog" status for 2004 makes an immense difference to the outcome of the model. If you deem Howard to have been the underdog then the model is more predictive than modelling by government or party, but if you deem Latham to have been the underdog then the model is almost totally destroyed.  The actual situation is that the polling at this stage showed the Coalition ahead on primaries with the real 2PP more or less tied, but people who placed too much faith in Newspoll's 2PP would have thought Latham was winning.  The public generally did not believe Latham was going to win and the punters did not believe Latham was going to win.  The underdog effect is supposed to result from people getting behind the side they think is losing, not the side that is claimed to be losing on some (at the time new-fangled) indicator of polling results that people evidently placed little trust in.  A similar issue applies to 1998.  No version of the underdog model projects a Labor win from the current position (the best it does is about a 30% chance) but I think the whole question of who was the underdog at this stage in some past elections is too poorly defined to make it worth persisting with.

There is next the question of what to do with the Rudd bounce deflation issue.  Is Labor's polling still being dragged down by the washing-out from the system of the Rudd-return honeymoon effect? This week's polls are consistent with three possibilities: that the Rudd bounce has stopped deflating and Labor just had a randomly bad week of polling, that the Rudd bounce is still deflating but the pace has slowed, and that the Rudd bounce is still deflating at a steep linear rate but Labor had a randomly good week of polling given that fact.    The worst case projects Labor downwards to around 46% (give or take the usual point or two) with about a 1% chance of winning, but it shouldn't carry as much weight as it did last week, as there is some evidence the bounce deflation might have slowed.

All up my back-of-envelope scribbling here based on the past record of the two parties and considering the Rudd bounce issue gives Labor an expected average 47.4% 2PP (about what it is now), with a victory chance of only 9%.  Even if someone wants to argue whether there should be corrections made for the Rudd bounce issue or not, or prefers to model by the fates of governments rather than parties, it all doesn't make a lot of difference.  The central point is that past data imply Labor's victory chance is still real but fading week by week, and imply that it is slim. My rough 61-seat projection looks optimistic at the moment but I'm going to construct a multi-source seat data model after next week's national polls to see if it needs to be reduced.

Incidentally the AFR gives Labor a polling-based 19% chance of victory.  I am not sure what method was used but I would expect it is one that does not include any correction for the probability that the washing out of a leadership-change bounce is continuing to drag Labor down.

My assessment here is about as bad for Labor as the headline betting, so I obviously don't agree that $9 for Labor in a two-horse race at 52.5-ish two weeks out is as silly as some people reckon.  The fact is, nobody has yet retained office from this kind of position this close to the day - not at national level and probably not at state level either.  The governments that came from behind during the campaign to win in 1980, 1993 and 1998 (the last even without winning the 2PP, which Labor probably needs to do) were not this far behind at this point, while Howard had already equalised in 2004.  A win from here would be even more surprising - a good deal more surprising - than 1993.  If the betting headline rates are really displaying a run on themselves fuelled by belief that markets are predictive (and I think this may very well be true), then that might be just cancelling out the longshot bias that in the past has seen only one of nine betting underdogs win, although the final odds implied that on average two or three of them should have done so.

Any model based on past polling data should be treated with some degree of caution while Labor remains within a few points of 50-50.  The number of elections for which we have rich polling data from the campaign period is small. The challenges facing phone pollsters now are very different to those that existed in 1987, and there is a belief that accurate polling by landline, or even by telephone at all, will become harder and harder and eventually become obsolete.  But this is not an issue that should be expected to suddenly bite hard and do two or three points damage out of nowhere at a single election, and it wasn't even really nibbling in 2010.  

Seat Betting Picture

This week the Centrebet/Sportingbet exact seats market has seat totals from 87 to 90 seats as joint favourites (this would probably leave Labor with 58 to 61).  The correct election result market, which may be slightly skewed by longshot bias (but not very since it only gives Labor a 10% chance of winning outright), implies 61 Labor seats. 

Here is the individual seat graphic:

 The colour-coding again:

Medium blue: A seat in which the Coalition is favoured in all betting markets.
Pale blue: (none this week) Coalition favoured in some markets, level in others.
Grey:  All markets tied or both parties ahead in some markets and behind in others.
Orange: Labor favoured in some markets, level in others.
White: Labor favoured in all markets.

There are four colour changes this week.  The famous "litmus seat" Eden-Monaro has gone alkaline for the first time since just after Rudd became PM again.  There is a split market in Brand, with Labor favoured on one exchange and the Coalition on two.  And two Labor seats, Kingsford-Smith and Lyons, are now tied on one market.  (In Lyons there currently exists a small arbitrage between Centrebet and Luxbet.) I have simply tacked Lyons on the end of the graphic above without including all the Labor-held seats in between.

Here's the trend tracker for seats that have changed:

The close seat analysis is very unpleasant for Labor this week.  Despite gaining a seat from Labor this week, the Coalition is now favoured in only four (-3) seats in which Labor is inside $3 on some market.  They are the Labor seats of Eden-Monaro and Moreton and the Coalition seats of Brisbane and Bonner.  The mostly familiar list of expected Coalition gains - Corangamite, Deakin, LaTrobe, Dobell, Greenway, Banks, Reid, Lindsay, Robertson, Braddon and Bass - are all outside the close-seat window.  (Indi, where Sophie Mirabella is under a plausible level of threat from independent Cathy McGowan, is sniffing around the edges of the close-seat list having come down to around $3.50 this week.)

For Labor, the vulnerable species list just keeps growing.  Despite two close seats leaving Labor's list this week, the ALP still has seventeen (+3) seats that the market is edgy about.  New appearances on the list this week include Adelaide and (...drumroll...) Griffith!  Therefore, the seat-betting markets are likely to be expecting a somewhat worse outcome for Labor than the 59.5 seats in which Labor leads.  The markets don't seem to think much of the idea that most of these will go together (otherwise 101+ Coalition seats should be shorter than $6) so it's fair to assume there's at least a moderate degree of independence in the concern levels for various Labor seats around the country.

A list of likely and possible seat gains and losses apparently culled from party inside polling gossip appeared in The Australian today, with a close overlay with the seat betting favourites lists.  Petrie and Capricornia appeared as losses for Labor and Brisbane for the Coalition, while Deakin and Eden-Monaro were not on the main list of strongly likely Labor losses. Lyons was flagged as a possible loss.

Updates to this article are very likely to follow through the week; next week I will have a final volume in this series, and a comprehensive projection which may well be comprehensively wrong.  On election night I will almost certainly be doing live coverage of the election somewhere but my modelling currently shows a 75% probability it won't be here.  If it isn't here, there'll be a link to it posted prominently here on the night.  Post-coverage will be posted here.

ReachTEL (28 August):  A new ReachTEL was released on Seven News at some hour so early I almost stayed up for it.  It's 53:47.  ReachTEL publish their primaries to one decimal place, which in my view is useful and a good idea.  The "significant figures"/"precision of the instrument" type objections to doing so are based on misconceptions in my view - beware, I may discuss this at length one of these days.  However, when converted to 2PP the primaries come out to 53.0:47.0 to one decimal place anyway.  The ReachTEL shows Abbott as preferred Prime Minister 53.6:46.4, described by Seven News as a "turnaround from Newspoll".  That description is incorrect because, as noted here several times before, ReachTEL and Newspoll preferred PM scores run on a completely different scale and are not comparable.  (ReachTEL seems to have very little incumbent advantage in its PPM readings, which rather closely follow the 2PP most of the time.)

Anyway, given ReachTEL's history of very slightly Coalition-leaning federal readings compared to other pollsters, 53:47 is actually a fairly mild outcome at this stage of the game, and my aggregate has moved to 52.2 to Coalition.

The ReachTEL finds, as with the seat versions in recent weeks, that Coalition voters, having bayed for Labor to replace Julia Gillard with Kevin Rudd while Gillard was Prime Minister, now insist (54.7-25.4) that it was a bad idea.  There is a very strong gender skew on this question, with male voters overall approving the change 49-35.8 and female voters disapproving 35.1-45.1.  This is especially notable given that female voters typically tend to support Labor and the Greens more than male voters and that Labor voters are more likely to approve of the switch.

The Liberals' paid parental leave scheme gets the thumbs down from voters (30.7-48.4) with a significant minority of Liberal supporters not impressed with it.  On this question too there is a gender divide with male voters more supportive than females, probably by an amount exceeding the natural partisan skew of the gender divide.  Nielsen this week found female voters preferred Labor on this issue 49:37 (cf. 49:51 2PP) while male voters preferred Labor 45:42 (cf 43.5:56.5 2PP), with the differences in that case probably similar to natural partisan skew once the undecided rate is accounted for.  All up voters do not seem impressed with the Coalition's policy on this matter.

Oh by the way, Lyons is now tied on both the main betting markets, though Labor is still favourite on Luxbet.

Update (Aug 29) Bookmaker Pays Out: One bookmaker has paid out on the Coalition winning the election, meaning that those who placed bets on the Coalition to win up to this point collect their money whether this is the actual outcome or not.

Bookmakers do this not because they are utterly certain of the outcome, but because they think the outcome is sufficiently likely that the publicity stunt and customer goodwill benefits of doing so exceed the risk of being wrong.  I was especially amused when a bookmaker paid out on a Tiger Woods victory at the halfway point of a golf tournament that he didn't actually win. But even events like this probably help bookmakers to rake in more customers who believe that they can beat the house and make money.

When it comes to elections, I find this sort of thing a bit distasteful, because of its potential to influence the outcome.  That said, which direction it might have influence in is difficult to predict.

For this reason I am not naming the bookmaker (ir)responsible; they've had quite enough free publicity from my study of the predictiveness (or not) of their markets already!  (Especially not when they cannot count.  They say Labor is favourite in 56 seats on their own market; it is actually 57!)  The bookmaker also claims its odds predicted the result correctly in 147 seats, but my records of its odds as at 10:30 pm the night before the election suggest only 141 were "right" even then, so if more shifted it must have been very very late indeed.

Deterioration in Labor's position according to seat markets has continued today.  Brand is now in the Coalition's kitty on all markets, while Lyons has just crossed (being a tossup on two markets and a very marginal Coalition favourite on a third.) There is also a split market in Lingiari.

Update 30 Aug: Western Sydney Newspoll An aggregated Newspoll of the seats of Lindsay, Greenway, Banks, Parramatta and Reid (total sample size 800) shows an average swing of 9.1 points for a 43-57 average.  Rudd has a -18 netsat (for that 2PP lead, I'm surprised it's not worse), Abbott is at +1, and Abbott leads the all-unimportant Preferred Prime Minister 46-40. However, all these seats were also surveyed by Galaxy last week when they averaged 48-52 off a combined sample of 2875.  We therefore have a strong case where an automated poll with a large sample size has returned a much better result than a comparable phone poll, so that is surely bedtime for the idea that there is something drastically skewed about robopolling as such as a seat polling method.

One of the great mysteries of this campaign will be why Rudd has considered persistently downplaying Australia's largest city to have been a good idea.

Galaxy has Labor ahead 54-46 (which would be a modest 3.5% swing) in Adelaide.  Full details here.

Update 30 Aug: Morgan: Roy Morgan has issued a surprise "multi-mode" of a sort - this one consisting of internet and phone polling but excluding face-to-face.  The result is 54:46 to the Coalition by last-election preferences (53:47 respondent-allocated). It is difficult to say whether this poll should have the same house effect as the three-form multi-mode polls Morgan has been running, so I have treated it as having a house effect of half a point.

What may have been the phone component of the poll (it's not clear) has also been used to gather approval ratings.  It finds Kevin Rudd with a net approval of -17 (35-52), Tony Abbott with -10 (41-51) and Abbott leading as Preferred Prime Minister, 44-43.  Although the sample size is small, this is the first poll containing an undecided option to have Abbott leading on that indicator, which normally favours the incumbent in polls that contain an undecided option.  At this rate it might not be the last. 

Rudd's Melbourne Cup Analogy Is Silly: Yesterday when confronted with the news that one bookie had paid out on the outcome, Prime Minister Rudd responded by pointing out that only 35 favourites have won in the 150-odd year history of the Melbourne Cup.  It was a strange comment.  It's true that favourites have a slightly worse than one in four strike record in said race, but favourites in the Melbourne Cup usually aren't actually expected to win.  Rather, in most cases, the "favourite" is the horse considered least unlikely to triumph. 

I don't have a full list of Melbourne Cup favourite betting odds but here's a list from 1999-2011.  In this time, odds for favourites ranged from 11-2 (implied 15.4% chance, not adjusted for rake) to 2-1 (implied 33% chance, ditto).  The average implied chance for a favourite to win in each year in this time was only 22.5% without adjustment for the bookies' profiteering, and less than that with such adjustments.  None of them were given a greater than 50% chance, so none of them were expected to win.  Only two of them (actually the same one twice) won in that period, which is about the strike rate that should be expected.

There are rare cases when a horse is actually expected to win, and even then very cautiously.  The shortest odds a horse has started at in the Melbourne Cup are apparently 8-11 (implied 58% chance) for Phar Lap in 1930, which duly won.  Bookies currently give the Coalition an implied chance of around 92%, implying that it is about five times less likely to fail than the shortest-priced favourite in Melbourne Cup history.

The relatively long odds for the average favourite, and hence the poor strike rate of Melbourne Cup favourites, result not only from the Cup being a race for 24 horses (whereas the federal election is essentially a race in two) but also from the use of handicapping to create a contest in which no-one really has that much idea what will happen.  Horse races that don't employ such handicaps sometimes have very short-priced favourites (which tend to duly almost  always win.)

Election favourites don't always win, but have a very good strike rate.

There are examples of electoral contests being lost by $1.03 favourites.  Jonathan Jackson (Labor, Denison) was at such odds in Denison last election and lost.  But that contest was little nourished by polling, and what polling was available was unreliable, so it was hard for the market to estimate independent Andrew Wilkie's chances.

Newspoll NSW marginals:  Newspoll released a batched result of 53-47 to Coalition in the combined marginals of Dobell, Robertson, Eden-Monaro, Page and Kingsford-Smith, these being the NSW marginals not already covered by their Western Sydney poll.  The batching included the 54-46 in Dobell and Robertson a few weeks ago, leaving about 52-48 for the new inclusions.  Working out whether this tells us anything we don't already know is a difficult exercise, but a reasonable extrapolation is that out of the ten NSW marginals Labor holds, most will go.  Much the same follows even using the milder (and larger) Galaxy Western Sydney samples if a small deterioration in Labor's position since then is assumed.  Abbott has the better netsat (-3 plays -20) and a small PPM lead (45-42)

Newspoll Vic trinity: Newspoll also released a batched result of 53-47 to Coalition in the combined Victorian marginals of Corangamite, Deakin and LaTrobe, which have consistently been considered as Labor losses.  This is right on the average for existing seat polling for these three.  Abbott is the less unpopular leader (netsat -9 compared with a nasty -23 for Rudd) and has a small PPM lead (44-43)

Aug 31: JWS
Griffith ALP 57.8-42.2 (compare Newspoll 48-52, Lonergan 48-52, ReachTEL 54-46)
Lilley 53.8-46.2 to Coalition (this seat has a long Gillard-era seatpoll history, but hasn't had that much attention since, see here)
McMahon 53.1-46.9 to Coalition (identical to ReachTEL 53-47)
McEwen 54.7-45.3 to Coalition
Bendigo 50.6-49.4 to Labor

With the exception of Griffith these continue a pattern of JWS producing rather horrible seat poll results for the Government.   McEwen and Bendigo have not been the subject of public polls before, but McEwen has been considered at serious risk despite being on a 9.2% margin. Bendigo is on a margin of 9.4% and has a retiring sitting member.

Aug 31:  Galaxy has recorded 55-45 to the LNP in the Queensland seat of Herbert and 57-43 in Dawson.  At one stage these seats were considered at risk to the Queensland surge; not any more.

Sep 1: Galaxy has also had Hasluck 55-45 to the Liberals, Brand 52-48 to Labor and Perth with Labor ahead 58-42.  The latter attracted the attention of William Bowe as the first seat poll since the return of Rudd to "show" a swing to Labor!

Galaxy also has Griffith at 54-46 to ALP (same as ReachTEL some weeks ago) and the somewhat shaky Queensland Labor seat of Blair at 50-50.  (There is also of course a national Galaxy at 53-47 to Coalition.)