Poll Roundup: This Is The Way The Honeymoon Ended

 2PP Aggregated Estimate: 52.9 To ALP (-1.4 since mid-August)

ALP would still win election "held now", probably with increased majority

Time for another federal voting intention poll roundup as there have been several noteable results in recent weeks.  In a previous edition I reported that while the end of the Albanese Government's polling honeymoon had been declared by many hasty false prophets, we weren't quite there yet ... but we could be soon.  My standard for the honeymoon phase still existing had been a 54-46 estimated aggregated polling lead for the government, but in the event of the government falling slightly below that level I would want to see at least a month of evidence that that was the case.  (It is somewhat like how a single quarter of negative growth does not count as a recession).  

Anyway I can now report that on my estimates the rear-vision window shows that it's been two months.  The Albanese Government's polling honeymoon ended not with a bang but with a gradual slip into the twilight zone of not-quite-enough-ahead in early September.  There were several individual poll results better than 54-46 since then but on a weekly rolling basis I have had Labor in the 53s ever since.  Furthermore following this week's Newspoll the Government dipped just below an aggregated (and Newspoll!) 53% for the first time.  

Poll Roundup: Are Australian Voters Grumps?

2PP Aggregate: 52.1 to Labor (-0.3) - updated after Ipsos
Labor would win election "held now"

Another four weeks on since my last update on national polling and not much has changed.  We still have only two regularly active pollsters and their results continue to show a very gradual shift away from the returned Turnbull Government since the July election.  I'm expecting some improvement on the former front shortly, and may update this article should that occur.  As for the smoothed 2PP tracking, it looks like this:

Since last time there's been little variation in the released polls, all from pollsters that don't do a lot of variation anyway: two 53-47s to Labor from Newspoll, two 53s and two 52s from Essential.  I consider these two polls between them to have had a bit of form in skewing to Labor during the Turnbull phase of the previous parliament and so the current aggregate comes out at 52.4 to ALP. Before house effect adjustments, I've aggregated the recent polls at 52.7 and 53.0 from Newspoll and 52.5, 53, 53.4 and 52.6 from Essential.  However, the 53.4 currently isn't in the mix, because of the way I use only alternate Essentials at any one time.  Those who also follow BludgerTrack, which should be everyone reading this, may have noticed my aggregate is about 0.4 of a point more Coalition-friendly at the moment.  A fair slab of this is because I assume that all the pollsters know what they are doing with their 2PP calculations, though a lot of the time I have my doubts.

Since Essential started using the 2016 preference distributions for its 2PP figures, I have its average published 2PP as 52.2 for Labor, but the average 2PP derived from the published figures without any knowledge of decimals or state breakdowns would be 52.72.  So the Coalition has done better on the published 2PPs than would be expected from its primary votes, by about half a point.  In contrast, Newspoll has had a published average of 51.72 for Labor but its derived average would be effectively identical to the published figures at 51.84.  The differences in Essential's case could be caused by rounding, but rounding differences should be randomly distributed.  If this continues for another, say, ten polls it will be strong evidence that something unusual is going on - not necessarily an error, as state breakdown factors could be at play.

We're still not seeing any panic-stations 2PP polling, and the rate of change is slower than the last time the Coalition lost support in a more or less steady fashion over a few months (in late 2014).

Field Guide To Opinion Pollsters: 45th Parliament Edition

This is an outdated guide version which has been superceded by the 46th Parliament Edition.

----------------------------------------------------------------------------------------------------------
Just before the 2013 election I posted a Field Guide to Opinion Pollsters, which has become one of the more enduringly accessed pieces on this site.  However, over time parts of its content have become dated or specific to that election, and with more and more pollsters emerging as others disappear, the thing has got too long.  I've decided therefore from now that I will post a new edition shortly into the life of each parliament, editing it through that parliament as the need arises.  Pollsters not expected to be active in the life of the current parliament will be removed, but the old edition text will remain on the previous page.

There are a lot of polls about in Australia these days.  But how do they all work, which ones have runs on the board and which ones can you trust the most? This article describes what is known about each pollster and its strengths and weaknesses and includes coverage of general polling issues.

The gold standard for success for an opinion pollster is seen to be that its polls at election time get the result as close to right as possible.  However, some pollsters are little-tested against actual elections, and getting a specific election right is a combination of skill and luck.  In elections where there is a swing on the last day or two of the campaign, a pollster that is actually not polling correctly may have its errors cancelled out by the swing, and hence record a lucky hit.  There is more to being a good pollster than just getting it right at election time - a good pollster should also provide useful data between elections and do so using well-designed questions that are easy to interpret.  And a pollster should also present their data in a way that makes sense and isn't misleading or confusing.

2013 Federal Election: Best And Worst Pollsters

Ideally I would have written this article a few weeks after the election, while debates about the quality of polling before the election were still fresh in everyone's minds.  It seems a world away now but earlier this year there were actually plenty of people arguing that the day of the landline was done, that online and mobile polling were the way to go and that Palmer and Katter supporters would preference Labor in droves thus making the election much, much closer than all those pesky polls said.

But now ...

This was, in general, a pretty good election for the pollsters (most of them anyway) and a bad election for those who wanted to outguess them.  

For basic information about the different polls and their methods, and a lot of general stuff about polling methods and debate surrounding them, you may find the recently updated Field Guide To Australian Opinion Polls useful. 

I'll consider pollster quality in three categories:

Too Much Information: The Mixed Performance of Seat Betting Markets At The Federal Election

Advance Summary

1. Seat betting markets, considered by some to be highly predictive, returned an indifferent final result at the 2013 federal election, overpredicting the number of Labor losses by at least seven and predicting fourteen seats incorrectly.

2. Better results were achieved not only by local/state projections based on polling data but also could have been achieved by a simple reading of the national polls.

3. Seat betting markets in the final week most likely misread the election because of an overload of contradictory data.  They placed too much emphasis on local-level polling and internal polling rumours and too little on national polling.

4. Prior to the final week of the campaign, however, seat betting markets performed well in projecting an uncertain situation that was difficult to model.

5. Seat betting markets were most accurate immediately following the return of Kevin Rudd.  However this probably reflects on the modelling skills of bookmakers rather than punters.

6. Modellers wanting to know what seat totals betting markets expect should look at direct seat total markets rather than attempting to derive that information via complex and uncertain processes from seat betting markets.

7. Final direct seat total markets were very accurate.

Essential: Not Bouncy Enough?

Admin note: Following the farcically misinterpreted Galaxy poll of "best" Prime Ministers of the last few decades (in which a huge lead for Howard based largely on him being the only Liberal in the sample while the Labor vote was split four ways was supposed to prove he was the best), I thought it would be fun to have a "Not-A-Poll" opt-in for "best" Tasmanian Premier!  It's on the right. Vote early stack often! Results are almost totally meaningless and for amusement/interest purposes only.  Ballot order random.

------------------------------------------------------------------------------------------------------------------------ 
 Advance Summary

1. The regular online pollster Essential Report has recently polled figures that lean to the Coalition compared to other pollsters (by about 1 point or slightly more) and that change very little from poll-to-poll.

2. Essential's two-party preferred results became much less volatile from early to mid 2010 onwards.  Prior to that they had been very bouncy and tended to have a large house effect in favour of Labor.

3. The current lean to the Coalition is a more recent development.

4. It would be expected that Essential would bounce less from poll to poll compared to other polls because of its use of a two-week rolling average, which gives it a larger effective sample size and reduces the impact of random factors on the result.

5. However, even with sample size taken into account, Essential polling in 2011 and 2012 was 30% less bouncy than a simulated "random" poll of similar size, even after assumptions are made that make the random poll less bouncy than such a poll normally would be.

6. This suggests that (i) Essential's scaling system is remarkably effective in reducing poll-to-poll bounce, or (ii) there is a problem with the underlying subsampling methods that is causing overly repetitive results, or (iii) both of these things are happening. At this stage I think (ii) is most likely.

7. It would be easier to trust a poll that produced remarkably constant results if it did not drift off the 2PP average of the other pollsters.

Disclaimer: These are provisional findings relating to a complex modelling question.  They may be amended based on any feedback received.  If so, significant edits will be noted at the bottom of the post. Oh, and this post comes with a Level Three Wonk Alert.  You have been warned!

Silly Lilleys: Is Wayne Swan Losing His Seat? - Updated

Note: this is an older article that has been updated a number of times.  Everything below the line is not very relevant to the upcoming election following the change in Prime Minister, and with Swan now on the backbench, the answer for many will be "who cares?"

Update: 31 August: The great poetic irony that was supposed to occur if Kevin Rudd's resurrection as PM saved Swan's seat is again in danger, with the seat jittery based on seat margin polling, reports of internal polling and now a public poll.  A JWS Research robopoll reports a 53.8:46.2 result in this seat in favour of the Coalition, although it should be noted that seat polls have been delivering generally nasty stuff for Labor compared to national polling.  By no means a surefire indicator of defeat for Swan, but it is better not to poll such numbers than to poll them.

=====================================================================

Advance summary of original article:


1. An opinion poll has been reported as showing the Treasurer, Wayne Swan, headed for defeat in his seat of Lilley.

2. While this finding may yet prove correct, there are many reasons to treat the finding cautiously.

3. These reasons include the likely closeness of the poll once preferences are considered, the poll's small sample size, the non-naming of the key candidates, and the poll being commissioned by an interest group.

4. With all these reasons considered, the poll provides no clear insight into the likely fate of Swan's seat, which is already known to be marginal and at substantial risk.

5. Polls commissioned by interest groups should be treated with much caution, because even if they are impeccably designed, groups may decide whether or not to release such polls depending on whether they like the results.

March Update: (More details at bottom of article.) Some similar comments apply to a second Lilley poll in early March, except that the key candidates are named, the sample size is larger, and this is the second released poll now claiming this result, so the level of overall doubt about the findings is lower.

The primary figures provided would produce a closer 2PP result than claimed.  The second poll's finding that a change to Rudd would result in an increase of about eight points is unreliable as polls in which voters anticipate how they would respond to hypothetical situations are generally not to be trusted.  Nonetheless a return to Kevin Rudd is capable of making a big difference to the chances of this seat being held. 

June 3 Update: Things are looking dodgier for the Treasurer now, considering the dire state of the national 2PP and the recent Galaxy Queensland poll which showed a state swing large enough (but only just) for the Treasurer to lose if uniformly applied.  An "internal poll" has been reported as showing him on a primary of just 28%, however the usual caveats apply to internal polling that has not been released in full detail and that may be being released selectively or be methodically suspect.

June 11 Update: However we now (see here) see the first publicly released polling result (Fairfax-commissioned ReachTEL) in which the Treasurer has been in front (53:47), although the lead in this sample is within its margin of error based on a sample size of 600.  This does not necessarily mean Swan has turned anything around; it is quite possible that all the published polls are within the neighbourhood of an actual picture in which the fight for the seat is very close.  Full details of the poll are yet to be released.
------------------------------------------------------------------------------------------------------------

Why Preferred Prime Minister/Premier Scores Are Rubbish

Note added 2020: This article now has a sequel.  See Why Better Prime Minister/Premier Scores Are Still Rubbish


Advance Summary of this Article:
1. All pollsters who currently conduct Preferred Prime Minister/Premier polling but do not conduct approval rate polling for each leader, should conduct leader approval rate polling either instead of PPM/PP or as well as it. 

2. Preferred Prime Minister scores have been historically maligned on the basis of a history of failing to predict election results.  Although frequently reported in horse-race style by mainstream reporters, they are often dismissed as "beauty contest" scores by informed psephologists.

3. When a "house advantage" to the incumbent Prime Minister is taken into account, Preferred Prime Minister scores align better with election results and voting intention.

4. Preferred Prime Minister ratings are driven by, and lag behind, Prime Ministerial approval ratings, and are not especially good forecasters of future poll results. 

5. The focus on Preferred Prime Minister/Premier scores not only leads to misleading commentary but also obscures useful data that are reflected in approval ratings but lost in scores that just compare leaders.

Disclaimer:  This article only applies to the usefulness (or otherwise) of comparisons between existing party leaders, such as PM vs Opposition Leader.  It does not (necessarily) apply to surveys comparing hypothetical leaders of the same party, or comparing an existing leader of a party with a potential leader of another, or to surveys like the recent Galaxy 4-way preferred prime minister poll.

--------------------------------------

Many pollsters ask voters to decide which of the Prime Minister and Opposition Leader they would prefer to be Prime Minister.  At state level, the same thing happens with Preferred Premier polling, and some pollsters (this includes you, EMRS) poll Preferred Premier but do not poll approval ratings.

The actual value of Preferred Prime Minister scores has long been a source of difference of opinion, and at times abuse, between some parts of the mainstream political media and the online "psephosphere".  In 2007 this got particularly heated when some mainstream journalists kept arguing that the election was in the balance because John Howard was not far behind Kevin Rudd as Preferred Prime Minister (Howard trailed Rudd consistently from mid-Feb 2007 onwards, but only by an average of seven points, at times closing to as close as one.)

Do Voters Care About AWU? A Review Of Polling

Advance Summary

1. The AWU "scandal" involving Prime Minister Julia Gillard's former links to a union-related "slush fund" recently dominated the news cycle and resulted in accusations of criminality by Opposition members including Tony Abbott against the PM.

2. Current polling shows that a movement to the Labor Party over the course of several months appears to have, at least temporarily, halted and probably slightly reversed.

3. Although the AWU debate may have contributed to the slowing of momentum towards Labor, there is insufficient evidence that it has caused damage to Labor's vote.

4. Questions surrounding the AWU affair by three pollsters show varying levels of design quality, and many are unsatisfactory.

5. Those polls that have produced the worst results for the Prime Minister are typically the worst designed, while the better designed polls show that the PM's handling of the issue is fairly well approved of.

6. At the height of Parliamentary debate about the issue, both the Opposition (for its handling of the matter) and its leader recorded very bad ratings, without pro-Coalition voting intention being affected.

7. The impact of the issue upon the standing of the Opposition Leader is not yet clear because of insufficient evidence.

8.  Comparisons between this issue and the Ozcar/Utegate blunder by Malcolm Turnbull in 2009 are simplistic as there are many differences in the nature of the issue and the existing popularity of those affected.

The LegCo's Claimed Reasons For Rejecting Same-Sex Marriage

(See also Legislative Council Voting Patterns since the last state election.)

Advance Summary:

1. Analysis of the speeches of the eight Legislative Councillors who recently opposed the Tasmanian Same-Sex Marriage Bill 2012 shows that at most five spoke against the concept of allowing same-sex marriage at this time in general.  These five were Tania Rattray, Dr Vanessa Goodwin, Adriana Taylor, Ivan Dean and Rosemary Armitage.  (Of these, in Rattray's case the evidence is arguable and limited.)

2. The remaining Legislative Councillors to vote against (Paul Harriss, Jim Wilkinson and Greg Hall) did not indicate an explicit view for or against federal marriage equality, but expressed reservations about the concept of state-based legislation and/or its delivery.

3. The defeat of the bill cannot therefore be interpreted as a rejection of the concept of marriage equality by the Upper House, and represents only the rejection of a given state-based proposal.

4. Every MLC arguing against the bill argued that it was a federal issue, and every MLC arguing against the concept generally supported the idea of marriage as being only between a man and a woman.  

5. Other common arguments against the bill included the risks, success chances and costs of a High Court challenge, the bill being a second-rate version of the concept, that the bill would not end discrimination, and that the State Government lacked a mandate.

6. Some MLCs voting against the bill made extremely unsound arguments about public opinion.  This suggests a need for politicians to be better briefed on how to (and especially how not to) understand and measure public opinion.