Wonk Central: Morgan's Tasmanian State And Federal Sampling

Advance Summary:

1. Roy Morgan Research has issued five recent Tasmanian SMS state polls with small but usable sample sizes in which the Labor Opposition has small primary-vote leads over the Hodgman Liberal Government.

2. Both state and federal evidence suggests these samples have house effects in favour of Labor and the Greens and against the Liberal Party and that Labor's "lead" is therefore probably not real.

3. While pro-ALP house effects are seen in Morgan's federal polling (which partly uses a face-to-face component known to skew to Labor), they are not clearly apparent in Morgan's state SMS samples in other states.

4. Morgan's release of a two-party preferred estimate for Tasmania is misleading, since most Green preferences are never distributed in the state, and even if they were they would not flow as strongly as Morgan's model suggests.

5. If Morgan's recent state samples were actually repeated at an election, the result would be not an easy Labor win as Morgan says, but a hung parliament in which the Greens would determine who governed.

6. Given the dissenting stance of current Greens leader Kim Booth during the previous Labor/Green coalition government, it is not clear who would govern in the event of another hung parliament.

7. An aggregate of all recent Tasmanian state polling does not currently point to a hung parliament if an election was held "right now", but is extremely close to doing so.

(Warning: This piece is very "numbery" and is rated Wonk Factor 4/5.)



------------------------------------------------------------------------------------------------------------------------
Welcome back to Wonk Central, the intermittent series in which I look at something that could only possibly interest a handful of very dedicated poll-watchers in far more detail than it can possibly deserve.

Just over a year ago, the Tasmanian Labor government was swept from office in the state's fourth most lopsided election ever*, with the party's lowest vote share since the advent of Hare-Clark in 1909.  Burdened by very long incumbency, a difficult economy and a hugely unpopular coalition with the Greens, the government copped a 9.6% swing against it, following the 12.4% swing against it in the previous election, and lost three of its ten seats.

Last week a Roy Morgan-SMS poll was released with Labor ahead on primaries in Tasmanian state polling.  Labor were on 41.5% (+14.1 from election), the new Liberal Government of Will Hodgman 39% (-12.2%), the Greens 15% (+1.2%), Palmer United a laughable 0.5% (-4.5) and Ind/Other 4% (+1.4).

The poll's sample was only 358, which means the readings for the major parties have margins of error of about 5%.  So as evidence that the state has seen this sort of swing against its new government, this poll alone is flimsy.  However, this is not the first time Morgan have produced this sort of result in the state.  In fact, it's the fifth in a row!  They had Labor leading on the primary vote 41.5-39 in November, 39.5-38.5 in January, 39.5-39 in February, 41-39 in March and 41.5-39 in April.  Indeed, far from the poll being prone to noise caused by random bouncing, this is a surprising level of stability given the small sample sizes involved.

There's little doubt the Hodgman Government has experienced a backlash since the state election.  At that election it would have gone close to maxing out its possible support level, bar the odd point or two peeled off by PUP.  Since the election it has been affected by drag from the unpopular Abbott federal government, but it has also suffered blowback from its attempts to cut the size of the public service.  The state's last Liberal majority government under Ray Groom (Premier 1992-6) went down a very similar path in terms of public service cutbacks and lost its majority after one term with a 12.9% swing.

However, it seems unlikely that the backlash would be that big after just one year. So what is going on?  Might Morgan be onto something here, or is their Tasmanian state polling simply nonsense?

(* The indicator here is difference between the primary votes of the two biggest parties.  2014 (23.9 points) trails only 1941 (26), 1992 (25.2) and 2002 (24.5).)

Morgan's Tasmanian State Samples

Determining the level of possible bias in Morgan's Tasmanian state samples isn't easy, because there is not a lot of other polling to benchmark against.  Morgan's first Tasmanian state sample was issued at the start of October 2014, and they have now released seven samples.

Since the 2014 state election the only other samples released by anyone have been a solitary ReachTEL in September 2014 and four quarterly EMRS polls.  Determining the house effect of media-commissioned ReachTEL polls for the Tasmanian state election in 2014 was a little difficult because they initially seemed to favour the Liberals, then in the end underestimated their vote, however there was at least one methods change along the way (the inclusion or exclusion of an Undecided option).

EMRS, however, is a pollster that has a history going back about 15 years in the state, and it is well known what it does.  At each of the last few state elections its final poll has had the Liberal vote about right, underestimated the Labor vote by a few points and overestimated the Green vote likewise.  A slight complication in using EMRS to benchmark is that it wasn't hired for a final week poll in 2014, but voting intention didn't change a lot in the last month anyway, and the differences between EMRS's final result and the election result in 2014 were very much true to form.

The table below shows: three actual EMRS results in bold, four interpolated EMRS figures (in grey cells), adjusted EMRS results for the differences between their Feb 2014 poll and the March 2014 election, the Morgan polls released so far, and, in the "M House" columns, the differences between Morgan and my adjusted estimates of EMRS. (Click for larger version).

By weighting the more recent differences more heavily while not completely ignoring the earlier differences, as of the February Morgan it appeared that Morgan were underestimating the Liberals by four points and overestimating each of Labor and the Greens by about two.   (Another method I've been using to estimate Morgan's skew produces a slightly wider gap, and the method I use to adjust EMRS scores in my main EMRS articles is slightly different too, but the above illustrates the point using the most recent election as the benchmark.)

If those house effects are roughly stable then the last two Morgans are very slightly negative news for the Hodgman government, providing a very weak indication that their lead might be slipping still further.  However, it's also possible that voting intention hasn't changed and that Morgan's house effects are larger than the previous polls suggest.  A new EMRS next month will allow a more accurate assessment.

There is a possible objection to all this.  While EMRS has a long history in the state, its previous polling history was entirely while Labor was in government.  EMRS tends to have high undecided rates compared to other pollsters, so perhaps this plays out differently when a different party is in charge.

Hence the following hypothesis, which I'll now proceed to test: If, as appears to be the case, Morgan's SMS sample in Tasmania has a strong skew to Labor, then this should also show up, to a reduced degree, in Morgan's SMS federal sampling that relies partly on SMS polling.

Morgan's Tasmanian Federal Samples

The first thing to note here is that it is great that Morgan actually publishes Tasmanian federal data at all - other pollsters tend not to release it, even if they release polling for all other states.  And while the Morgan state data for Tasmania is based on very small samples, it is possible to draw useful conclusions from it.

The reason I want to look at patterns with Morgan's Tasmanian federal samples is that they partly use SMS polling.  If Morgan's SMS polling has a skew to Labor in Tasmania at state level then it would be expected that this skew was at least partly present in its federal polling.

Morgan is a confusing poll to follow for casual poll-watchers, because in fact the company produce polls by a wide range of different methods.  In the last three years they have produced polls by face-to-face, SMS, phone and at least three different variants of "multi-mode".  The Morgan polls that include face-to-face (which includes two of the multi-mode variants seen so far) have displayed a skew to Labor, even when preferences from the previous election are used.  Currently, the skew to Labor in Morgan's multi-mode series seems to be about 1.5 points to Labor.

A further issue with Morgan is that their federal polls use a headline figure that is derived from preferences offered by respondents.  In this term, this has skewed polling further to Labor by about 0.75 points on average, compared with using last-election preferences.

Therefore, in considering Morgan's federal Tasmanian samples, it's unrealistic to just look at the Tasmanian 2PPs compared to the actual Tasmanian result at the federal election, since the former might be skewed by the non-SMS part of the Morgan sample.  Rather, it's important to look at the Morgan Tasmanian 2PPs as compared to Morgan's national 2PPs at the same time.

2013

Morgan have published Tasmanian federal results that include SMS results in two recent bands.  The first commenced in June 2013, not long after they switched their regular federal poll series from the purely face-to-face poll to a "multi-mode" poll (at the time a mix of internet panel, SMS and face-to-face).   In the leadup to the 2013 federal election, ten results were published.

The following graph shows the relationship between Morgan's Tasmanian and national 2PPs for the ALP in that period (which started with the June 21-23 poll just before Julia Gillard was rolled, and ended with the poll taken August 23-25).  The 2013 election result is shown at the far right.

Every Morgan poll in this period had the Labor federal 2PP in Tasmania in 2013 above the eventual result of 51.23%, but this is partly because Labor's 2PP declined nationally during the campaign, and partly because of pro-Labor house effects.

The lines showing the difference between Morgan's Tasmanian and national Labor 2PPs at the bottom of the chart show that at the start of this period (the last week of Gillard and the first under Rudd), the Morgan sample had Labor in Tasmania way ahead of their national average.  At the time of the election the difference was running at seven points, but the actual difference on the day was 4.7.

Given the small size of the Tasmanian samples in this period (typically well under 200) this proves nothing by itself.  But it is, at least, consistent with the idea that something was causing the difference between the Tasmanian and national 2PPs in Morgan's polling to be greater than it actually was.

An obvious objection is that Labor's 2PP might have fallen further in Tasmania in the last two weeks than elsewhere.  However, there is not a lot of juice in that.  Other pollsters certainly weren't getting the same thing.  The BludgerTrack aggregate (slow-loading link shows Wayback archive of final 2013 election reading) had the difference between Tasmania and the national picture, if anything, increasing as election day approached.  (BludgerTrack nailed the national 2PP exactly and had the Tasmanian 2PP for Labor just a point above the actual result.)

There was also one SMS sample taken during this period.  Taken in between the first and second samples above, it had Labor on 63% in Tasmania compared with 49.5% nationwide.

2014-5

Morgan resumed publishing federal data with state breakdowns in late February 2014, this time for a multi-mode series consisting of SMS and face-to-face polling.  Apart from the 1.5-point Labor skew (which is easy to adjust for), this has looked like a good poll so far.  It is slightly less bouncy than the live-interview phone pollsters and much less prone to go off on prolonged tangents to the trend than Essential.

In Tasmania's case, the resumption of Morgan's state figures came just prior to the state election.  Here's the same graph as above for the period since Morgan started publishing federal state figures again.  (A few notes: I have assumed a figure of 51% to Labor in Tasmania for the third poll in this series, in which their Tasmanian result was not published so far as I know, but was described as a narrow lead. I also omit the June 7/8 one-week sample (60.5% to ALP nationally) because no Tasmanian 2PP for that sample was provided).

This graph shows that mostly through this period the Labor 2PP at state level (dark green) has run above the Labor 2PP at national level (blue), hence the smoothed difference line (red) has been generally positive.  However this was generally not the case for the first eight polls, up to early June 2014.  The BludgerTrack state tracking graphs show that between about January and June 2014 there was a blowout in the Coalition's favour and against Labor in Tasmanian federal samples.  It looked like state politics contaminated federal polling in the state until such time as disgust at the federal 2014 budget took over and the state election receded from the picture.

If we consider polling from that point, the average difference has been 4.9 points, though even the smoothed average (covering five surrounding fortnights) is noisy and has run over seven points at times.  While 4.9 points is very similar to the gap at the last federal election, BludgerTrack's average 2PP for Tasmania for the year April 2014-March 2015 was about 53.8, while Morgan's was 58.9.   At most half of this difference is explained by Morgan's house effects, so it could well be that the difference between the national and Tasmanian Labor 2PPs is now lower than at the last election, and that Morgan is not picking this up because its Tasmanian SMS sampling has a Labor skew.

While there could be many reasons in theory why that difference might reduce, there is at least one reason why it should reduce, all else being equal.  That is that at the 2013 federal election, Labor lost the personal votes of three incumbent MHRs, two of them long-serving, who were defeated.   The new Liberal MHRs would now be building their own personal votes.

Thus, the available indications from both the 2013 and 2014-5 state sample runs are consistent with Morgan's Tasmanian SMS pool being skewed to Labor.  They don't prove this is the case, but they could disprove it, and neither does.  Indeed, both suggest about the level of skew in the SMS part of Morgan's Tasmanian sampling as is expected based on the state picture.

Why this might be, it's hard to say, since the other states don't seem to have this problem.  But a small state pool is more likely to be skewed than a large one.  The unusual stability of the Tasmanian state results so far is also consistent with the Tasmanian pool being small.

The Poll Equivalent Of Junk Food?

Since Morgan introduced state SMS polling late last year, the series has had a very bumpy ride.  In the Victorian state election the poll was extremely skewed to the Greens until the final poll, and was volatile between the major parties.  In the Queensland election its final poll was OK, but the NSW election was another disaster for the method with a rogue final poll that got the difference between the major parties wrong by 8.5 points.

This has led to some perception that Morgan-SMS is basically a quick rubbish poll with no predictive value.  I doubt that it's quite that bad (especially given the sparseness of state polling of any kind outside of election times), but when a poll has a history of volatility and a tendency to go way off the path of polling trends shown by others, it makes sense to take whatever it says with a great deal of caution.  For this reason while I'll keep using these polls in my assessments of state pictures for now, I can only give them very low accuracy weightings. They're like junk food or snack polls - OK if you're starved for data, but otherwise not very nutritious.

Aaargh, the 2PP ...

Oh yes.  Now this bit is annoying.  Each of the Tasmanian state poll samples so far has come with a Tasmanian 2PP estimate, so far ranging from 52% (for the poll with Labor seven points behind on primaries) to 56% (for the current sample.)

For those not aware of Tasmania's electoral system, we elect five members from each of five electorates, using the Hare-Clark system of (vaguely) proportional representation.  Unlike the mainland states, in which Greens candidates are excluded and their preferences distributed in almost every electorate, in Tasmania most Greens preferences are never distributed.

At the 2014 state election, there were only two large distributions of Greens preferences: in Braddon (where all five Greens were eliminated) and in Denison (where second Greens candidate Bill Harvey was excluded with .38 of a quota to his name).  There were also preference leaks outside the ticket from minor Greens candidates, though these included votes that came from outside the ticket to begin with.   All up around 18% of Green primary votes were actually passed on in a case where they had a choice of Labor, Liberal, perhaps some other non-Green party or exhaust.  And as with optional preferencing (OPV) in Queensland and New South Wales, some of those exhausted.

A similar story in 2010 where the Greens had surplus votes after their first candidate crossed in four electorates, but in one of those by the time the Green #2 was excluded, there were no Labor candidates left in the race.  And even more the case in 2006 in which the Greens were either last elected or defeated without being cut up in three of the five electorates.  So in the last three state elections, we've only had from 13-18% of Greens preferences distributed outside the party.

On that basis, even if we assume some friendly figures for Labor, like an 80:20 flow of continuing preferences with 33% exhaust, that friendly flow should only lead Labor to expect a one-point leg-up from Greens preferences statewide.

Morgan know that two-party preferred is a useless measure in Tasmania.  They even write "Two-Party preferred vote is not applicable to the Tasmanian lower house which uses the Hare-Clark proportional voting system".  So having said that it is not applicable, why do they insist on publishing it? 

And given that they do publish it, why on earth are they giving between 70 and 75% of all minor party preferences to Labor, when even if all the minor party votes were Green and all of them left the party, the percentage not exhausting under what is effectively OPV wouldn't even be that high? (The average preference flow between the two electorates with major Green preference exclusions in 2014 was 48.5% Labor, 10.5% Liberal, 41% exhaust, so the 2PP for the last poll should be something like 54.5%, not 56%.)

One does hope Morgan doesn't think their 2PP model is actually useful as a model of who would win the election. In reality, all of Morgan's seven samples so far, if reproduced exactly at an election, would result in a hung parliament with the Greens determining who would govern.  Current Greens leader Kim Booth was never happy with the coalition with Labor in the last term and there is no guarantee the party under his leadership would put Labor back into office, especially with Labor apparently refusing to provide Cabinet positions in the future.  There is also no guarantee, on the lower estimates of the Greens vote, that Booth himself would keep his seat.

(A bit of explanation re the first two Morgan samples: sometimes a party can win majority government with a narrow primary vote lead over the other, as happened in 1998 when Labor won 14 seats with a lead of 6.7 points.  I've also found that the Liberals might win the next election outright with a primary vote lead as low as 7 points.  However, this only works when the Green vote is low enough to keep them down to 1-3 seats, and not when it is 19%).

Compounding errors

So what in the end do we make of five consecutive Morgan samples in which Labor would supposedly win the election easily?  Well, that's what you get when you fit an inapplicable preference model with preferences that wouldn't happen that way even if they could, to primaries that appear to have a house effect - all of these things favouring the same party.  It's not even "garbage in, garbage out", since that expression applies most often to the sound transformation of very suspect data.

However, even after adjustment for apparent house effects, the five most recent Morgan samples would each probably produce a hung parliament if those results (as adjusted) were polled at an election.  I've been adding them to my state aggregate, at very low weightings indeed, but they have had no impact at all on the overall seat distribution and only tenths of a point here and there on the primaries. The latest Morgan leaves my aggregate unchanged at 13 Liberal, 10 Labor and 2 Green, but it is very close to tipping over to 12-10-3.

This piece may be updated later as further evidence on the behaviour of this poll series comes to hand.

May 28: A new Morgan sample out yesterday has been added to my calculations.  In the table above, the "average" line changes to Liberal -5 Labor 2.5 Green 2.1 Other 0.5 and the "Trend" line changes to Liberal -5.3 Labor 3.5 Green 1.8 Other 0.  So the evidence of the new Morgan sample (which showed a sharp swing back to Labor, as did the EMRS) is that the Morgans might be even more skewed than this article first suggested.  It is, however, possible that voting intention changed only in the last month, in which case the skew probably hasn't changed.