Uneducated Preferencing: NTEU/UMR Robopoll

Wirrah Award For Fishy Polling (image source)

This article initially covered the report in the Examiner today of a poll commissioned by the National Tertiary Education Union and said to show first-term incumbent Andrew Nikolic trailing in the federal seat of Bass.  However it has since become clear that this poll is part of a national bulk robopoll of 23 marginal (and in some cases not so marginal) electorates and raises enough issues that I've deemed it necessary to rewrite the article and give it a national focus.

The NTEU is not a slavish supporter of the ALP (for instance endorsing independent Andrew Wilkie at the last election) but it is obviously an opponent of the current government's education proposals.  The poll was conducted by UMR, which is best known as the Labor Party's standard pollster for internals (yet is described by the NTEU as "independent"), as part of a large series of national seat robopolls. The NTEU has now published a full data set for all seats (download from link).  The poll claims to find an average 2PP swing of 11% to Labor in the 23 surveyed seats, from 52:48 to Coalition at the election to 41:59 now.   Simultaneously the NTEU has released a message testing poll taken nationally via online panel sampling also by UMR and taken in late May to early June.  The 2PP in that sample is given as 56% to Labor.



Neither poll release states how preferences are distributed, but it would seem that they are using the Labor-favouring (and historically less accurate, in most cases) method of respondent-allocated preferences and that this inflates the results in favour of the ALP.  This is the only way other than freaks of rounding to explain how a swing of 10% on primary votes to the Coalition, with 8% going to Labor and 2% to the Greens (plus a swing of 2% from Others to PUP) becomes a national swing of 11% to Labor, rather than something more like 9.7%.  Looking at the individual electorates it is possible in some cases to clearly rule out rounding as the explanation.  For instance in Bass, 42% Liberal 40% Labor 11% Green 5% PUP and 2% Other becomes 52:48 by national last-election preferences rounded to a point, or 53:47 if Bass-specific last-election preferences are used.  No assumption about rounding gets it up to 55:45 by last-election preferences.  Likewise, while the 2PP for the online panel sample was given as 56, by last-election preferences it comes out to around 54% for Labor (which would be more accurate for the time of polling.)

When last-election preferences are taken into account, some of the findings of the mega-robopoll become less clearcut.  For instance while a 2PP of 45:55 would indicate Nikolic to almost certainly lose his seat in an election "held now", victory is within the outskirts of the poll's margin of error if (as I maintain) his 2PP on these figures should really be placed in the 47-48 range.  Similar comments apply to Bonner (stated 2PP 45 but last-election 2PP 48).  Aside from those, if the poll's findings were broadly correct Labor would likely win all the polled seats, give or take the odd seat for extreme sample error.

The overall finding of an 11% swing has been described by William Bowe as "hard to credit".  I agree. Readers may recall the series of JWS robopolls commissioned or co-commissioned by a firm with Liberal Party connections that showed exaggerated swings to the Liberals in aggregated marginal seats in early 2013.  Subsequent neutral robopolls by a range of pollsters also showed Labor in more trouble in marginal seats than nationally.  The idea that Labor would do much worse in the marginals than in either side's safe seats in swing terms turned out to be nonsense, with the main explanation for it being that the robopolls (excluding Galaxy's) were displaying large pro-Coalition house effects.  This has a whiff of being exactly the same thing in reverse about it, and I would treat even a last-election-preferences based swing of 9.5% or so in these seats with very much caution given that my national aggregate showed national swings of "only" about 6.7% against the Abbott government during the time the poll was conducted.   That the poll is commissioned by a non-neutral source provides an added reason to be wary, especially given that it is a robopoll and that there is not much on the public record from which to assess the quality of UMR robopolling.

The poll was reported by the Examiner as showing a rise in Nikolic's approval rating within the seat of Bass of 8% and a fall for Tony Abbott of 25%, but it turns out +8 is Nikolic's net satisfaction rating; I'd assume the -25 is Abbott's netsat within Bass.  For the full range of seats polled Abbott's netsat comes out as -31, very similar to recent Newspolls.

The poll has been reported as showing high levels of opposition to cutting university funding, increasing student fees and providing government funding to privately owned universities.  Andrew Nikolic has disputed the poll findings (other than the voting intention finding) claiming they are based on misleading questions.  The questions are put in the following manner:

"The following are measures that will be introduced in the recent federal budget, please indicate if you support or oppose each one:

Students' university fees will significantly increase
Cutting public funding to university courses by 20%
Providing government funding to privately-owned higher education institutions".

Time is short and I hope to provide an objective view of whether the questions are valid or not later.  For the time being, no specific evidence that they are not valid has been seen, and the complaint from Mr Nikolic seems to be that they do not address the full scope of the Coalition's budget changes.