Underwhelming Result In Wilderness Poll

Porcupine Fish Award for Ultra-Fishy Polling (image credit)

Advance Summary

1. Recent media articles have carried some reports of a poll said to show support for protection of the Tasmanian Wilderness World Heritage Area, and at least partly commissioned by the Bob Brown Foundation.

2.  Such limited details of the poll as are available show that the poll wording was ambiguous and implied unfounded conclusions in a way that probably pushed respondents towards certain beliefs.

3. Also, the details of the polling thus far publicly released are insufficient to be sure that prior questions were not used to train responses to later questions.

4. The result for protecting wilderness is in my view surprisingly low, and might be explained by negative reactions to the use of unsavoury styles of polling.



=======================================================================

Last week I started seeing negative mentions on social media of a commissioned robopoll about the Tasmanian Wilderness World Heritage Area.  A few respondents, including regular site commenter intuitivereason, reported being asked poorly framed questions that were unfit for yes/no answers and seemed designed to create dodgy results.  The term "push-poll" is often thrown around for polls that aren't actually such, but in this case the comments suggested this was an especially dire example.

Well it hasn't taken too long for some results of this exercise to surface.  It's a commissioned poll by the Bob Brown Foundation, with robopollster Lonergan doing the polling. As usual with such polls, responsibility for the content of the question lies with the commissioning source and not the pollster (beyond their willingness to accept a contract rather than refuse it).  Results of questions 2 and 3 have been released by the Foundation.  It is unknown whether the Foundation was the commissioner of all questions asked, or whether other environmental groups may have also been involved.

The release does not say what the other questions are, nor how many there are (I'm advised it was about a dozen), but I have been told that the first question was not voting intention (which if true, is unusual - many issue polls ask voting intention questions first to ensure the sample is accurate).   Without knowing what question 1 was, it is impossible to say whether it could have contaminated responses to those after it, and that alone is reason to consider the release of the poll results by the Foundation so far to have failed due diligence.

The next odd thing about the poll is its sample size.  Normally polling 2,200 respondents for a commissioned issue poll would be an expensive case of accuracy overkill.  Compared to a sample of 600, it's hardly worth polling four times as many people to knock the margin of error down from four points to two when dealing with issue questions on which the difference betwen "yes" and "no" is generally at least 20 points, and in this case more.

Since the original version of this article I have had feedback that a benefit of large sample size for robopolling is that some demographics such as young voters are difficult to reach when dialling random numbers, and if the overall sample size is modest then the number sampled in those demographics will be tiny, and among other things will magnify errors upwards when results are scaled.  I'm also advised that Lonergan's robopolling price plan is reasonably flat.  Fair enough; for these reasons the above paragraph has been toned down!

Push and skew polling

I frequently distinguish between two kinds of bad polling practice:

push-polling: Push polling is the use of what appears to be polling to push a political message.  The "sample" size is typically large and the "results" are discarded as the purpose of the poll is propaganda.

skew-polling (my term for it): Skew-polling is the use of misleading preambles and question wording techniques to skew the response of respondents, including the use of lead-in questions to "train" the respondent towards a particular answer. The sample size is typically modest.  The "results" are used to create pressure for a cause by exaggerating public support for it, or to create fodder for media releases so that media will report the commissioning group's views on an issue.

While nearly all polls that are called "push-polls" are actually either just skew-polls or "message testing", it's possible in theory for a poll to both push and skew.  Given the suspicion this poll is generating, apparently with good reason given what of it is visible, all the question wordings and responses should be released.

Questions thus far available

On to the two questions available so far:

Question 2
Tasmania's Hodgman state government proposes to remove all references to "protecting wilderness" from the Tasmanian World Heritage Area management plan.  Do you think the Tasmanian Wilderness World Heritage Area should continue to have its wilderness protected?

Firstly, this is quite an absurd question.  Intentionally or not, it blatantly pushes the implication that because the government proposes to change the way management objectives are referred to in a plan, that therefore it intends to actually cease protecting all wilderness within the WHA.  It is not surprising that a question such as this is viewed as push-polling.

Secondly here, just what does a yes response to this question actually mean.  Does it necessarily mean that the respondent believes, for instance, that there should be no tourism in wilderness areas?  Is it consistent with protecting wilderness to allow increased access to it, if the wilderness is not significantly damaged and remains wild?  Is "protection" a matter of total avoidance of impacts, or is it interpreted substantively?  If someone thinks the wilderness should be protected from a wide range of possible impacts while allowing one or two minor impacts, do they consider that the WHA should have its wilderness protected, or not?

"Protect" is such an ambiguous word that argument over its meaning formed an important part of one major Tasmanian forestry court case (Wielangta) and even then the judge ended up trying to extract what Australian law meant by the concept by dredging various inferences out of international treaty waffle we had signed, and ended up with something pretty silly.  (The case was overturned on appeal, though not specifically for that reason). It is especially ambiguous as applied to wilderness, because it is possible to alter the wilderness quality of an area without touching it, but whether this has actually happened or not depends on whose definition of wilderness you're using.

I reckon virtually all Tasmanians believe that the wilderness WHA - at least prior to recent contentious additions - should remain in large part a reserve.  I reckon there is virtually no-one left, bar a small nostalgic Robin Gray rump, who really still believes the WHA should be extensively mined, dammed, logged or turned into shopping malls.  Of course there is disagreement about how large the WHA should be, what should be allowed in it and what protections should exist.  But that genuine wilderness in the WHA should receive some level of protection from some levels of impact, and that for those areas reserved means reserved is surely as close to a no-brainer as you would get in modern Tasmanian politics.

Hence it was surprising to see just how low the yes results were. Only 72% agreed with a nebulous request for the wilderness to enjoy at least some vague level of protection. Whether the rest said no, or whether there was a don't-know option, isn't stated.

Click for larger version.  (It isn't relevant to a critique of the poll but it is interesting that the poll's demographics are extremely heavily weighted towards respondents over 50 and had around twice as many female subjects as males (the pollster has corrected for these biases correctly).  It gives a useful insight into what robopollsters are up against in scaling their samples.)

Question 3
The Tasmanian Wilderness World Heritage Area, named in 1982, is the only world heritage site in the world to have the world 'wilderness' in its name.  Now it has been proposed that the word 'wilderness' be removed or replaced in the name. Do you think the World Heritage Area should have its name changed?

This question is rather innocuous compared to Question 2, although the accuracy of the second sentence is under dispute, with claims that the intention was co-branding or dual naming.  Anyway the result is given as 76% against changing the name so as to remove all reference to "wilderness" from the name.

If other question wordings and results are released or fall off trucks I will report them.

The Foundation Has Form ...

This is not the first time the Bob Brown Foundation has commissioned polls with dubious wording.  In the noise of the state election campaign it escaped due censure after trotting out a commissioned Galaxy (online, Australian residents from outside Tasmania) that asked as follows:

"It is proposed to remove 74,000 hectares of forests from the Tasmanian Wilderness World Heritage area to allow it to be logged.  Which one of these options do you prefer?"

Options were:

Retain the protection to prevent logging in the 74,000 hectares of forest

Remove the 74,000 hectares from the World Heritage Area list to allow it to be logged.

Neither/Don't Know"

The wording could have created the impression that the whole 74,000 hectares of forest would be logged if the excision was accepted (highly unlikely in reality) and perhaps even the impression that the broader WWHA would be at risk of logging if the excision was passed.

These quibbles aside, the bigger issue here is that the debate was not simply "hey, let's kick bits of forest out of the WHA so we can log it."  Rather the 74,000 hectares had been very recently and contentiously added through the actions of a state government coalition that was widely seen as lacking political legitimacy (and was indeed drubbed in its re-election attempt), as a result of a process that had far more to do with politics than science.  Objectors tried to unconvincingly maintain that the presence of some small degraded areas within the 74,000 hectares was a major objection to the addition, but at base the debate was about whether those areas should have been nominated in the first place.  Nationwide audiences answering the poll, who would mostly not have heard any of this before, would not have been aware of the full background.

...And Pals Who Don't Like, Or Get, Criticism

The rest of this piece goes into rant mode just a tad.

On seeing the results of this poll, a tweet from me that said that given the question design, the result for Question 2 was "pathetically low" resulted in an immediate troll attack from Adam Burling.  Burling, a long time "forest defender", is married to Jenny Weber who is Campaign Manager with the Foundation.  Making no attempt to address the point, Burling then dished out numerous ad homs including that I was a "great defender of logging", that I had a history of attacks on "Bob Brown, the Greens and forest campaigners", that I engaged in campaigns for "forest and wilderness destruction" and so on.

Now, it's true that I have a history of criticising anti-logging campaigners who don't have their scientific facts in order, and it's true I've given the Greens all manner of grief in my non-pseph comments down the year (though I've done much the same, on various issues, to just about any party that exists).  But I don't really have a personal history of getting stuck into Bob Brown specifically all that much; indeed it is Brown who once got stuck into me, in the Senate, with his facts riddled with errors.  (I don't hold this against Bob; he did read a correction letter into the Hansard, though he ideally should also have apologised).

All Burling could come up with as evidence of me attacking Saint Bob was "you well know, your long term attacks on forest campaigns which includes his work". Yet when I mentioned Brown's own prior form in coward's castle, Burling's response was "cry me a river".  Apparently, Bob is the Messiah; any disagreement on forestry policy or campaigning tactics means you are personally degrading His Holiness, but since His Bobness can do no wrong, if anyone is criticised by Brown they must have deserved it.  If I was Bob I'd be beside myself in despair at having such pathetically blinkered acolytes.

Shortly after this Burling made a complete goose of himself by calling Australia's greenest psephologist "not progressive" for defending my critique of an obviously crap poll.

My actual views on the issue

I suspect Burling carried on as he did because he believed I was supporting expanded logging in the WHA or some other attempt to dismantle it.

In fact I don't.  I believe that it is very important when areas are reserved to know that reserved means reserved, and very important to be vigilant against any process in which it suddenly means something else.  That's not to say the Government is doing any such thing, claims to that effect looking like they are largely beatups, but we need to look at the proposed changes carefully.

That said, I believe the Government's current WHA tourism review may well identify excellent tourism proposals that will not do serious harm (and I am baffled that some green figures are supporting a concept that by normal green standards would seem very obtrusive, for Recherche Bay, while opposing others for seemingly random reasons.)  There will also be a lot of silly pie-in-the-sky ones, and hopefully those will be rejected.

Frankly, if someone from the alleged white-shoe brigade wanted to find ammunition for the idea that Tasmanians were not that big on the status of the WHA, they would need to look no further than this clumsy poll.  They could say that 28% weren't even convinced the WHA needed any protection, let alone the levels demanded by purist wilderness-obsessed zealots.  They could say that given the poll's manifest design flaws, even the 72% for protection in some form was an overestimate, and that maybe Tasmanians just weren't that keen on having big reserves in the south-west after all.  And so on.

The reality, I suspect, is that support for wilderness protection of some kind is much higher than this poll shows.  Most likely, this crude style of polling is so anathemic to many Tasmanians that some gave negative responses just to express their disgust with it.

The Bob Brown Foundation therefore joins PUP's internal polling and the Mt Wellington Cable Car opt-in as recipients of this site's ultimate accolade for dubious polling, the Porcupine Fish.