Holding The Ball: Polling And The Proposed Stadium

Summary: There is adequate evidence of strong overall opposition to the Macquarie Point stadium proposal, but most of the individual polls being cited are unsound.

--

A proposed stadium at Macquarie Point has now become a major Tasmanian political issue.  The proposed stadium, intended as part of a deal for Tasmania to finally get an AFL team, has been so divisive that two Liberal backbenchers quit the party citing concerns over the stadium approval process, taking the Rockliff Liberal Government into minority.  Unless approved or killed off by then, the stadium is highly likely to feature as an issue at the next state election.  

The stadium becomes the latest in a long line of Tasmanian contentious development proposals - the Bell Bay pulp mill, the kunanyi/Mt Wellington cable car and the Ralphs Bay canal estate proposal being some prior examples.  Typically these have in common that they greatly polarise the community for a long time and suck a lot of oxygen out of other political issues, but also that virtually none of them end up going ahead.  Something else they have in common (and share with some other long-running controversies such as old-growth logging) is that they inspire a lot of mostly terrible polling.   On this site I previously published reviews of polling about the cable car and polling about the pulp mill showing that the great majority of polls on both these issues were biased and/or of poor quality.  

With polling on the proposed stadium having ramped up to the point that at one stage there were at least three polls in the field about it simultaneously, I thought it was time to start a rolling article that over time will cover every poll I see about this issue (at least those for which results have been discussed in media, but perhaps also those I have simply found the wording of).  In place of my normal fish awards for smelly polls, I will be awarding footballs to polls I consider to be good, and whistles to polls I consider bad.   A poll may be awarded footballs, whistles, both or neither, together with a flippant "umpire says" which is an excuse for me to demonstrate my limited knowledge of AFL metaphors.    The more footballs the better, the more whistles the worse.

Whistles can be awarded, for instance, for the following:

* the poll is conducted by an unreliable pollster or polling method

* the poll employs bad (not just debatable) question or answer design

* there has been inadequate transparency regarding the poll

* it actually isn't a poll at all

A poll that receives a whistle for bad question design cannot receive any footballs at all.  A poll that is exceptionally bad will receive four whistles (equivalent to the Washington Post's Four Pinocchios) irrespective of how many of the above boxes it ticks.   Awards may be changed (for instance if I slam a poll for poor transparency but results are later published or sent to me) but I won't necessarily do it quickly - because everyone commissioning polls should be releasing them properly as soon as they discuss any result of the poll in public.  

How To Be Best And Fairest

In this introduction section I just want to explain some principles of sound polling design.  Issue polling is actually extremely difficult to get right even when a pollster or commissioning source is doing their very best to get the question right.  But many people involved in issue polling commission questions that can skew the response to their side.  Common issues here include the use of skewing preambles, the provision of arguments for one side or the other, and the inclusion of facts that the respondent may not actually know.  Many left-wing groups in Tasmania have a very long and disgraceful history of using longwinded preambles and/or biased questions that focus the respondent's mind on some aspects of the issue while ignoring others.

Likewise I have seen many stadium supporters insist that the question should make it clear to respondents that the deal for an AFL team will not happen without the stadium going ahead.  But firstly if a respondent currently opposes the stadium but doesn't know it is connected with the team, then they are opposed to the stadium and should be marked as such.  It might be that telling this respondent of the connection changes their mind, but in that case the respondent ceases to represent others who were not aware of this argument, and the sample ceases to represent the public at large.  Furthermore the respondent might not actually be convinced that the team and the stadium are linked.  Even if the respondent's view that the AFL if pressured will agree to something else is completely wrong, it's a view that's out there and it is wrong to discourage the respondent from answering accordingly.  (Of course there is nothing wrong with conditional follow-up questions - so long as they are clearly reported as such, including by media.)

A further pitfall is the framing of questions in an agree/disagree format, because acquiescence bias may drive up the "agree" response.  

If you actually want to know whether people currently support the proposed stadium or not then the way to find that out is to ask them exactly that, and since it is the primary controversy at the moment, it should be asked before any other questions about the stadium or AFL (ideally, just after the voting intention question that pollsters should in general ask first.)  

My idea of a good question is something like:

"Do you support the proposed construction of a stadium at Macquarie Point"?

I include the "at Macquarie Point" because this eliminates any confusion arising from any other stadium proposals, and also because some of the opposition to the current proposal is specific to its proposed location.  

The Roster

Here is my list of polls and poll-shaped objects that have taken the field in this season, in rough order of me becoming aware of them.

In summary as of May 2023 there is sufficient evidence of strong overall opposition to the stadium. 

1. Tas Labor October 2022

Reported result: 67.3-16.6 against.

See previous full-length article.  In summary, the poll used a skewed question drawing attention to the cost, the pollster has a poor and limited public polling track record and uses primitive weighting.  Oh and doesn't know how many Tasmanians there are.  

Umpire says: Out of bounds on the full.

2. Big Issues Survey (not a poll) late 2022

Reported result: 69.46% opposed (forced choice so presumably 30.54% for)

The Mercury's Big Issue survey may have a solid number of respondents (around 3200 in this case) but it is still not a scientific poll.  Those who respond to this survey are most likely to be southerners and politically engaged.  A similar reader poll in the leadup to the 2021 state election found voting intentions (after redistributing undecided) of Liberal 36.5 (actual 48.7), Greens 20 (12.4), Independent 20 (6.3) and Labor 18.8 (28.2).  In fact this was much closer to the results for the very left wing seat of Clark than for the election as a whole.  Aside from it being an unrepresentative and unscaled sample, this question is extra-useless because forced choice yes/no options were unnecessarily sprinkled through the survey, and also because newspapers commission surveys of this kind then can't be bothered publishing a permanent record of what the questions were.

Umpire says: Fifty metre penalty.

3. Our Place May 18 and 22 2023

Reported result: 61.3-34.1 against

This is a widely reported commissioned poll for Our Place, a group of opponents of the stadium who are spruiking an alternative vision for the site.  The primary problem here is the question:

Do you support or oppose the Tasmanian Government spending $700 million on a new stadium at Macquarie Point in Hobart?

This question in effect gives an argument against the stadium (it will cost a lot of money) while not giving any argument for it, such as any possible economic benefits, and it is therefore a biased question.  The factual correctness of the question is also debatable - the federal government has said it will contribute $240 million, although there is debate about whether the state government will end up effectively bearing this cost through reduced GST receipts.  (In the other direction, it is London to a brick that the project will blow its current budget.)

The other major problem here is that the pollster, uComms, recorded a dreadful result at the 2021 Tasmanian state election where its robopolling for the Australia Institute, the sole poll released during the campaign, underestimated the Liberal Party primary by 7.3% and generated wrong predictions of a hung parliament.  There has been no explanation of why this poll - among the worst errors in a state poll that I have seen - was wrong but I would guess primitive scaling and poor response rates were big parts of it.  

This poll result is also feeding a view that young voters are more supportive of the stadium than older voters based on a 39.5% support rate among 18-34 year old voters.  But in fact there is a very common problem with this age group producing right-wing responses in uComms polls, perhaps because nobody much in this age group answers a robopoll.  As for the follow-up question claiming to show 57.8% would be less likely to vote for a party supporting the stadium to 34% more likely, the idea that this proves the stadium to be an electoral kiss of death is absolutely ludicrous.  The respondent by this stage has been primed with a question about the cost and a question about building 1000 houses on the site instead, and voters frequently overestimate the impact of specific issues or else engage in cheerleading when asked to express a view on an issue in isolation.  (Voters may say they are more likely or less likely to vote for a party they were certain to vote for or not vote for respectively.)

Umpire says:  Free kick to team stadium.

4. EMRS May 15-19 2023

Reported result: About two thirds to one third against

This is a commissioned poll attached to the EMRS quarterly omnibus (which includes public voting intentions polling) and I understand it to have been commissioned by tourism/hospitality groups.  On the good side EMRS is the only poll with a good public polling record to have yet been used to poll the matter, but at this stage there has been nowhere near enough disclosure of the results, with only vague descriptions of the numbers and no verbatim details of the question.  There is also a minor methods issue, which is that respondents were asked first if they supported an AFL team (apparently about two-thirds did) but that has some potential to affect the stadium question results.  And, because the poll result has been transmitted through Tasmanian politicos rather than being published, it's not clear whether the reported two-to-one margin is with undecided excluded, or whether it was a forced choice question.  A press release mentioning this poll claimed it to have showed support for the stadium had doubled, but no previous EMRS poll on the matter is known, and I suspect this is only a comparison to the skewed Labor Community Engagement poll which had a substantial undecided rate (and therefore not even close to true).  

Umpire says: Boundary throw-in.

5. Mercury reader "poll" (not a poll) May 2023

Reported result (current): noting that this one in particular is utter bollocks, 60-40 in favour

The Mercury has unfortunately besmirched its current coverage with an ongoing online reader "poll" that asks "Would you support abolishing the stadium even if it meant not having a Tassie AFL team?" and allows options of "No, we need a team. It has been 30 years in the making" and "Yes, I don't care about AFL".  Aside from the usual problems with such reader "polls" (including in this case that only those who are subscribers or can get around the paywall can vote) the problem with this one is that the answer designs are extremely biased.  The reason given for answering No is one that anyone who supported the stadium would hardly disagree with, and this answer effectively cheerleads for the stadium.  The reason given for answering Yes, however, is one that many opponents of the stadium would not agree with - there are plenty of people who love football but still think the stadium is wrong.  The idea that one can abolish something that does not exist and has not even commenced an approval process may also intrigue those whose Mastermind special subject is the life and works of Bertrand Russell but makes very little sense to the rest of us.  

Umpire says: Reported to tribunal.

6. Claimed Mystery Poll Feb 2024

Reported result: about 60-31 against (Lyons missing)

A mystery poll with a sample size of around 4000 was reported by The Mercury; I have heard it was conducted for the Tasmanian Hospitality Association but have not found out who the pollster was.  Claimed results included support for the team ranging from 35% in Bass to 53% in Clark, stadium results of 36-55 Clark 39-55 Franklin 27-62 Braddon and 21-67 Bass, and a 40-12 result for the "Stadium 2.0" proposal vs the current Macquarie Point stadium (33-14 in Clark).   The team results seem curiously low and there is no information regarding question wording.

Umpire says: Sent off for not wearing a uniform.

This piece will be continually updated as more items come to hand, but in some cases specific items will be given their own articles as well.