Federal 2PP Aggregate Methods (45th Parliament)

This is the methods post for the 45th Parliament edition of the aggregate (sometimes nicknamed "Aggedor") that I post in the sidebar of this site, and which will form the basis for Poll Roundup posts and, later, my attempts to forecast the next election.

The current version is essentially the same as the version running at the end of the 44th parliament, with some minor changes to the weightings and the interpretation of Newspoll.  One substantial methods change was made half-way through the 44th parliament, which was to switch from just using the 2PP figure supplied by pollsters using previous-election preferences, to using a hybrid of the supplied figure and a figure calculated from their primaries.

A simpler version ran before the 2013 election and fluked getting the 2PP exactly right as a result of a preference shift cancelling out a possible late swing to the Coalition.  The 2013-6 version had a final error of 0.4 points, almost half of which resulted from slight shifts in preference flow patterns.

The aggregate is mostly a weighted average of two-party preferred polling derived from all recent polls of sufficient standard.  The 2PP figure assigned to each poll is multiplied by various weightings based on the poll's recency, accuracy and other issues, and the sum of the multiplied poll scores is divided by the sum of the weightings.

The aggregate is designed to be transparently checkable in theory and to use basic mathematics only.  However, it is not entirely codified in advance.  Decisions will be made on issues of pollster weighting and house effects, and possibly other matters, and will be updated to this page at the bottom when made.

Victorian Election: Final Aggregate And Seat Model

 Live analysis here on a new thread from 6 pm on election night. 

2PP Aggregate using 2010 preferences: 51.7 to Labor
Polls point to highly likely but not certain ALP victory, general picture of modest swing and minimal to low seat turnover
Seat Projection: ALP 47 Coalition 41
(A very small number of these seats may be won by Greens or independents, but there is not enough objective evidence to back such wins.)

Going into the main voting day for the Victorian election, I have to say it's a bit different to elections I've devoted major modelling efforts to on this site (the 2013 federal election and this year's Tasmanian state election) in that there is still some room for doubt about who will win.  Although Labor has led in almost every poll in the last year and a bit, including 15 or 16 (depends how you count 'em) of the 17 polls released in the last six weeks, their lead is not much, and sitting-member effects left over from the last election make the deck a slightly unfriendly one.

The polls suggest that probably Labor will get home with a seat tally in the high 40s, but other still-plausible scenarios (in decreasing order of likelihood) are a fairly comfortable Labor win with around 50 seats (just maybe if the polls have herded a few more), a scraped and very lucky Coalition win, or some kind of tied or otherwise hung parliament.  Anything else (eg a Coalition win where they win the 2PP as well, or a really lopsided Labor win) would be a serious surprise.

Victorian State Election: Late October Polls And Seat Model

2PP aggregate of recent Victorian state polling:  ALP leads 52.6-47.4

"Nowcast" seat estimate based on this 2PP: ALP 48 Coalition 40

After a week buried deep in the Hobart City Council count I've finally found time to get stuck into the Victorian state election.  This thread starts with a roundup of recent polling, then goes into the early version of a seat-probability model similar to my one that picked 93% of seats correctly at the 2013 federal election.  The federal election version was whipped up very quickly in the shadows of the post; I have much more time to work on this one so it will probably do much worse.

The new polls

Six polls have been released in the last week or so, by a range of methods.  Two of them have had suspiciously high Green votes.  For the Morgan SMS poll it also had a suspiciously high Green vote in late September, so I'll assume this is systematic.  The Ipsos poll is the first of its kind, and their federal poll published today hasn't shown any skew to the Greens, so I'm assuming for now that their methods are no more prone to green skew than the four more established pollsters.  Anyway in aggregating these six polls at the bottom I've pinged the Greens 0.9 of a point in every poll except Morgan SMS, for which I've applied a very lenient deduction of four.  For more information on this decision see my new blog header.

Wonk Central: What Do We Do With The Poll Rounding Problem?

This supplement to this week's Poll Roundup concerns recent results comparisons between my poll aggregate and BludgerTrack, and explains a small methods change I've brought in to my aggregate this week, and also how the hell Newspoll might have got a 51 to Labor 2PP off this week's primaries.  I was firmly expecting this to be the least read article on this site all year (but 24 hours after release it is beating the main roundup), and it's the only one I've ever published with a jump break included from the start. Click on the "Read more>>" below the warning sign (if you didn't arrive here via a direct link) to read on if you dare.

(Image lambied from a widespread internet meme of unknown (to me) origin, example here)

Too Much Information: The Mixed Performance of Seat Betting Markets At The Federal Election

Advance Summary

1. Seat betting markets, considered by some to be highly predictive, returned an indifferent final result at the 2013 federal election, overpredicting the number of Labor losses by at least seven and predicting fourteen seats incorrectly.

2. Better results were achieved not only by local/state projections based on polling data but also could have been achieved by a simple reading of the national polls.

3. Seat betting markets in the final week most likely misread the election because of an overload of contradictory data.  They placed too much emphasis on local-level polling and internal polling rumours and too little on national polling.

4. Prior to the final week of the campaign, however, seat betting markets performed well in projecting an uncertain situation that was difficult to model.

5. Seat betting markets were most accurate immediately following the return of Kevin Rudd.  However this probably reflects on the modelling skills of bookmakers rather than punters.

6. Modellers wanting to know what seat totals betting markets expect should look at direct seat total markets rather than attempting to derive that information via complex and uncertain processes from seat betting markets.

7. Final direct seat total markets were very accurate.

Federal 2PP estimate feature added

 (Note: This article documents the experimental phase of the aggregate, which ran from February to September 2013.  For transitional arrangements post the election see here.)

I've just added a subjective two-party-preferred vote (2PP) estimate feature on the sidebar.  The reason I have added this feature is that I commonly get involved in discussions on various sites in which someone is making claims about the likely state of the national 2PP that aren't even remotely credible, or just getting confused about all the different polls and the strange range of values they spit out.  I think a fair few people will from time to time be interested in my view of where things are at, especially when having those kinds of debates while I'm not around.  There are some handy formal aggregators about, but they often take a few days to update and I often find my view a little out from theirs (and usually somewhere in the middle of them all), typically because everyone has slightly different views on the best underlying assumptions.  When I can, I'll be aiming to update this estimate quickly.

The 2PP estimate is not a fully formalised aggregator and is not a scientific test. I'm not at the point of being ready to attempt something like that yet, and while I have a long-running Newspoll rolling-average of sorts for historical comparison purposes, I've only recently developed an interest in trying to gauge the picture across all federal pollsters.  It's just my hopefully informed opinion on the basis of an informal and at this stage loosely defined aggregation process.  The rough assumptions I make in thinking about the national 2PP are as follows.  (Note: Article has been edited to give the current version.  Legacy text appears at the bottom so people can see how this model has developed.)

Attitudes To Attributes (Not Great News For The Greens)

Advance summary

1. Recent polling on voter views of the importance of a list of issues, and which parties are best trusted on those issues, shows that many voters regard a range of parties as having policy strengths in different areas, rather than assuming their preferred party is always right.

2. Party trust scores, when weighted by the perceived importance of issues, produce surprisingly accurate predictions of party vote share over the last two and a half years.

3. The Greens' current mediocre polling position, especially compared to before the 2010 election, is probably connected with the party neither "owning" key issues as strongly as it used to, nor being able to convince voters that those issues are important. 

Thoughts on Forecasting and the US Pres Election

UPDATE 4 pm: Nothing has changed with only Virginia and Florida genuinely close and this is looking like it will be a triumph for state-polling-based models for the second election in a row.

UPDATE 2:40 pm: At present only Virginia and Florida counts are looking really close (as well as NC where Obama has done better than expected but still looks like falling just short).  Even if Romney wins all this he still falls short so there is no sign at present of a path for him to victory.

UPDATE 1:40 pm: Obama has in my view won Ohio based on similar methods to those used to model Australian counts in progress. 

--------------------------------------------------------------------------------------------------------------

Apparently, some 8% of this site's readership so far is US-based, ten times more than any other non-Australian country to this stage.  That inspires me to say a few quick things about the massive logistic exercise unfolding over there at the moment and the debates about what will very soon happen.

For those who are not familiar with the US system, making sense of the endless history-laden data-drenched arguments about how to predict who will be President can be a daunting task.  Hopefully the following comments will be useful in informing people about some of the pitfalls when it comes to what to take seriously. 

Beware Overfitted Models

No we're not talking catwalk stuff here, but these things are certainly overdressed and each particular one will go out of fashion very quickly.  When you see someone claiming to have found a model that predicts a certain candidate will win based on a shopping list of items that have supposedly "predicted" (despite being made after) every presidential election since the year dot, run for the hills.  The chief offender I noticed this time was the Uni of Colorado study that claimed that Romney would win, but there were plenty of equally shoddy examples calling it for Obama.