Friday, September 23, 2016

Federal 2PP Aggregate Methods (45th Parliament)

This is the methods post for the 45th Parliament edition of the aggregate (sometimes nicknamed "Aggedor") that I post in the sidebar of this site, and which will form the basis for Poll Roundup posts and, later, my attempts to forecast the next election.

The current version is essentially the same as the version running at the end of the 44th parliament, with some minor changes to the weightings.  One substantial methods change was made half-way through the 44th parliament, which was to switch from just using the 2PP figure supplied by most pollsters, to using a hybrid of the supplied figure and a figure calculated from their primaries.

A simpler version ran before the 2013 election and fluked getting the 2PP exactly right as a result of a preference shift cancelling out a possible late swing to the Coalition.  The 2013-6 version had a final error of 0.4 points, almost half of which resulted from slight shifts in preference flow patterns.

The aggregate is mostly a weighted average of two-party preferred polling derived from all recent polls of sufficient standard.  The 2PP figure assigned to each poll is multiplied by various weightings based on the poll's recency, accuracy and other issues, and the sum of the multiplied poll scores is divided by the sum of the weightings.

The aggregate is designed to be transparently checkable in theory and to use basic mathematics only.  However, it is not entirely codified in advance.  Decisions will be made on issues of pollster weighting and house effects, and possibly other matters, and will be updated to this page at the bottom when made.



Pollsters Included:

Polls by the following pollsters are included when available: Galaxy, Newspoll, ReachTEL, Ipsos, Essential, Morgan.  Other pollsters may be added subject to their completion of three polls of national 2PP voting intention by the same methods, and adequate methods documentation, and previously active polls may be reincluded.  Even if a poll is by an included pollster, the poll is not included if it is an "internal poll" (defined as a poll commissioned by a political party, union, business or lobby group).  Polls that are known to include aggregations of other polls are rejected.

2PP calculations:

For every poll, a 2PP is calculated from the primaries, using last-election preferences.  For instance,

Coalition 2PP = Coalition Primary + .181*Green Primary + .508*Others Primary - 0.13

or

Coalition 2PP = Coalition Primary + .181*Green Primary + .398*NXT Primary + .505*One Nation Primary + .529* Others Primary - 0.13

(The formula used depends on whether the poll publishes breakdowns for NXT and One Nation or not.  The -0.13 is an adjustment for the impact of three-cornered Liberal/National contests).

If the poll does not release a last-election 2PP then the above figure alone is used, rounded to one decimal place.  This also applies when the pollster was using figures from the election before last (eg polls using 2013 preferences in July-August 2016), and I reserve the right to apply it if a pollster's 2PP calculations are persistently suspicious (in which case I will log it on this page).

If the poll releases a last-election 2PP but the primaries have a greater level of detail, then the above figure alone (rounded to one decimal place) is used, provided that it rounds to the same 2PP.  If it doesn't then the closest figure that so rounds is used.

For other polls the 2PP that is used is the average of:

1. The last-election 2PP released by the pollster
2. The calculated 2PP based on the released primaries (formula above).

This is rounded to one decimal place, with the constraint that the maximum allowed difference between the number thus found and the pollster's released 2PP (prior to applying any house effect correction) is 0.2 for Morgan, and 0.4 for all remaining pollsters.  So if a Newspoll released 2PP is 51, the aggregated 2PP will be in the range 50.6-51.4.

Ipsos uses "batched" last-election 2PPs which assume preferences from all minor parties will flow the same way overall, even if the ratio of Greens to other minor parties changes.  For these I use a weighting of 40% for the last-election 2PP and 60% for the calculated 2PP.  The maximum allowed difference constraint does not apply for Ipsos.

Time weighting:

The aggregate is designed to be fairly cautious in adjusting to apparent changes, in case these are actually caused by sample noise. The base time weighting is 5 for a poll released this week, 3 for last week, 2 for the week before last, 1 for the week before that.   At present a new polling week is considered to start from Saturday with the reset being applied from midnight Friday.  Intermediate weightings of 4 for a poll bridging this and last week, 2.5 for a poll bridging one and two weeks ago, 1.5 for 2-3 weeks ago, 0.5 for 3-4 weeks ago, are used in the case of polls that use rolling data for two weeks (a la Essential) or that derive their data entirely from the middle or end of the previous week (eg ReachTEL often polls on Thursday.)

So for instance in the typical week the new Essential carries a time weighting of 4 when it is released, but this is reset to 2.5 when the next polling week starts.  A ReachTEL released on Thursday or Friday based on polling on Wednesday or Thursday is weighted at 5 but this is reset to 4 when the next polling week starts.

During election campaigns, in the last three weeks a weighting of 8 is applied for polls based entirely on data sampled (not released) in the last four days.  In the last week a weighting of 10 is applied for polls based entirely on data sampled (not released) in the last two days.  Only the most recent poll by any pollster can qualify for these weightings.

Number of polls included:

Only the two most recent polls by any pollster are included.  In the case of Essential, the most recent non-overlapping polls are counted.  This normally means the polls included are the most recent and the one from two weeks before.

Quality weighting:

Pollsters are weighted based on my view of their performance - not just in final polls but through campaign cycles and between campaigns.  I reward polls for consistently using the same methods with good results at multiple elections, and for being the best pollster at the most recent election.  Polls can be marked down for irregular tracking behaviour, for consistently having the vote of a particular party too high, for changing methods or not conducting polls very close to an election, and so on.  Based on experience in the previous cycle, I have increased the spread of ratings for this one.

The maximum weighting for a poll that has appeared seven or less times is 0.8.

The current quality weightings are:

1.35 Galaxy, Newspoll
1.2 ReachTEL
0.9 Ipsos
0.75 Essential
0.4 Morgan

If only the 2PP is released for a poll and not the primaries then the quality weighting is halved.

For my full review of pollster performance at the 2016 election see here and for the 2013 election see here.

Respondent vs Last-Election Preferences:

Most recent elections have shown that last-election preferences are the most reliable method while respondent preferences skew to Labor and create needlessly volatile poll readings.  The 2016 election was no exception to this . Therefore this aggregate ignores respondent preferences.

House Effect Correction:

Poll types by a specific pollster are applied a house effect correction when it is considered that they are displaying a house effect in favour of a particular party.  This can either be a systematic long-term feature of that poll series or can reflect a pollster developing a sustained medium-term difference from the tracking shown by other polls.   The minimum house effect correction is 1 point, so if a pollster is suspected of having a house effect smaller than that, then that is treated under Global House Effect (see below).

The current house effects applied are:

Morgan multi-mode (SMS / face to face): +1 added to Coalition

Global House Effect Correction:

A global house effect correction of up to a point is applied to the aggregate if it is considered that the pollsters included in it (after the adjustments above) tend on average to lean to one side or the other.  This is partly determined by considering the performance of polls at a range of election campaigns, with federal elections considered most important but state elections also taken into account.  However short-term changes in house effect in specific polls are also taken into account. The frequency of the polls included is also taken into account - if polls that lean slightly to one party are more common or more heavily weighted than those leaning slightly to the other, then this is corrected for.

The current global house effect correction is +0.2 to Coalition.

Sample size:

No scaled adjustment is made for sample size at present (since most reliable national polls use broadly similar sample sizes which they increase at election time), except that a poll with a sample size of less than 950 automatically has its value in the aggregate halved.

Unusual polls:

From time to time a poll is available that includes data from around half the Australian voter base.  Examples are a Galaxy poll of only female voters, and a ReachTEL poll of only NSW and Victorian voters. In these cases a national figure is estimated based on known differences between the surveyed and non-surveyed components, and the poll is aggregated at half its normal value.  However such polls by pollsters without well-established national 2PP tracking (eg JWS Research bulk polls of dozens of "marginal seats") are not included.  Single-state polls of federal voting intention are also not included.

New poll types:

A new poll type by an established pollster may be either included immediately, or excluded until three polls by that type have been conducted.  The new poll type is always excluded until three polls by that type are conducted if it appears to have a substantial house effect.

Example tracking:

The following is example of tracking for the current parliament four weeks after the condition for using the aggregate was first met:


Labor has led throughout this period (and by 2016 preferences on every poll released since the election), although by my assessment this lead is relatively narrow.  The aggregate more heavily weights Newspoll than Essential and assumes both usually lean slightly to Labor based on form in the previous parliament, although neither did so in their final poll.

Here is the final output of the smoothed aggregate for the previous parliament:



Seat Projections and Interpretation:

Formal seat projections will be added in the leadup to the next election.

For the time being, while the Mackerras pendulum has Labor requiring a 50.7% 2PP based on uniform swing to win more seats than the Coalition at the next election, it really shouldn't be that hard.  Labor has a massive advantage in personal votes arising from its many seat wins at the 2016 election.  This may well be blunted by redistributions (or if any of the new MPs don't recontest), but for now I estimate Labor needs a 2PP of only 50.1% for a 50% chance of winning more seats than the Coalition (ignoring any possible crossbench changes).  It is similar to the situation at the 2001 election, at which a 1.93% swing in its favour gave the Coalition not the eight seat gains implied by the pendulum, but only two.

Smoothing

A smoothed version of the aggregate is used as a basis for graphs.  The smoothed version for a given week is calculated by the following formula:

(current reading*2 + previous reading + next reading)/4

or where the next reading has not happened yet:

(current reading*2.5 + previous reading*1.5)/4

Other Aggregators:

Several aggregators operated during the 2016 election cycle, some of which have since been discontinued.  Links to good-quality aggregators, doubtless including BludgerTrack, will be added here as they emerge from hiding.  Dodgy ones may be linked as well if they are high-profile.

Conditions for operation:

The aggregate is suspended (frozen at its existing value) at any time at which the number of individual polls included in it falls below three.  When the number falls below five a cautionary note is added.

Mandatory reset events (all existing polls discarded and the aggregate restarts after three new polls) are (i) a federal election for the full House of Representatives is held (ii) there is a change of Prime Minister.  Resets may be applied in other cases but this will only occur, if at all, with great reluctance!

Log of adjustment and methods changes:

All changes made to the pollster-specific and global house effects, and any other changes made, will be listed here.

27 Nov: Global house effect correction reduced from +0.4 to +0.2 for Coalition following re-emergence of Ipsos.

No comments:

Post a Comment