Putting madness into the method?

As you’re I’m sure all aware, the Irish Times/ MRBI published a poll this morning, which resulted in a lot of checking, at the end of which, has led to the discovery that the final figures are not a measure of the likely percentage votes, but of the likely percentages of “useful” votes.

Early look at the figures for me raised alarm bells as the LP ‘core figure’ was reported as having gone up 2% to 6% but final figure (after excluding 22% DKs) was unchanged at 7%. This was mathematically implausible, and I suspected either ‘at the edges’ rounding issues in both this & previous poll i.e. one rounded down, the other up, ( @Gavreilly pointed out the total came to 97%) or a transcription error in the Times.

Enquiries resulted in the revelation that there was indeed such a transcription error on the part of the Times, and I was informed that it was amended (current version on the web doesn’t appear to make mention of these values at all..) but that wasn’t all.  A footnote pointed out that the methodology had changed, and there was an additional question asked, as to the likelihood of the person’s vote being wasted (scale 1-10), and if they responded “1” (“makes no difference”), those people were omitted from the total figures, in the same way as people who say they won’t vote.

While I can see the intent behind this change, i.e. to measure shifts in those votes that make a difference (or at least that their owners think will), and those old enough to remember Ted Nealon’s Election analysis in the 70s will recall his concept of “Last Effective Vote” (admittedly measured ‘after the event’)– it does mean a ‘swing from last election’ approach is of little value, as this poll does not measure the First Preference Vote (FPV) but the % of FPVs that are considered, by their owners, may impact upon the result.  The swing is from one sort of measurement, to another.However, people aren’t comparing this poll with previous such measures, they are measuring it against the outcome last time, with a view to working out the sort of change #GE16 will make to the composition of the next Dail.

This causes a number of difficulties, aside from making it a different measure to other polls (which are attempting to measure FPV intention), including the last MRBI.

In the first instance, there is the issue of how one measures this accurately, and in a way that doesn’t instead skew for something entirely different.  Technically, the chance of any single vote impacting on the outcome of a single seat is so low as to be practically zero – there have been about a thousand constituency elections since the State was founded, and only a handful were decided by less than 5 votes, and, even in those cases 1 vote makes no difference. My vote will not change the outcome in my constituency.  Neither will yours. In theory, all votes should be excluded by this test as none are remotely likely to change outcome. However, they won’t, as most people will cling to the hope that their individual march to the polling station isn’t a waste of time.

Secondly, the more realistic/intelligent/engaged/experienced (or pessimistic) you are, the more likely you are to recognise this, meaning that cohort are more likely to drop out of the survey (i.e. them, rather than people whose votes won’t count in practice).  By contrast, the gullible/innumerate/inexperienced (and optimistic) are likely to be over-represented.  Of course, nearly everyone will think that hits their party most, however most of them will be wrong…I have my own view as to who this would benefit and who it would hit – you no doubt have yours…

But most crucially, the wasted votes are effectively ‘priced in’ where there is a projection to each constituency.  When I (or Adrian Kavanagh or Michael Marsh, or you) run a spreadsheet with whatever formulas, it produces plenty of constituencies where large swathes of votes – certainly their first preferences, which is what this poll is testing – do not count.  Take the last RedC, where Labour had 10%, there were 23/40 constituencies were the Labour votes did not elect their candidate (in my projection).  In many cases they may have elected someone else on transfers, but even here, in most cases, small beer.

In other words, of the 10% Labour got in the last RedC, as little as 6% of that may have contributed to someone’s election, and may therefore technically “makes no difference”.  Additionally, even those who were elected, many would see their vote as making no difference (Lowry voters, for instance, should know he is as safe as houses – their individual votes “make no difference”).

Of course, this will impact upon everyone, but least of all on FG as they are in with a shout everywhere, often could be 1 or 2 seats depending on how it goes, and are likely to get elected in at last 36-37/40 constituencies, perhaps more.  FF & SF will have some slippage (FF in most of Dublin, SF in parts of Dublin) but won’t lose much, LP lose a lot as are a national party but can expect 3-4% ‘wasted’ (or more) in a lot of places, GP similar, but AAA/SocDems/Renua all are likely to have quite concentrated votes in their areas (and no candidates in a lot of the areas they are weaker).  In the rebalancing, this would be expected to inflate FG, AAA, SDs & Renua, neutral for SF/FF, and hit LP & GP. (That is aside from the gullibility/inexperienced bias mentioned above).

The annoying thing about this meddling is that there was no reason for the change.  None at all. MRBI has been the most accurate in both the last 2 General Elections, and this change actively moves their final figure away from what their survey is telling them will be the result, and in a manner that is untested, to a figure that is of unpredictable usefulness, and is not a measure of FPVs (which is what the Irish Times are selling it as being in their reporting).  Of course, there are other adjustments that are made in polls, e.g. for likelihood to vote, however this is measuring for something that has a real impact on the FPV, in the control of the voter.  There is no evidence that I’m aware of that being a ‘wasted’ vote makes a vote less likely to be cast, and if it does it should be picked up on the likelihood to vote question anyway (edit: I’ve since read that there is no such question about likelihood to vote asked in MRBI polls, which makes their past accuracy – better than those who do – even more remarkable, and calls into question how effective that test is on improving accuracy).

I have asked MRBI via Twitter if they have figures on what this poll would have produced without that adjustment, and if I get it I will create a 2nd spreadsheet for MRBI polls that adjusts for that difference.  Without it, I can’t, and anyone saying they can  doesn’t understand what this data is.

One can only hope that someone who understands what this change does picks up on this, and it disappears before their next poll.

So, no projection at this stage, as the figures don’t measure likely FPVs, and so there are no FPV to project from.  Sorry about that, but gotta be honest.  It wouldn’t be accurate, and I’ve no reason to publish something that isn’t.

D