Westminster projection - 27.07.2018 update

The big picture

There have been six UK-wide GE polls over the last two weeks and all have Labour leading from 1% to 5%. This is far from the big Labour surge of 2017 but still enough to make a Conservative victory the least likely outcome if the surprise snap GE of 2018 finally happens. For some reason Labour have done their best so far to prevent it so maybe some more favourable polls will make them change their minds.

Anyway here's what the weighted average of the last six polls says, with Labour now leading by 3%. Super-sample size is 10,620 with a 0.92% MOE.

And again this is not the result of Labour doing especially well as their vote share would be roughly the same as in 2017. It's about Conservatives doing awful and losing votes to both UKIP and Liberal Democrats. Fun part of course being that Brexit brought on UKIP's demise and is now the root of UKIP's rebirth. Guess the Brexit bùrach has taken its toll and will continue to do so. Current polling has about 1.4mil voters switching from the Conservatives to UKIP and 300k switching from the Conservatives to the Liberal Democrats, more than enough to offer Labour significant gains in marginal seats on a silver platter.

And here's the projected House of Commons on these numbers:

Quite an interesting situation with Labour as the first party but 25 seats short of a majority. Best they could probably do would be risking a minority coalition with the Liberal Democrats. Which would work only if Labour manage to reach some sort of agreement with Greens, Plaid Cymru and the SNP. Not a done deal but anything between full confidence and supply or just neutrality would have to be considered, especially with the Scottish Branch Office of the Labour Party down to two seats and no longer having much leverage.

Seats changing hands

Under current polling 50 seats would change hands including 40 from the Conservatives to Labour. Here are the summary of gains and losses and the complete list which includes some interesting potential Balls moments. This would be about the same number of changes as is 1966, 1987 and 1992. But none of these brought on a change of government while the snap election of 2018 might very well achieve just that.

Apart from the seven SNP gains the most significant projected change, and probably the one that would be most commented upon, would be Iain Duncan Smith losing Chingford and Woodford Green which he has held continuously since the seat's creation in 1997. Or even 1992 if you include its predecessor seat Chingford which had been Norman Tebbit's seat from 1974 to 1992.

But this is North East London where Labour usually do well. In 2017 Chingford and Woodford Green (outlined in yellow) was the odd blue spot in a red area. Also remember that IDS' margin shrunk from a peak 30% in 2010 to just above 5% in 2017. Enough to make it a credible Labour target under currently projected swing. And an upset that wouldn't be that much of an upset after all.

Alternate scenarios

On current polling fewer seats would find themselves in the 'danger zone' (winner's margin below MOE) than was the case in previous projections. About 20 against 40+ previously. Main reason is that a 3% Labour lead would switch the bulk of marginals and beyond to them and leave relatively few seats potentially switching either way on a smallish change in voting intentions.

And now the summary of gains and losses under the two 'extreme' scenarios and the possible coalitions (or lack thereof) the UK might end up with. Even the 'Labour Maximum' scenario would see 60 seats at most changing hands. So you can expect anything but a landslide and probably some interesting days of negotiation after the election.

Note that even in their best case scenario Labour would require a solid coalition deal with the Liberal Democrats if anything like that is even remotely possible. This is worth mentioning as, if this scenario becomes more likely, it would certainly revive the old 'vote SNP get Tory' mantra. I'm not implying it would work or even be statistically feasible but it's quite obvious the Scottish Branch Office of the Labour Party would use it.

Under the current 'Labour Maximum' scenario Labour would bag 303 seats in England and Wales but only 3 in Scotland. Which means they would need an additional 16 Scottish seats to get a one-seat majority and forget the LibDem deal. Achieving this would require a 3% swing from the 2017 result or a 6% swing from current Scottish polling. The SNP surge we have seen recently makes this highly unlikely but that certainly would not deter Labour from campaigning on it. We've been warned.

What about the 'Preferred Prime Minister' polling?

Not all pollsters ask this question and when they do they phrase it differently. But here's the trend anyway based on my usual six-point rolling average.

Theresa May has gone down five points in as many weeks but is still leading by ten points. This result still amazes me even if there are many reasons for it: the never-ending story of anti-semitism within the Labour Party, endless noisy infighting, ambiguity about Brexit, Corbyn's perceived lack of proper leadership, you name it and you get it. Pretty much the same arguments could be used against Theresa May but oddly the Conservatives manage to get away with it. So far.

What to expect next?

Public opinion is as volatile as ever and prone to overreact even to minor events. So nothing can be ruled out. For the foreseeable future my best educated guess is that polls could deliver anything between a 3% Conservative lead and a 5% Labour lead. So as much uncertainty as ever about the possible outcome.

Guess only the Liberal Democrats, UKIP and the SNP will find reasons to be happy with the results of the next batch of polls.

Stay tuned for further updates

Saor Alba gu bràth

Holyrood projection - July 2018 update

A wee flashback as an appetizer

About two weeks ago Survation released a comprehensive poll of Scottish voting intentions. I have already covered the parts about the General Election and the Independence referendum. Now let's have a look at the part about Scottish Parliament. Constituency results are pretty good for the SNP who are at their highest since September 2017. Regional list results on the other hand are far less satisfactory with a higher than usual proportion of voters switching to Scottish Greens. I have a hunch that the (wrong but widely held) belief that the list vote is a second preference is the most likely explanation here.

Survation publish their findings with one decimal place which is kind of irrelevant when the sample size is about 1,000 so the margin of error is about 3%. But it's their choice and I respect it. Another interesting part is that the Conservatives would fall to third place on the regional list vote 'thanks' to an unusually high UKIP vote.

Another interesting aspect of this poll is that it is a textbook case of how different approaches to modelling seat projections will lead to slightly different results. I compared the seat projections using ScotlandVotes (to my knowledge the only website that features Holyrood seat projections from polling data) to those using my own model. First here is what ScotlandVotes delivers.

So we have the SNP down four seats but pro-Indy majority bags 69 seats as in 2016 thanks to the Scottish Greens' surge. Labour are back to second party and UKIP get no seats. And now here is what my own model projects.

The SNP are still down four seats but pro-Indy majority gains one seat. Labour is still back to second place but here UKIP manage to gain three list seats (Highlands and Islands, Mid Scotland and Fife, South Scotland). This is the direct result of how ScotlandVotes' model and mine handle the list seat projection differently.

A wee smitch of PsephSpeak you're allowed to skip but might want to read anyway

ScotlandVotes rely on straight Uniform National Swing (UNS). Think of UNS as a 'flat rate' swing. Votes in every electoral division (constituency or region) move in the same direction by the exact same amount. It is the most ancient prediction method and the most commonly used by psephologists. It is reasonably accurate when applied to a large number of seats (like House of Commons projections) as outliers tends to cancel each other out.

In my Holyrood model I chose not to apply straight UNS to the list vote across all regions but instead to calculate individual sets of voting intentions for each region. I use straight UNS when the pollster does not provide a crosstab of voting intentions by region, which is the case with YouGov or Panelbase. When the pollster provides such a crosstab, as Survation do, I use their data instead.

In this specific case, UNS based on a 4.7% national average doesn't take UKIP in any region over the implicit »6.5% threshold built into AMS to bag a list seat. Though they would come pretty close in Highland and Islands. The Survation crosstab has UKIP over the threshold in three regions, and so you get UKIP sitting in Scottish Parliament for the first time ever. See Table 19 on page 22 of the Survation report.

For the record I don't use straight UNS for my First Past The Post (FPTP) seat projections either. I use a mix of UNS and Proportional National Swing (PNS). Both are explained in layman's terms on this Canadian site. Currently my FPTP model is tuned to 70% UNS + 30% PNS which seems a reasonable compromise to approach what would likely happen in an actual election.

Anyway minor methodological differences shouldn't make us overlook the true political significance of this poll. Whatever the approach, an actual election delivering the same results would be a major success for the Yes movement. Pro-Independence parties would hold their majority and possibly even strengthen it. The Scottish Government would have a renewed mandate for a second independence referendum (if it hasn't already happened and been won). And the SNP would get their fourth consecutive term, a major achievement even if they would have to content themselves with being a minority government again.

And now where does the polls' trend take us?

Of course the average of recent polls tells a different story than the Survation poll I discussed at length to give you the good news first. My current super-sample (four constituency polls and three regional list polls fielded in June and July) is less favourable for the pro-Independence parties. Super-sample size is 4,116 (1.48% MOE) on the constituencies and 3,098 (1.71% MOE) on the lists. Here is the current weighted average of voting intentions, with the detail of list votes by region:

Seat projections then. Constituencies by SLLM rating, list seats by region, full Scottish Parliament. Despite the good results from the Survation poll, rolling average still delivers pro-Independence parties two seats short of a majority and Conservatives remain the second party.

For comparison I fed ScotlandVotes the same data and their model delivers very similar results. Despite the methodological differences noted above they too have pro-Independence parties in a minority on these numbers. There are also some minor differences in the overall distribution of seats but well within the super-sample's MOE.

Any alternate scenarios?

Right now six seats are projected to change hands. All from the SNP to the Conservatives. But it's not by any means a Conservative landslide as their margins is these seats would range from 0.8% to 6.5%.

We also have six marginal seats which, under the proper circumstances, could go either way. Five of the projected Conservative gains qualify as marginals but only one seat currently held by the SNP.

As I explained in my first post about Holyrood projections, it's difficult to build a simple Min/Max scenario here because of the odd way AMS works. Remember a change in constituency seats can result in a change in the other direction in the allocation of list seats, with the overall result remaining the same. This is the case again here with Angus South. If the SNP lost this seat it would automatically be 'compensated' by a list seat for the SNP in North East Scotland where they are currently projected to win none.

The other five marginals tell a different story as all are within regions where the SNP is currently projected to win no list seats. So if the SNP prevails in these seats it would have no impact on their number of list seats. The only consequence of the changes would be to reshuffle list seats among the other parties. This 'SNP max' scenario would deliver this result:

Even with Greens losing one list seat to the Conservatives in Lothian as a direct 'compensation' for the SNP holding Edinburgh Pentlands, pro-Independence parties would hold a two seat majority. Two seats down from the 2016 results but still enough for a renewed mandate for independence, if that hasn't happened already by 2021.

Seat breakdown by region:

What to expect next?

Recent polling for both Scottish Westminster seats and Scottish Parliament definitely show a surge in SNP voting intentions. Best educated guess is that next polls will show the same trend and confirm that a renewed pro-Independence majority in Holyrood is not only possible but the most likely outcome. Of course upsets can't be ruled out but I honestly don't expect any major shift away from the SNP, quite the opposite actually.

Anyway stay tuned for further updates.

Saor Alba gu bràth

The truth about #IndyRef2

What got me started

Every day we hear Unionists compulsively ranting about how the SNP should 'stop obsessing about a second referendum' because there is 'no appetite for independence'. They have been mantraing this for years and especially more loudly since last year's General Election campaign. I remember vividly the first Scottish leaders' debate. Ruth Davidson mentioned independence 15 seconds into her opening statement. Nicola Sturgeon did not mention it until 26 minutes into the debate proper. Now tell me who's obsessing.

We also have a bunch of Unionists on Twitter traying to make their point with fancy graphs. Sorry but you're going to have some PsephSpeak here (that's PsephSpeak for statisticians' jargon). They pretend to accurately represent the trend of voting intentions using a linear regression. Only problem is that their graphs are misleading and dishonest and basically don't mean fuck all.

Even the greenest newbie statistician could tell you the purpose of linear regression is to determine whether or not correlations exist between two variables. This has nothing to do with the analysis of polls and trends. Using it in that context is one of two things: gross incompetence or deliberate attempt to mislead. Either way it's cooshite.

The correct way to represent trends when you deal with polling data is to use the n-point rolling average which will give you a more accurate trend on the long term or the short term, depending on the value you assign to n. Personally I settled on six which is sort of a compromise value when you have smallish datasets with unpredictable variations between datapoints.  

What the true trend is

Here is what what you get from the 45 referendum polls we have had since September 2014, when you use the proper method and actually seek to display the real trends.

Of course there have been ups and downs in Yes voting intentions over four years. Nobody denies it and nobody ever pretended it wouldn't be a bumpy road. This graph shows what matters: how Yes and No voting intentions really evolved over the last 22 months, not a fantasy version of it. 

Now let's concentrate on the most recent period like the past 15 months. Because obviously voting intentions from back in 2015 are totally irrelevant today. Arguably those from 2017 are also irrelevant to assess the current situation but there are some interesting points to be made from a closer look at the 2017 trends. Including identifying mistakes made then, that the Yes movement should want to avoid repeating in the future. 

First thing you see in the graph is that Yes voting intentions took a nosedive between April and June 2017. Which was (not coincidentally) the timeframe of the General Election campaign. Yes was on 47% average the week the GE was called and plummeted to 42% in early June. There might have been many reasons for that but I think the main factor was the sloppy SNP campaign. Remember when the SNP let themselves be cornered into debating devolved matters and independence which were anything but germane to a GE campaign. 

Lesson one: don't let the opposition dictate which issues are relevant or not. Make your own strong points the center of gravity of the campaign and ignore distractions or else you're gonna pay dearly. The SNP should bear that in mind if this year's surprise snap GE finally actually happens despite Labour doing their best to prevent it. 

Then Yes voting intentions went up again from 43% (Panelbase, 07.09.2017) to 47% (Survation, 10.07.2018). That's when grassroots seriously and strongly acted on their impatience with the SNP's 'cautious' approach, which many (myself included) felt had turned into procrastination, and decisively took the matter into their own hands. The huge success of the 'All Under One Banner' marches is obviously a major factor in the Yes 'revival'. While the SNP were debating the fine points of the 'why?' and the 'when?' at the Aberdeen Spring Conference, AUOB spread the word about the 'why?' all over Scotland while putting pressure on the SNP about the 'when?'. And it worked.

Lesson Two: never ignore the grassroots, embrace and trust them. They know what they're doing and they're good at it. In the last few months they got the message through better and faster than traditional parties. They have earned their position as first tier players in the next referendum campaign.

Where do we stand now?

Let's focus now on just the 2018 polling and you will see it paints quite a different picture than Unionists would like us to believe. We have had eight #IndyRef2 polls so far this year and here are the results (with and without undecideds):

This is a half empty-half full situation. Unionists will of course say that we are still far far away from a Yes majority. But numbers actually say it's getting better for the Yes camp. Current rolling average of voting intentions also shows it. I used only the four most recent polls to get as close a snapshot as possible of voters' current state of mind. Here it is, with and without undecideds and compared to the 2014 result.

So despite all the negative campaigning by the opposition parties and all the help they get from English and Scottish MSM, Yes voting intentions are up 1% from Referendum Day. Which is a good start in a possibly uphill battle. Just remember that on the week of the Edinburgh Agreement polling was roughly 65-35 No. Two years of campaigning delivered a 10% swing to Yes, about 360k votes. 

Today the Yes camp need a 5% swing 'only'. And it can be done. Day after day the English government themselves make the case for Scotland to get away from all the already self-inflicted damage and impending disaster.

So what's next?

First of all more #SNPBad stories all over the place. It has already started with the military pay fake news. The story was debunked by The Ferret and The National four months ago already yet this week it comes back, not coincidentally after a series of positive polls for the SNP. Similar situations will happen again as we all know the MSM guideline these days is: don't let facts get in the way of the narrative Tory Central Command just sent us.

Now we also have this hilarious attack on Visit Scotland's latest campaign. Which is in fact a good thing as it shows yet another Tory MSP making a total arse of herself. And will help convince anyone who still had doubts what a sorry bunch of glaikit fuds they all are.

Nevertheless a rerun of Better Together and The Vow is highly unlikely. All of Scotland would laugh their asses out if we  hear the likes of 'Scotland stronger in the Union'; 'a Union of equals', 'Scottish Parliament will be made permanent' or 'extensive new powers will be devolved' ever again. So never again anything like this:

Then of course we also had 'Better Together because Scotland will enjoy continued membership of the EU'. Too bad this one has already been nicely taken care of by Unionists themselves. So when the SNP write down their 'Why?' talking points, it has to carry a strong 'Post-Brexit Why?' message. Obviously Keith Brown is already working on just that.

Interestingly the English Government themselves have already provided some data that might help Keith hone the pivotal argument for an Indy Scotland. Their own figures demonstrate how devastatingly the now-certain Hard Brexit would wreak havoc with the economy and employment. On these figures 2028 would see the UK's economy back to 1998 levels.

The impact on Scotland can be approximately assessed by transposing the UK-wide estimates on population share. The result is even more frightening as it points to Scotland's economy being wrecked beyond recovery within the next ten years. Based on the English Government's own estimates.

I guess we already know how to avoid this. Now let's have a look at one of many ways we can achieve it.

What about Yes-to-No switchers?

Back in 2014 Lord Ashcroft fielded a thorough post-referendum poll. The final document has 52 pages of tables dissecting the referendum vote on many different criteria: gender, age, social grade, political affiliation,.... The one I will focus on is the referendum vote / political affiliation crosstab. The Ashcroft poll has the referendum vote distributed by 2010 General Election vote. Current polls have referendum voting intentions / 2017 GE vote crosstabs. So here's the comparison between Yes vote then and now for the four major parties (minor parties are excluded from the crosstabs both by Ashcroft and current polls):

So now we know SNP voters are not 100% Yes though it's closer to that today than four years ago. And Conservative voters are not 100% No so Scot Yes Tories may have gotten their message through after all. But what is more interesting (and can be acted on) is the massive slump in both Lab-Yes and LibDem-Yes votes.

I have tried a simulation based on current Scottish voting intentions and the average Ref/GE crosstab from the three most recent referendum polls. If I arbitrarily input 90% Yes for Greens (same as SNP), 50% for Others (neutral) and 0% for UKIP (plausible) then the aggregate is just the 45.9% current polling average for Yes.

This without tweaking any of the data. Which would be daft anyway as all can be easily checked from the detailed tables available on the pollsters' sites. More interesting is the final step. Now if I tweak Lab-Yes and Lib-Yes votes back to the levels Ashcroft found in 2014, then Yes wins by a nose and we get Indy Scotland.

Achieving this would certainly require some creative campaigning from the Yes camp and especially the SNP but it's not like it would be squaring the circle. Narrowing the target to Remain/Yes-to-No voters is possibly the most sensible option. You already know I have my doubts about rejoining EU being the best option for Indy Scotland. But recent polls show that Scotland would vote Remain even more massively than in 2016 if there was a rematch of the EU referendum.

Right now I would go for a more cautious option for once. Just mention the Single Market without any direct reference to the EU. Since being part of the Single Market can be achieved through EEA membership too, that would leave more options open and make everybody happy. Remainers would subconsciously translate that into 'EU' while Leavers would translate it into 'EEA option open' making it a win-win situation for the Yes camp. Just a (not so) random thought though.

And what next?

Probably voting intentions flatlining on about 47-53 for the foreseeable future. Then by the end of March 2019 Scottish Government formally demands that House of Commons vote to trigger Section 30 again. ScotGov would have the upper hand with the SNP relying on the 2017 mandate and a commanding majority of Scottish Westminster seats (probably even bigger by then than it is now if the long-awaited snap GE happens in the meantime).

My hunch is that right now we have even odds that the English Government would cave in and accept an 'Edinburgh II' agreement. These odds can only get better with time as Scottish public opinion becomes more and more aware of the catastrophic consequences of the Brexit bùrach and support for holding a second Independence referendum rises above 50%.

Just my opinion anyway as always.

Stay tuned for further updates and provocative thoughts (and possible upsets too).

Saor Alba gu bràth

#IndyRef2  #ScotRef

Scottish Westminster update - 15.07.2018

A bit of history first

Since last GE thirteen Scottish Westminster polls have been published over thirteen months. Which does not mean 'a poll a month' as we have had four in quick succession over the last six weeks. 'Somebody somewhere' definitely wants to feel Scotland's pulse in almost real time since the SNP walkout from the House of Commons and the power-grab controversy.

Not all of these polls have been good news for the SNP. Some were even catastrophic. But the overall trend is getting better recently. Culminating with a Survation poll this week that projects into an unprecedented (since last GE) 49 seats for the SNP. Here is the full history of Scottish Westminster polling over the past months (voting intentions first and then seat projections). Rolling average is based on the last four polls published in June and July.

But all this doesn't tell us what the likely result would be if a surprise snap GE were to happen next week. So let's dig a little deeper.

What do the polls say?

Weighted average of the last four polls says this:

The SNP are 2.6% up from their 2017 result while Labour are 2.6% down and Conservatives 2.3% down. The Liberal Democrats also do relatively well as they're roughly level with their 2017 vote. Usual caveat applies to Greens' and UKIP's voting intentions, both being obviously overestimated. But it actually doesn't matter as my projection model has an inbuilt 'auto-correct' that factors in Greens and UKIP actually standing in only a small number of seats.

Trend also shows the SNP with a strong lead while both Labour and Conservatives have gone down in recent polling. But of course the SNP should not take anything for granted as any future GE campaign will obviously be hard-fought. Also Liberal Democrats could prove tougher competitors than expected as they manage to hold their 2017 votes and could even do a wee smitch better.

Where does that take us?

Here's the projected breakdown of seats by SLLM rating under current polling. Summary is: SNP 42, Conservatives 11, Liberal Democrats 4, Labour 2.

As expected Labour seats prove easier for the SNP to gain back as most of them were won with tiny margins at last GE. Conservative and above all LibDem seats are tougher as most of them saw a double-digit swing away from the SNP last year. The SNP would need to score well above 40% of the vote to have a good shot at most of these seats. Mind you it can be done though probably not in time for this year's surprise snap GE.

Right now seven seats are projected to switch back to the SNP, five from Labour and two from the Conservatives. And in most cases the SNP would have bigger margins than Labour or Conservatives had last year when they gained these same seats.

Alternate scenarios

Seven seats qualify as 'ultra-marginals' and are in the 'danger zone', meaning the winner's margin is within MOE. Remarkably only two of the SNP's projected gains and one current SNP seat fall into that category. This means current polling credits the SNP with a solid electoral base. Of course the surprise snap GE is still at least two months away which is like an eternity in politics. Public opinion could possibly change their collective mind but I honestly don't see that happening anytime soon, at least not on a massive scale.

Even on their 'worst case scenario' the SNP would still manage a projected net gain of four seats (five gains and one loss). Which is no mean feat in the current climate when everything the SNP do or don't do is fair game for a flurry of #SNPBad tweets and brass-necked criticism from the opposition.

I certainly don't want to sound over-optimistic but the Scottish Branch Office of the Labour Party still have a long way to go before they can mount a credible challenge to the SNP's dominant position. They were 10% behind at the 2017 GE and are now projected 15% behind on average and even a massive 19% in the last Survation poll. 

This also applies to the Scottish Conservative and Unionist Party who were 'only' 8% behind last year but are now projected 13% behind on average and 18% in the Survation poll.

Right now only the Liberal Democrats can be expected to hold all their seats and maybe even gain one from the SNP (North East Fife). A remote possibility but yet it can't be totally ruled out. Just being cautious here.

Seats breakdown

I know lots of my readers are eager to know which way their own constituency would go. So here's the full list, as usual from worst case to best case scenario for the SNP.

A few steps ahead

Seeing how SNP voting intentions have risen recently I tried a 'what if?' simulation. The point of reference is current polling average with the SNP on 39.5%. I then projected what would happen with SNP vote varying from 38% to 50%. Guess even the SNP can't get much higher than that.

Note that David Mundell would not be the last Conservative MP to go. John Lamont would. Mundell goes down when the SNP reach 49% and Lamont only when the SNP hit 50%. And even with the SNP on 50% Ian Murray and Alistair Carmichael stay. Guess these two have some sort of secret lucky charm.

Also note that there is a noticeable return on investment for the SNP up to 46%. There's not much to be gained beyond that as it is a typical case of diminishing returns:

• up from 38% to 42% = 9 seats
• up from 42% to 46% = 5 seats
• up from 46% to 50% = 3 seats

Most probably the extra cost in campaign expenses would not be worth the extra gains. Except of course for the sheer pleasure of unseating the last two Scottish Tories.

What comes next?

Your guess is as good as mine. Right now my tenner is on at least another poll before the end of the month. With this year's snap election fast approaching and August the worst month for serious polling, I guess 'somebody somewhere' will want an update on Scotland's state of mind quite soon. 

Stay tuned.

Saor Alba gu bràth