Brexit report: Do 29 per cent of Remainers really want to deport EU citizens? And does 70 per cent of UK back hard Brexit?

A report in The Independent last Friday relayed the findings from a new study that suggested almost a third of Remain voters “would accept” the expulsion of all EU workers in 2019.

The same study was also reported by a host of other media outlets as showing that an overwhelming majority of the British public are now in favour of a hard Brexit – meaning the UK leaving the single market and customs union – in less than two years’ time.

The first finding caused understandable distress to many Europeans living in Britain, since they were led to believe that even many people who had wanted the UK to remain in the EU now wanted them expelled.

And the second has been seized upon by Brexiteers as evidence that their hardline approach actually enjoys wide public backing.

But do these reports accurately represent what the study shows? 

Do 29 per cent of Remainers want EU nationals expelled?

No. This is a deeply unfortunate misreading the results of the research, which was conducted by Sara Hobolt of the London School of Economics, Thomas Leeper, also of the LSE, and James Tilley of Oxford. It was funded by the UK in Changing Europe initiative, which describes its mission as being to provide “an authoritative, non-partisan and impartial reference point for those looking for information”.

So what does the research show then?

First it’s important to understand that this was not a conventional opinion polling exercise in which a sample of respondents are asked a straightforward question and their answers published after some standard statistical adjustments for national representativeness.

Instead, it was a complex and innovative process designed to “reveal” people’s underlying preferences when faced with forced trade-offs over potential outcomes.

What happened then?

The researchers asked the polling company YouGov to recruit 3,000 people to take part in an online survey in April.

Each person was presented with two menus of Brexit-related outcomes and asked to choose which one they preferred. The option of “neither” was not available.

These two menus each had eight outcome items on them relating to, among other things, how much Britain pays as part of the EU divorce settlement, the jurisdiction of the European Court of Justice post-Brexit, the new immigration regime, and future trade arrangements with the EU.

The outcomes in each item ranged between extremes. For instance in the Britain’s exit payments to Europe the options ranged from “nothing” to “£70 billion”. For immigration the range was between “full control” and “no control”.

The mix of the two menus offered to respondents was randomly generated. After their first choice, respondents were asked to repeat the process a further four times, every time with two new randomly-generated menus of options.

What’s the value of all this?

The researchers used the pattern of choices by respondents to generate a kind of favourability rating for each item.

And because they knew in advance whether the respondents had voted Remain or Leave in the June 2016 referendum they were able to break this relative favourability split between the two camps.

So where does the part about EU nationals come from?

One of these menu options was the “future rights of current EU nationals in Britain”. And one of the extreme options presented for this category was “all must leave”.

And here is the important part. No one was directly asked: “Do you think all EU nationals should leave Britain after Brexit”?

Instead, this was an option that cropped up as part of large bundle of outcomes that respondents were asked to choose between.

On the favourability index the researchers constructed this option got a score of about 29 per cent for Remainers and 42 per cent for Leavers.

But that does not mean 29 per cent of Remainers support the deportation of EU nationals.

As the researchers stress: “These values cannot and should not be interpreted as the raw or unconstrained percentage of respondents supporting a given feature level; instead it measures the degree to which individuals trade-off any given level for all of the others.”

But a score of 29 per cent still sounds significant…

Actually, it demonstrates how unpopular the idea is, both among Remainers and Leavers.

The “neutral” score on the favourability index is 50, meaning anything below that is viewed negatively.

It is true that the score could technically have been zero, implying that whenever the option was presented to respondents as part of a package they rejected the entire package every single time.

But the “all must leave” was easily the lowest rated item of all the options, generating a lower score even than other extreme options such as “no control over immigration” among Leavers (which got a score of 34 per cent).

All in all this research corresponds with those from other conventional opinion polling, which suggest very strong public support for allowing EU nationals who have settled in Britain to stay post-Brexit.

What about the idea of massive public support for a hard Brexit?

To some extent this is based on a similar misreading of the results of the research by journalists.

Some newspapers reported the findings of the favourability index for propositions such as “paying nothing towards a Brexit divorce bill” (54.7 per cent among Remainers) and “full control over borders” (51.3 per cent among Remainers) as if they were raw percentages of support among the Remainer population – the same mistake as suggesting 29 per cent of Remainers back expulsion of EU nationals.

But there was one additional element of evidence for this proposition. As well as asking respondents to choose between various randomly-generated baskets of outcomes, the researchers also presented them with three “fixed” baskets.

And these were constructed by the researchers to roughly resemble three Brexit outcomes often described as “soft”, “hard” and “no deal”.

So for “soft”, it included no control on migration, continued jurisdiction of the European Court of Justice and no tariffs.

For “hard” there was full immigration control, trade administration barriers and 5 per cent tariffs on exports.

The “no deal” basket included no divorce payment, passport checks on the Northern Irish border and no access to the EU Single Market.

Then respondents were forced to choose between the baskets (without being told what they were designed to represent). And this showed that when forced to choose between a hard Brexit and a soft Brexit 67 per cent preferred the hard basket. And even among Remainers 53 per cent preferred hard.

Furthermore, 66.5 per cent of all respondents even preferred a no deal Brexit to a soft Brexit.

And these were genuine percentages of recorded preference, unlike the favourability index scores.

So does this mean that people really favour a hard Brexit?

This is a highly contestable inference. One reason is that, again, this was not a normal opinion polling process. Respondents were forced to state a preference. They were not given the option of saying “don’t know” or “neither”.

Further, respondents were not given any sense by the framing of the question of the consequences of the choice of basket. For instance, in the “no deal” basket there were superficially attractive items such as “no payment” to the EU and rather anodyne-sounding ones such “some administrative barriers to trade and 2.5 per cent tariffs”.

There was no mention of the fact that “no deal” would mean the UK being in breach of its international treaty obligations. There was no hint at the travel chaos and economic pain that would inevitably follow such a “cliff-edge” outcome.

Similarly, the hard Brexit scenario contained no mention of the economic pain relative to the soft scenario.

While the researchers, understandably, wanted to avoid prejudicing the choices of respondents by elaborating on consequences in what might be seen as a leading way, their failure to give respondents any relevant economic context to their soft, hard and no deal baskets arguably went to the other extreme and undermined the value of their findings.

The claim of Sara Hobolt that “there is on aggregate higher levels of support for outcomes that resemble the ‘hard Brexit’ position put forward by the government” does not seem reliable given that the “outcomes” selected by the researchers were so limited.

To put it another way, the researchers claim they were forcing respondents to express a preference in the face of an inevitable trade-off. Yet how can it be a genuine trade-off exercise if all relevant consequences are not outlined?

So what lessons can be drawn from this fiasco?

There is blame right down the line for the misinterpretation of this research.

The researchers released their finding as an “exclusive” to a single news source – Buzzfeed – on Friday which then presented the findings with a heavy dose of analysis from one of the authors but with no substantive explanation of the methodology.

Buzzfeed also presented the findings in a series of charts which gave the appearance that they were showing levels of support for various propositions in the manner of a conventional opinion poll. Confusion was added by the fact that the headline findings presented by Buzzfeed did pertain to rough percentages of the population.

The result was that most journalists - including the Independent's- followed up the Buzzfeed story without fully grasping the methodology.

The researchers did make details of their methodology available on Friday and Saturday to journalists and anyone else that requested it.

Yet they did not publish the full document online until Sunday afternoon, meaning that for the best part of two days statisticians and the general public - unless they directly contacted the researchers - were unable to access it and had to rely on partial and flawed media reports.

When conventional opinion polls are published by news organisations the full raw results are published online by the polling company simultaneously, so that anyone can look at the underlying data and the precise framing of the questions.

It was a great shame that this was not done with this report, especially since the researchers were using a non-conventional opinion-sampling technique and where the results related to an area of such obvious and heightened politically sensitivity