James Cleverly is a happy man. He was one of the biggest campaigners to get Britain out of the European Union.

But even the Conservative parliamentarian was a bit uneasy when his side won.

“Deep down all of us — all humans — are a bit nervous about change. The British particularly so, and we have embarked on a really exciting, but very significant change.”

The referendum results revealed a tale of two Britains. In big cities like London, people voted overwhelmingly to stick with the EU. But those in smaller towns around the UK, particularly in deprived areas, wanted to bolt.

In those towns, says Cleverly, “They are disillusioned with politicians as whole — not just European politicians. And they want to see a big change. We need to respond to that.”

Cleverly represents people in Essex, where 61 percent of his constituents voted to give the EU the boot.

Aspects of the campaign made him uncomfortable, however. Cleverly, whose mother emigrated from Sierra Leone, did not like an anti-EU, anti-immigration poster that showed a refugee from the Balkans with the words “BREAKING POINT.” “I think it is deeply, deeply unpalatable. But those ideas have been given oxygen because a lot of people feel that the British immigration system isn’t working,” Cleverly says.

“One of the by-products of our membership of the EU is that we have to accept completely free movement of any EU citizen. To put that into perspective, it’s like the USA having a completely open border with every country in Latin America, and having absolutely no say in who comes. I’m sure that is something that many Americans would feel uncomfortable with. And many British people were uncomfortable with that. This needs to be addressed so that we can reduce some of the tension about immigration.”

Many Poles are wondering what the Brexit vote will mean for them, as there are hundreds of thousands of Polish immigrants living in the UK right now, many who deal with a negative stereotype that they are just depriving the British of jobs. Poles are the second largest overseas-born community in the UK after Indian-born residents.

Many Polish workers are “afraid of what Brexit might bring,” says Roch Dunin-Wasowicz, a sociologist at the London School of Economics who has researched Polish workers in England.

“Those who have lived here for more than five years automatically acquire a right to residency under British law,” Dunin-Wasowicz says. “However, around half of the Polish people living here have been here shorter than those five years and their status is very uncertain.”

UK citizens living abroad have expressed grave concerns over Brexit. Alastair Gee is a British expat living in San Francisco. He’s a journalist and frequent contributor to the New Yorker. Last week we spoke to him before the vote. He told us he was planning to get his brother to cast a proxy vote for him in favor of staying in the EU.

Gee voted, but the results are not what he and his family had hoped. On Friday he told us, “I think almost everybody in my family voted to remain and they can’t quite comprehend that its happened and I think are frightened for the future.”

There’s an eight-hour difference between San Francisco and London. Gee was watching coverage all day. At first he was encouraged, but when the results started to come in from cities in the northeast of England like Sunderland and Newcastle he started to get nervous. “At that point I got a sick feeling in my stomach … and whatever it was like for me watching the news unfold in slow motion, I think for them it felt like a punch to the stomach.”

Now, he feels deeply disappointed in the UK. “The decision has been made, and as hard as it is, Britain has to step into that future and live in it.”

Some are describing Britain’s vote to leave the European Union as the beginning of the end for a European Union. That ominous message is reaching Brussels, the heart of the EU.

“It’s actually really depressing right now, a lot of people are in shock and can’t believe it, “says Tara Palmeri, a Brussels-based reporter for POLITICO.

“My only analogy would be you’re having problems with your spouse, and then you wake up to divorce papers. Everybody was sort of in denial for the past year. But the British people clearly are not happy with their membership in the EU so that means the EU needs to change if it wants to keep the rest of the club together.”

The UK’s vote already is prompting far-right parties in other EU countries, including France, the Netherlands and Denmark, to demand their own referendums. So the question is: What should Brussels do or say to all of these countries that are running for the exits?

“I really do think that Brussels needs to evaluate this idea of a political union.” says Palmeri.

“Twenty-seven EU countries represent too much of a difference. Portugal and Poland for example are two very different countries with very different political situations, and if you tell these people, ‘You have to vote the same way, you have to agree with all the same rules,’ it becomes very difficult, and the EU may have pushed it too far. I think they really tried for this federalist super-state, the United States of Europe and this is what happened. It’s the backlash,” says Palmeri.

The Brexit vote has already sparked a domino effect. The far-right party in France has called for its own “Frexit.” But people in France, pained and panicked from Thursday’s UK vote, may need time to heal before they jump on to a referendum of their own. Reporter Adeline Sire in Burgundy, France explains the atmosphere there.

“It feels like the shocking end of a beautiful friendship,” Sire says. “It was all very abrupt. People feel abandoned by this very powerful country.”

There will be a difference in generations now; those who grew up with the UK in the EU, and those who will have not.

“I grew up with the idea of being a European citizen,” Sire says. “I like being able to travel through borders and the only thing that tells you that you’re in another country is the chime of a notification of a network change from your cell phone carrier.

“I just feel like we’re regressing, Europe is regressing. And I’m very concerned about that. I see that as an erosion of the freedoms we have taken for granted for all these years.”

Also, she adds, “British shortbread cookies might become very expensive.”