An Indian tricolour was torn down from a flagpole at London’s Parliament Square on Wednesday during protests against Prime Minister Narendra Modi. India’s Ministry of External Affairs said that it had taken up the matter with the British Foreign Office.

“We’re deeply anguished with the incident involving our national flag,” Ministry of External Affairs spokesperson Raveesh Kumar told a press conference. “The matter was taken up strongly with the United Kingdom side. They have regretted the incident. The flag was immediately replaced. We expect legal action against the people who were involved in this.”

The incident took place during Prime Minister Narendra Modi’s address to the Indian diaspora in Westminster. Hundreds of supporters and protestors had awaited Modi’s arrival in London, holding placards and shouting slogans at Downing Street and the British Parliament. The protestors held banners which said “Modi go back” and “We stand against Modi’s agenda of hate and greed”.

Some pro-Khalistani demonstrators from the Sikh Federation UK and some Kashmiri separatist groups also participated in the protests, PTI reported.

The United Kingdom’s foreign office expressed regret for the incident, the Hindustan Times reported. “While people have the right to hold peaceful protests, we are disappointed with the action taken by a small minority in Parliament Square and contacted High Commissioner Yashvardhan Kumar Sinha as soon as we were made aware,” a statement said.

“The visit to the United Kingdom by Prime Minister Modi has strengthened our relationship with India and we look forward to working even more closely together on a number of important areas,” the foreign office statement added.