The police stormed the Manus Island detention centre in Papua New Guinea on Thursday to force out the asylum seekers still living there, three weeks after the centre was shut down.

Australia holds asylum seekers who come to their shores by boat in camps on Papua New Guinea’s Manus Island and the small Pacific nation of Nauru. On October 31, the country decided to shut down the camp after a court called it unconstitutional.

But hundreds of men living there said closing the camp was a breach of their human rights and refused to move out as they feared violence and backlash from the local community living in the areas they will be moved to.

On Thursday, the Papua New Guinea Police entered the Manus Island centre and gave its residents an hour to leave, the BBC reported. One refugee, an Iranian journalist who was detained and released, claimed that the police beat them up.

“They handcuffed me for more than two hours in a place behind the prison camp,” Behrouz Boochani, the journalist, said on Twitter after his release. “The police commander yelled at me ‘you are reporting against us.’”

Several other refugees described the police as intimidating and aggressive and said they dismantled structures and threw away their belongings, The Guardian reported.

Australia maintains that the police operation was conducted by Papua New Guinea, but its Immigration Minister Peter Dutton confirmed that a “small number” of people were arrested on Thursday.

Dutton said several others had already moved out “of their own accord and otherwise” to accommodation elsewhere on the island, Sky News reported. “The operation is still under way, but it is important that we get people out of the regional processing centre into much better accommodation as quickly as possible,” he said.

Prime Minister Malcolm Turnbull said Australia will not be pressured by the standoff to take in the men being evicted. “They think this is some way they can pressure the Australian government to let them come to Australia,” he said in Canberra, reported. “Well, we will not be pressured.”

The United Nations and several rights groups have, for years, said there were human rights abuses in the Manus Island centres.