If Brazil doesn’t accept former settler leader Dani Dayan as its Israeli ambassador, it won’t get any ambassador at all, said Prime Minister Benjamin Netanyahu on Wednesday.

Israel’s Channel 2 news reported on Netanyahu’s comments, which came after rumors that Israel would withdraw Dayan as its nominee and send him to Los Angeles or New York to serve at an Israeli consulate instead.

Brazil objects to Dayan’s former position as head of the Yesha Council, which comprised the municipal councils of Jewish towns in Judea and Samaria. Leftist groups argue that Brazil, which in 2010 recognized Palestinian statehood in Israel’s territories, would be tacitly approving Israel’s settlements by accepting Dayan.

So far, Brazil has refused to accept or even officially acknowledge Netanyahu’s choice.

A senior official in Brazil’s Foreign Ministry said last month that he “did not see” Dayan’s appointment happening, and added that Israel would have to choose a different diplomat. However, Netanyahu has come out strong against Brazil’s resistance, refusing to offer an alternative or withdraw the appointment.

Earlier this week, 40 retired Brazilian diplomats signed a statement protesting Dayan’s appointment. The statement accused Israel of bypassing protocol by not holding prior communication with the Brazilian Foreign Ministry.

If Israel does not have an ambassador in Brazil, it would mean a downgrade of the already-strained diplomatic relations between the two countries. Israeli officials have made clear that they are prepared to accept those consequences.