Pompeo's warning came after the ICC prosecutor decided to consider Palestine a state with the ability to submit complaints that could trigger probes into war crimes it says Israel committed in the occupied West Bank and the besieged Gaza Strip.
In December, ICC chief prosecutor Fatou Bensouda announced that a five-year preliminary examination of the “situation in the state of Palestine” had provided her with "reasonable basis to believe that war crimes were committed" by Israel.
Before launching an investigation, Bensouda had asked the Hague-based court's pre-trial chamber to confirm whether the ICC had jurisdiction over alleged offenses committed there.
In a critical statement on Friday, Pompeo described the ICC's investigations "illegitimate" and deemed the international tribunal a "political body, not a judicial institution".
"This unfortunate reality has been confirmed yet again by the ICC Prosecutor’s attempt to assert jurisdiction over Israel, which like the United States, is not a party to the Rome Statute that created the Court,” the statement read.
Pompeo said the US does not "believe the Palestinians qualify as a sovereign state".
"The United States reiterates its longstanding objection to any illegitimate ICC investigations," he said, adding, "If the ICC continues down its current course, we will exact consequences."
Pompeo's statement came two days after his trip to Israel for a meeting with prime minister Benjamin Netanyahu to discuss Tel Aviv's plans to annex parts of the occupied West Bank.
The US and Israel have previously claimed that the ICC does not have jurisdiction over Israel and Palestine, that Tel Aviv is being "targeted unfairly" and that Palestine does not qualify as a state.
On Wednesday, a bipartisan group of US House and Senate legislators sent separate letters to Pompeo urging him to defend Israel, a firm US ally, against ICC investigations, saying the tribunal's assertion amounted to a "politicization" of the court's mission.
Palestine was accepted as an ICC member in 2015, three years after signing the court's founding Rome Statute, based on its "observer state" status at the United Nations.
Both Israel and the US have refused to sign up to the ICC, which was set up in 2002 to be the only global tribunal trying the world's worst crimes, war crimes and crimes against humanity.
Both have claimed they have credible legal systems that can properly adjudicate human rights violations which make ICC intervention dispensable.
Meanwhile, US President Donald Trump’s administration has backed Israel in its battle against the court.
The Trump administration maintains a tense relationship with the ICC, having previously revoked Bensouda's visa when she intended to investigate potential war crimes by US soldiers in Afghanistan.