A BBC UK report claims up to 40 countries could boycott the next Olympic Games, making the whole event pointless, said Poland's Sport and Tourism Minister Kamil Bortniczuk.
His comments came after Poland, Lithuania, Estonia and Latvia jointly rejected an International Olympic Committee (IOC) plan to allow Russians and Belarusians to compete in 2024.
Ukraine has threatened to boycott the Paris Olympics if that occurs.
But the IOC said on Thursday that any boycott would only "punish athletes".
Bortniczuk said he believed it would be possible to build a coalition of 40 countries, including Great Britain, the United States and Canada, to support a block on the IOC's plans before a meeting on February 10th.
He added: "Considering this I don't think we will face tough decisions before the Olympics and, if we were to boycott the Games, the coalition we will be a part of will be broad enough to make holding the Games pointless."
The IOC announced last week that it would "explore a pathway" to allow Russian and Belarusian athletes to compete in Paris under a neutral flag, adding that "no athlete should be prevented from competing just because of their passport".
The move prompted condemnation, with the UK Government saying the plan was a "world away from the reality of war".
Ukraine Sports Minister Vadym Guttsait said the country's sporting bodies needed to "strengthen communication" with international federations to keep a ban in place on Russian and Belarusian athletes, which was imposed by the IOC's executive committee immediately after Russia's invasion of Ukraine in February 2022.
However, IOC president Thomas Bach has since said that was only intended as a "protective" measure towards those athletes, and now insists they should not be discriminated against.
On Thursday, sports ministers from Latvia, Lithuania, Estonia and Poland said "any effort by the International Olympic Committee to bring back Russian and Belarusian athletes to compete, even under a neutral flag, should be rejected".
They added: "Efforts to return Russian and Belarusian athletes to international sports competitions under the veil of neutrality legitimize political decisions and widespread propaganda of these countries."