Caribbean Association of National Olympic Committees (CANOC) President Brian Lewis believes his organization’s member countries are being "sold short" by what he claims to be "undemocratic" voting rules at Panam Sports.
Lewis raised the issue when presenting his CANOC report at this week’s Panam Sports General Assembly in Chilean capital Santiago.
Under the Panam Sports Constitution, members that have staged the Pan American Games can cast two votes at General Assemblies while others have one.
The ruling only relates to the elections of Executive Board members and the host city of the Pan American Games.
Lewis told insidethegames it was a "fundamental" problem and urged Panam Sports to change the statutes.
"The majority of us in the CANOC Executive believe that it is undemocratic and with the way the Games are getting bigger, it is becoming out of the reach for many Panam Sports countries," said Lewis.
"For me, the real issue that we have with one country, one vote is fear.
"That fear is based on a lack of trust.
"I think it’s selling the CANOC region short, it’s saying to us that we are immature and that we can’t make decisions."
During his presentation at the General Assembly, Lewis also called for Panam Sports to make Anguilla, Curaçao, French Guiana, Guadeloupe, Sint Maarten, Turks and Caicos Islands, Martinique and Montserrat all associate members to allow them to compete at the Pan American Games.
"They have shown by their participation at the Caribbean Games that the young people of these countries seek an opportunity and want to participate against the best against the region which is Pan American Games," Lewis told insidethegames.
"It is a conversation that is ongoing.
"I would have thought there would be more progress by now but there are people that keep talking about the statutes.
"The people who continue to raise this create a barrier to moving the conversation forward.
"I remind that colonialism, slavery, segregation and women not being able to vote was once a law and all those statutes had to evolve to engage the reality of modern time."
Lewis underlined the efforts of CANOC to support Panam Sports but felt their issues were falling on deaf ears.
"We in CANOC sometimes feel that while we are heard, we are not listened to," said Lewis.
"In the absence of listening, it creates the environment for misunderstanding, jumping to conclusions and misconceptions and people feeling that issues that are maybe important to them are being diminished.