The US immigration debate had been hijacked by a vocal anti-immigrant minority in the Republican Party which did not reflect the rest of the party's views, experts said Monday.

"It's time for us to stop listening to the people with the smallest brains and the biggest mouths," said Richard Land of the Southern Baptist Convention at a panel organized by Washington-based think tank the American Enterprise Institute.

Panelists said the Republican Party was a natural fit for Latinos, the country's largest immigrant group, many of whom were socially conservative, family-oriented, heavily religious, anti-abortion and highly entrepreneurial, all of which reflected Republican values.

But a vocal minority within the party were successful in stirring up anti-immigrant fervor and intimidating moderate Republicans into keeping silent, experts said.

Republican presidential candidate Mitt Romney played to that minority when he made his "self-deportation" statement during the Republican primary debates, which was widely interpreted to mean undocumented workers should return to their home countries, the experts said.

Panel member Alfonso Aguilar, of the Latino Partnership for Conservative Principles, said Hispanics objected to Romney's tough stance on immigration.

Indeed, it caused many Hispanics to perceive the Republican Party was anti-Latino and, in the end, 71 percent of the Latino vote went to the Democrats, while Republicans took only 27 percent of the crucial demographic.

The massive loss of the Latino vote has opened a post-election rift in the Republican Party, with some party pundits calling for a more moderate stance on immigration in a bid to win more of the Hispanic vote in the next presidential race.

Conservative columnist Charles Krauthammer earlier this month called for an amnesty for those already in the country illegally, while conservative talk show host Sean Hannity said his thinking on the issue had shifted in light of the Republican Party's landslide loss of the Latino vote in the recent presidential election.

Meanwhile, the Republican Party has sent mixed messages on immigration. A host of Hispanic speakers were invited to the Republican National Convention held in August in Florida, including Cuban-American Senator Marco Rubio and Mexican-American New Mexico Governor Susana Martinez.

But, at the same time, the guest list included polarizing figures such as Sheriff Joe Arpaio, billed by many as a draconian anti-immigrant zealot, and Arizona Governor Jan Brewer, blasted by opponents for advocating a hardline approach to immigration enforcement.