In most assessments of Newt Gingrich’s presidential prospects (Gingrich began “exploring” a run for the Republican nomination in March), the former House Speaker’s three marriages, and the way he comported himself in the first two, are typically identified as his greatest liability.

But Gingrich’s past infidelity is typically seen as a problem for him solely among religious and socially conservative voters. As Newsweek‘s Howard Kurtz put it, “Religious voters won’t flock to a guy who’s had three wives.”

That characterization is, well, too conservative. There is reason to believe that Gingrich will find his past indiscretions are a liability far beyond the Religious Right.  

Marital infidelity remains a deep taboo in America. A 2008 Gallup poll found that nearly two-thirds of Americans (64%) said they would not forgive their spouse for an extramarital affair. Only 10% said they would definitely forgive him or her.

In a 2007 Gallup poll, a sample of Americans was asked to rate the morality of 16 social issues — from euthanasia and cloning to gambling and homosexuality. The issue that netted the lowest acceptability rate (the difference between the share that thought the activity was “morally acceptable” and the share that felt it was “morally wrong”) was cheating on one’s spouse.

Ninety-one percent of respondents felt “married men and women having an affair” was “morally wrong,” while just 6% said it was “morally acceptable,” producing an acceptability rate of -85. Cheating was less acceptable than polygamy (-82) and cloning humans (-75).

Of course, voters are often willing to tolerate behavior in their elected representatives that they would never abide in their partners. President Bill Clinton’s approval rating peaked during the Lewinsky sex scandal.

But Newt shouldn’t take solace. A 2007 Pew poll found that nearly four in 10 voters said that an extramarital affair would make them less likely to vote for a candidate. Among Republicans, more than six in 10 voters felt that way.

Gingrich is guilty of more than just having an affair. He allegedly handed his first wife divorce papers while she was in the hospital recovering from cancer treatment. And he was involved in an affair with his current wife, Callista, when she was a young staffer in his office in the mid-1990s — as he was pursuing impeachment of President Clinton for lying about an affair with a young White House staffer. It’s a perfect storm of infidelity, callousness, arrogance and hypocrisy.

Gingrich has plenty of other liabilities, mind you. He hasn’t held elected office in 13 years. And the last time he did, he left the stage in shame with a 70% disapproval rating. He became the first Speaker in U.S. history to be disciplined by the House for ethics violations.

Gingrich has been active since leaving office, writing books and producing documentary films. He’s still regarded as the one of the GOP’s brightest minds. But these days his most memorable forays into the political spotlight — the brash statements, provocative Tweets and erratic policy positions — often seem rooted more in a desire to be noticed than in sincere conviction or careful observation.

All of this has left Gingrich unpopular even with voters in his home state. An April survey found that just 39 percent of Georgia voters have a favorable opinion of Gingrich. Forty-seven percent view him unfavorably. Less than a third of voters surveyed (31%) feel Gingrich should run for president.

Gingrich hasn’t raised much money either. According to a report filed with the Federal Election Commission, his political action committee raised a paltry $53,000 in the first quarter of 2011.

Gingrich hasn’t handled his past indiscretions very well in public. When asked recently about his prior unfaithfulness, Gingrich told the Christian Broadcast Network “there’s no question, at times of my life — partially driven by how passionately I felt about this country — that I worked far too hard and things happened in my life that were not appropriate.” This statement earned him scorn from across the political spectrum, and the nickname “The Horny Patriot.”

Callista Gingrich appears more and more at public events with her husband. Mrs. Gingrich, a life-long Catholic, is credited with helping in Gingrich’s 2009 conversion to Catholicism. But her presence on the campaign trail may merely remind voters, especially women voters, that she was “the other woman” who broke up Newt’s second marriage.   

Worst for Gingrich, his sordid marital past limits how effectively he can talk about values. Gingrich regularly rails against President Obama’s “anti-Catholic” values.

And he is right to do so. Much of Obama’s policy agenda, including his abortion advocacy, is at odds with not only Catholic teaching but also the values of most Americans.

For Gingrich to raise values as often and as loudly as he does, however, is to invite voters (not to mention political opponents and the media) to contrast Gingrich’s personal conduct with the president’s, which by all accounts is exemplary.

When confronted about his marital past on the pre-campaign trail, Gingrich says he believes in a forgiving God and hopes voters do too. He recently said that if he runs for president, voters will have to “render their judgment” about his past indiscretions.

I suspect many voters will wish Gingrich well in his marriage and new-found faith. But if he runs for president, given the number of alternative candidates who don’t have Gingrich’s political or personal baggage, I suspect most will end up voting for somebody else.