Congress is inching toward a "fiscal cliff" at the end of year, when it will be forced to decide whether taxes go up and spending gets slashed, possibly stunting economic growth. That cliff has dampened investors' enthusiasm, according to a new survey.

Congress' alternative is to cancel some of the taxes and cuts and increase the federal deficit. Not only do investors fear how the economic issue will be resolved, but the presidential election and the divided federal government are seen as major concerns, suggests the survey, called the "Investor and Retirement Optimism Index," which was developed by Wells Fargo and Gallup.

If Congress does not resolve the issue, 61% of those surveyed feel the U.S. economy will go into a recession next year. In addition, 71% are concerned the looming fiscal cliff will force consumers and businesses to pull back on current spending and investing, which is likely to slow the economy for the rest of this year.

Those concerns and others pushed the Wells Fargo/Gallup "investor optimism index" down from a score of 40 in February to 24 in May and then to 16 last month. The index was developed from a baseline of 124 in 1996. It reached a peak of 178 in January 2000 at the height of the dot-com boom. It later hit a low of negative 64 in February 2009.

"It is the uncertainty of all of the looming issues that is pushing the optimism level down," says Joe Ready, director of institutional retirement and trust at Wells Fargo. "And investors also are saying the elections and uncertainty of dealing with economic issues could affect their net worth.

"What surprised us is that when investors were asked what factors are affecting the investment climate, 69% said a top factor is the politically divided federal government, an increase from 64% in May," Ready says.

Unemployment was ranked as a top concern by 67% of respondents, and the same number cited the federal budget deficit. The percentage of people willing to put faith in the stock market also declined. Only 41% of investors say now is a good time to go into the markets, down from 48% in May and 52% in February.

"People watched the impasse of the debt ceiling negotiations last summer and the effect the breakdown had on the markets and 401(k) balances, so it makes sense that investors are attuned to policy affecting debt and taxes," Ready says. "Investors are clearly telling us they worry about a recession."

Three-quarters of the respondents say the presidential and congressional elections will impact their net worth. Three-quarters also say it is up to the president, Congress and the private business sector together to fix the economy.

When asked to compare their financial position to where they were before the 2008 election, investors are almost evenly split; 37% say they are better off and 33% say they are worse off.

Although 78% say their household income has increased or stayed the same in the last four years, only 24% say they are saving more. Asked to rank their most important savings priorities, 41% say it's saving for retirement, 24% say saving for an emergency and 11% say saving for education.

--Karen DeMasters