The number of U.S. households with a net worth of $1 million or more, not including primary residence, rebounded by 200,000 to 8.6 million in 2011, according to the Affluent Market Insights 2012 report issued Wednesday by the Spectrem Group.

It marks the third consecutive year of growth following the 27 percent decline in millionaire households to 6.7 million in 2008, but the total millionaire population remains below the pre-recession high in 2007 of 9.2 million.

But even as millionaire households climbed to 8.6 million last year, over 80 percent of those wealthy investors queried said that reaching that financial pinnacle is now harder than ever.  

And despite their increasing portfolios, wealthy investors remain worried about the future. An estimated 83 percent of the 1,252 affluent investors with net worth of $100,000 or more, not including primary residence, indicate attaining the American dream will be harder for future generations. And while wealthy investors under 40 identify owning a home as achieving the American Dream, older investors say it means having sufficient retirement assets. All investors, however, agree the American Dream can be defined as "an equal opportunity for all people."

"Even if they are not directly impacted, continuing high unemployment and the depressed housing market are bedeviling wealthy investors," said George H. Walper Jr., president of Spectrem Group. "Investor optimism has climbed from negative in April 2011 to neutral in February 2012, according to our monthly Spectrem Affluent Investor Confidence Index, but investor outlook won't significantly improve until unemployment falls significantly lower."

Like unemployment, Walper says housing is a bellwether for investor optimism. "Even though the wealthy have substantial assets in addition to their primary residence, their homes remain an important investment and a touchstone of achieving the American Dream," he said.

The ranks of all affluent investors, based on net worth excluding primary residence, increased in 2011:

* Those with $100,000 or more reached 36.7 million from 36.2 million in 2010.

* Those with $500,000 or more climbed to 13.8 million from 13.5 million in 2010.

* Those with $5 million or more rose to 1.078 million from 1.061 million in 2010

* Those with $25 million or more grew to 107,000 from 105,000 in 2010

Spectrem Group's Affluent Market Insights 2012 report is based on surveys with 12,519 affluent investors conducted throughout 2011. Spectrem is a Chicago-based strategic consulting firm specializing in the affluent and retirement markets, integrating proprietary research with expertise in building business, marketing and M&A strategies.