The rising tide has not lifted all boats, but that may be because not everybody has wanted to get back in the water.

The mass affluent, as opposed to the very rich, have not seen as great a rebound in their ranks since the recession, according to a new report, “Affluent Market Insights 2013,” by the Spectrem Group. While households with more than $1 million are close to recovering to pre-recession levels, the number of households with $500,000 to $1 million have seen a more sluggish climb.

The number of households with $1 million or more (minus homes) has climbed nearly 400,000 in 2012 to 8.99 million. That comes off a 6.7 million nadir in 2008, but it’s still below the 2007 peak of 9.2 million. Those with $25 million or more grew to 117,000 in 2012, up 10,000 from the year before and up from 84,000 in 2008. Those with $5 million or more rose to 14.3 million from 13.8 million in 2011 and 11.3 million in 2008, according to Spectrem.

After the steep market implosion in ’08, the number of wealthy households naturally declined. “Those with more wealth were able to better weather the storm because of the diversity of their investments,” says the study. “Since that time, the mass affluent [households with $100,000 to $1 million of net worth, excluding their homes] have not been as eager to return to the market, while those with more wealth have been more willing to jump back into the market. Therefore, the growth of those with less wealth has not been as rapid as wealthier households.”

The study also found that while all investors are slightly more bullish after a less confident 2012, millionaires are far more bullish than the merely affluent. Risk tolerance plays into that. Though more investors define themselves as aggressive or moderate, as opposed to conservative, since 2009, and though all groups are more interested in equities, the mass affluent are more concerned about their current financial situation. Seventy-two percent of them said that maintaining their current position was the most important thing to them, while only 61 percent of ultra-high-net-worth individuals said the same. The latter group was more worried about setting aside money for children and grandchildren (66 percent rated it as a concern).

“When asked how they will invest in the next 12 months,” said the study, “ultra-high-net-worth investors named equities first. The mass affluent, however, named checking/savings accounts as their first choice. Millionaires chose money market funds.”

The study also says that during the market crisis, clients wanted to be more self-directed and turned away a bit from advisors. Of those surveyed, 15 percent said they were self-directed at the beginning of 2008, but 41 percent said they were by the end of that year. Four years later, people who said they were either advisor assisted or advisor dependent had risen closer to pre-recession levels.

Spectrem, a Lake Forest, Ill.-based consulting firm focused on the wealthy, bases the report on monthly online polls it performs with investors from all wealth segments.