The Spectrem Affluent Investor Confidence index released on Wednesday fell 11 points in June to -9, its biggest drop in more than 3 years, indicating that affluent households aren't very confident about a U.S. economic recovery.

The decline is the largest since January 2008 for the affluent index, which measures the investment confidence and outlook of households with $500,000 or more in investable assets. It brings the index to its lowest level since October 2010, when it stood at -13, and keeps it in neutral territory.

Meanwhile, Spectrem's Millionaire Investor Confidence Index fell 2 points in June to -1, also a neutral reading.

"Affluent investors posted their biggest decline in investment confidence in more than 3 years in June, with millionaires' confidence falling slightly as well," said George H. Walper, Jr., president of Spectrem Group, in a statement.

Walper said the nation's wealthiest investors overwhelmingly cited the economy as their biggest concern.

"Given the worrisome economic news that emerged in June, it's not surprising to see wealthy Americans struggling with their investment optimism," Walper said.

In response to an open-ended question about the most serious threat to achieving their household financial goals, affluent investors in June cited: the economy (35%); the political climate (14%); market conditions (13%); inflation (9%); unemployment (7%); and health-related issues (1%).

Those citing the economy rose from 22% in March 2011, the last time this question was asked, while those citing the political climate fell from 22%
Millionaires were less focused on the economy (28%) than the affluent but more focused on market conditions (19%).

-Jim McConville