Most of the glitterati who flock to the Hamptons in the summer months are newly rich, said Miller. That includes Howard Stern, the radio talk-show host, and actor Jerry Seinfeld. Taking part in the annual summer exodus from Manhattan, either by roads jammed with other refugees or, for the wealthiest, via helicopter, is considered a status symbol by investment bankers and CEOs, he said.

In the town of Mount Desert, many summer residents earned their money the old-fashioned way -- they inherited it. The great-great-grandchildren of the Rockefeller dynasty tend to spend summer days sailing on the Gulf of Maine or hiking in nearby Acadia National Park, where John Rockefeller Jr., the son of the Standard Oil founder, built a network of carriage roads almost a century ago so he could ride his horse without being annoyed by newfangled automobiles.

Exclusive Ticket

In the Hamptons, planning and attending charity fundraisers is the main summer sport. One of the most exclusive tickets is the VIP tent for the final day of the Hampton Classic Horse Show on Labor Day weekend, where ladies wear elaborate hats and waiters dress in white coats.

Mount Desert summer residents reserve that weekend for the Labor Day Series, an annual sailboat race sponsored by the Northeast Harbor Fleet, the yacht club where the younger David Rockefeller Jr. and Armour Mellon, whose family founded Mellon Bank, keep their boats. At the awards ceremony, the dress code is fleece and boat shoes.

"You couldn't get more different than the Hamptons and Bar Harbor, but what the two real estate markets have in common is that disconnect with other segments of the housing market," Miller said.

The assets of Americans who own at least $1 million, in a tally that excludes real estate, grew 8.4 percent in 2010, according to the Merrill Lynch Global Wealth Management and Capgemini World Wealth Report in June. In the same period, U.S. personal per capita income rose just 2.8 percent to $39,901, according to the Census Bureau.

Lone Survivor

David Rockefeller Sr., 96, the only surviving grandchild of oil tycoon John D. Rockefeller, owns more than a dozen properties in Mount Desert with a combined value of more than $37 million, including most of Bartlett Island off Mount Desert's west coast, according to assessment records. Sascha Rockefeller, the granddaughter of former U.S. Vice President Nelson Rockefeller, owns a $1.7 million home, and Abby Rockefeller, the great-granddaughter of the founder of Standard Oil, has a $1.1 million estate.

In all, there are more than a dozen Rockefellers with property along the southern coast of the island.

"Mount Desert is quiet money, discreet wealth," said Brad Emerson, a historian and antique shop owner in nearby Blue Hill, Maine. "From the time the Rockefellers and the Vanderbilts and the Mellons and the Astors first started coming to the island, until today, it was considered bad taste to flaunt your money."