(Bloomberg News) Slumping stocks of real estate investment trusts that buy mortgage debt signal the companies are paying a price for growing ten times larger since 2004, according to Stifel Nicolaus & Co.

Much of the growth since a similar point in the last interest-rate cycle to a total market capitalization of about $53 billion has reflected equity sales to retail buyers and large dividends "that a sizable portion of the investor base doesn't understand," Michael Widner and Sean C. Tillman, analysts at Stifel Nicolaus, said yesterday in a report.

The group is "reaping what it's sown" after attracting "a lot of investors that always have one foot out the door," the analysts said. "When widespread consensus amongst most holders," including institutional investors, "is 'don't be the last one out' it is difficult to bracket downside risk when somebody thinks they might have heard somebody yell, 'FIRE!' "

A Bloomberg index of shares in the REITs tumbled as much as 2.3 percent yesterday, the most intraday since November, with the measure declining 2.7 percent this week as of 12:35 p.m. in New York. The Federal Reserve's expanded mortgage-bond buying announced last month is also creating pressures on the companies' earnings and dividends, Stifel Nicolaus said.

While still positive on the stocks, with even reduced dividends likely to be attractive to investors amid depressed bond yields, the Baltimore-based analysts said they are "less enthused than we've been any time in the past four years" and expect the shares to be volatile.

Canceled Trades

The REITs' dividend yields are 12.9 percent, based on yesterday's share prices and the firms' recent payouts, according to data compiled by Bloomberg. The stocks have returned 23.4 percent this year, assuming reinvested dividends.

Trades on Oct. 9 that were later canceled after causing split-second disruptions in about 140 exchange-traded funds and companies, such as slumps in mortgage REITs including Annaly Capital Management Inc. and Hatteras Financial Corp., may have caused investors to reevaluate holdings, said Brad Golding, managing director at hedge fund Christofferson, Robb & Co.

"You've been pulling people in" who have no "idea what they're investing in," he said yesterday in a telephone interview. "In talking to various retail and institutional investors, it's the 'greater fool' theory: They think they can get out before everything collapses" once either short- or long-term interest rates rise.

Mortgage REITs have raised $17 billion through equity sales this year, after a record of almost $20 billion in 2011, according to Bloomberg data.

Gary Kain, president of American Capital Agency Corp., the second-largest mortgage REIT after Annaly, said there's probably no greater misunderstanding of the companies' business among shareholders than there is among investors in firms such as Apple Inc., Facebook Inc. and Wells Fargo & Co.

Rising Prepayments

The stock declines are "much more" about fundamental issues "than people being completely scared," he said today in a telephone interview.

The Fed's mortgage-bond buying has reduced yields on the REITs' new investments, while sending home-loan rates to record lows and refinancing applications to the highest level in more than three years. The firms have more to lose from rising prepayments after being forced to pay greater amounts above face value for their holdings, according to Stifel Nicolaus.

"The market was confused by QE3," Kain said, referring the third round of the central bank's so-called quantitative easing, which the Fed announced Sept. 13. "Initially, the market looked at all the benefits and ignored the negatives."

The positives for REITs include the rising values of holdings they can sell to the Fed, while the biggest negative will be an increase in prepayments that return bondholders' investments at par, Kain said. American Capital Agency has targeted debt with protection against refinancing, he said, while declining to comment on its future dividends.

"People are starting to realize the prepayment environment is not benign," he said. "We've been telling people that all along and it won't be affecting everyone equally."