The era of making a broad bet on mutual fund companies as a way to play America's self-funded retirement boom appears to have ended.

After outpacing the market during both the 1990s and early 2000s bull runs, the stocks of standalone mutual fund companies such as T. Rowe Price Group Inc, Eaton Vance Corp and Franklin Resources Inc have fallen behind, gaining just half the amount of the broad S&P 500 over the last five years.

Analysts say there is little reason to expect a broad snapback in shares anytime soon, largely because the firms, big in the retirement space that mushroomed once 401ks and IRAs began to replace traditional pensions in the early 1980s, are now facing the flip side of those savings plans: withdrawals.

Last year marked the first time that more money came out of 401(k) plans than was added, with $377 billion in withdrawals outpacing $338 billion in deposits, according to Cerulli Associates. Those declines are expected to accelerate as the baby boom generation ages.

At the same time, standalone fund firms are losing assets to lower-cost and less-profitable passively managed mutual funds and exchange traded funds. Private firm Vanguard, best known for its index funds, is the world's second-largest asset manager, behind only BlackRock Inc, whose iShares line makes it the world's largest ETF provider, according to IPE Research.

Even fund companies that establish their own ETFs may find themselves retaining assets by cannibalizing their own more fee-intensive funds. Adding insult to injury, fund company margins have been hurt in recent years by near-zero short-term interest rates that have prompted companies to waive fees on some funds.

After selling off since peaking late in 2014, many fund companies have become cheap when compared with their historical standard and the broader S&P 500 index. T. Rowe Price’s price-to-earnings ratio hit a 20-year low of 14.3 on Oct. 7 and has edged up slightly to 14.5. Franklin Resources’ forward PE stood at 12.1, up slightly from a 3-year low. As of Oct. 12, BlackRock Inc's forward PE was at a 1-year low of 15.1, even with the S&P's.

In the past, these companies have had PEs in the high 20s. Thomson Reuters Starmine holds an intrinsic value of $49.80 for Franklin Resources stock, $407.90 for BlackRock and $90.15 for T. Rowe Price, all well above their recent trading levels.

Tailored Bets

Mutual fund companies still have a lot of money in their coffers. Actively managed funds oversee a total of approximately $8.1 trillion in assets, compared with $1.6 trillion invested in ETFs, according to Lipper data.

But analysts and fund managers say they have to be much more selective to make money by buying shares of fund companies. Even with their depressed share prices, not every one is a bargain.

Investors say it requires specific attributes – a fund lineup with a history of beating peers or a sizable money market business – to help a company expand while the broad industry slows.

“Not everyone is going to be comfortable going with a completely passive approach, especially if the market continues to be volatile,” said Stephen Biggar, an analyst at Argus Research. 

Judson Brooks, an analyst who works on the $17.5 billion Oakmark Fund, said his fund owns shares of T. Rowe Price - down 14 percent year to date - largely because its track record of beating its peers should continue to attract assets.

The company announced in July that 76 percent of its funds outperformed their Lipper averages over the last three years, and 86 outperformed their peers over the last 10 years.

"The essential element of the company is that they have been able to maintain their outperformance, and we think they have a culture where that is going to continue," Brooks said.

Higher interest rates, meanwhile, should help the companies expand their margins. A 0.25 percent increase in interest rates would allow mutual fund companies to recoup 65 percent of fee waivers, while a 1.0 percent increase in rates would allow firms to eliminate fee waivers entirely, analysts say.

T. Rowe Price waived $12.5 million in fees last quarter to maintain positive yields for investors in money market accounts, an amount roughly equal to what it spent on advertising and marketing. Once interest rates rise high enough, ending fee waivers should boost its revenues by about 1.3 percent, analysts say.

Higher rates would especially help Federated Investors, which has approximately 69 percent of its $354 billion in assets under management in money market funds, said Argus' Biggar. Shares of the company are down 9.8 percent for the year to date and trade at a forward price to earnings ratio of 16.1, according to Thomson Reuters data.