It may be the CEO's favorite business, but Wells Fargo & Co.'s wealth management unit is not growing.

Choppy markets in the fourth quarter cut revenue 5 percent to $3.51 billion at the San Francisco-based bank's top fee-generating business. The unit offers mutual funds and other investment products, and serves a range of clients from those with more than $50 million in assets served by its Abbot Downing brand to mass market consumers.

In the brutal revenue environment since the financial crisis, Wells Fargo and other banks have turned increasingly to wealth management to boost returns.

But because of competitive pressures, tougher regulations and ever shakier markets, that strategy has produced less-than-sterling numbers at Wells.

Rough Sledding

Barclays analyst Jason Goldberg has estimated that Wells rivals Morgan Stanley and Bank of America Corp.'s Merrill Lynch, also will post wealth management revenue declines when they report quarterly results on Tuesday.

Rafferty Capital Markets analyst Dick Bove said he expects wealth management to be "one of the businesses that'll do most poorly over the next couple of years" at banks.

Wells CEO John Stumpf does not share that view. At a December conference hosted by Goldman Sachs, he called the wealth investment management division the bank's "biggest opportunity."

Stumpf said Wells Fargo had room to expand its wealth business, noting that the bank has about 11 percent of U.S. deposits, but just 1 to 2 percent of the wealth.

"It's not like the business is not great; it just could be a whole lot bigger," Stumpf said.

Wells Fargo's ability to sell several products to the same customer is one reason investors value it more generously than banks like J.P. Morgan Chase & Co, Citigroup Inc. and Goldman Sachs Group Inc. But investors are typically reluctant to part with their wealth advisors.

"A lot of the clients don't view it as going with the brand: they trust their individual advisor," said Michael Mattioli, portfolio manager at Manulife Asset Management who owns Wells Fargo shares on behalf of clients.

Recruiting Battle

CFO John Shrewsberry said Wells Fargo is better than peers at selling wealth management products to retail customers.

"We have been recording more than $1 billion per month of closed investment referrals out of the community bank into the wealth and financial advisory business for a couple of years now," he said in an interview on Friday.

Wells is not relying on internal referrals alone. It struck a deal in October with Credit Suisse Group AG, which is winding down its U.S. private banking business, hoping to gain a leg up on rivals in recruiting financial advisers.

However, rivals like UBS and Morgan Stanley have lured away nearly a third of the 270 Credit Suisse brokers despite Wells' effort to offer generous incentive packages, according to press reports.

On a call with analysts Friday, Shrewsberry said the Credit Suisse brokers Wells Fargo has recruited will not be "a huge game-changer in terms of the numbers." He said it equated to between three and five months of regular recruitment. A Wells Fargo spokesman declined to say how many advisors it recruited.

Competition from rival banks is getting tougher. Morgan Stanley has retooled under CEO James Gorman, shrinking its securities division to emphasize wealth management.

The business has also grown more prominent at other banks including Bank of America's Merrill Lynch, UBS Group AG, Bank of New York Mellon Corp and Goldman. The threat from smaller firms has also grown as technology has made it cheaper for advisers to gain access to clearing, marketing and compliance and investment products.

Tighter Regulation

Tighter regulation is another issue. A proposed conflict-of-interest rule from the U.S. Labor Department would make it much harder for firms to sell proprietary investment products to clients. A Morningstar report estimates the rule will cost the financial sector at least $2.4 billion annually. The Department of Labor countered it would cost $2.4 to $5.7 billion over 10 years, while saving $17 billion annually for retirees.

One advantage for Wells is that it has controlled costs better than rivals have. Bank of America and Morgan Stanley locked up advisors with multiyear guarantees following the financial crisis, while Wells mostly steered clear of those deals.

Shrewsberry said if markets do not recover and Wells Fargo does not gather new assets, the impact on annual net income would be in the hundreds of millions of dollars. He compared that to the $23 billion the bank earned in 2015.

"It's -- in the scheme of things -- a manageable outcome," he said.