In less than seven days, hedge funds have been subject to a three-pronged attack by some of the biggest names in finance.

Steve Cohen, the billionaire trader whose former hedge fund had racked up average annual returns of 30 percent before pleading guilty to securities fraud three years ago, became the latest critic of the business, saying he’s astounded by the shortage of skilled people in the business.

“Frankly, I’m blown away by the lack of talent,” Cohen said at the Milken Institute Global Conference in Beverly Hills, California, on May 2. “It’s not easy to find great people. We whittle down the funnel to maybe 2 to 4 percent of the candidates we’re interested in. Talent is really thin.”

‘Very Hard’

Cohen’s comments come after billionaire Warren Buffett said over the weekend that large investors should be frustrated with the fees they pay hedge funds, which fail to match the returns of index funds. Daniel Loeb, founder of hedge fund Third Point, said last week that industry performance this year was “ catastrophic” and that funds were in the early stages of a “washout.”

Cohen, who had started his hedge fund SAC Capital Advisors in 1992, said the business has “gotten crowded” with too many managers following similar strategies. Hedge funds seem to think that by hiring skilled people, they can “magically” generate returns, he said.

“It’s very hard to maximize returns and maximize assets,” said Cohen, who runs $11 billion Point72 Asset Management. It’s difficult to balance size with carefully managing an organization and delivering good risk-adjusted returns, he added.

Cohen rarely speaks publicly at industry events. He made the comments on a panel discussion with money managers Cliff Asness and Neil Chriss.

’Cost of Being Excellent’

SAC Capital agreed to return outside money to clients as part of a 2013 agreement with U.S. prosecutors targeting insider trading on Wall Street. The firm was renamed Point72 and now manages Cohen’s fortune. He wasn’t accused of wrongdoing.

Point72 President Doug Haynes said in October that there could be more closures and money pulled out of the hedge fund business as the “cost of being excellent” in the industry keeps rising.

Cohen said at the conference one of his biggest worries last year was that his firm might become the victim of an indiscriminate market selloff as other funds endured troubles and reduced risk. He said his worst fears were realized in February when his firm lost 8 percent. U.S. stocks fell to an almost two-year low that month.

Earlier Monday, hedge funds came under criticism at the conference on the heels of Buffett’s comments at his firm’s annual shareholder meeting on Saturday.

High Pay

Chris Ailman, who runs investments at the $187 billion California State Teachers’ Retirement System, said in a Bloomberg Television interview that the hedge fund industry’s fee model is “broken” and “off the table” for large institutional investors. Chriss, founder of hedge fund Hutchin Hill Capital, said investors will pull out of funds that aren’t giving them returns to justify the fees.

Hedge fund managers are among the highest paid in the finance industry, traditionally charging clients 2 percent of assets as a management fee and taking 20 percent of profits generated. Buffett described the fee structure as “a compensation scheme that is unbelievable” to him.

Since the global financial crisis, some managers have cut fees in exchange for getting larger investments from clients and locking their money up for longer periods. Even so, New York City’s pension fund for civil employees voted last month to end investing in hedge funds, determining that they didn’t perform well enough to justify high fees.

Worst Start

Cohen said he was amazed that investors aren’t more demanding. He said pension plans and endowments tend to follow trends instead of being forward thinking.

The $2.9 trillion hedge fund business is having its worst start to a year in terms of returns and client withdrawals since 2009, when financial markets were reeling from the global financial crisis. Managers including Bill Ackman, John Paulson and Crispin Odey posted declines of at least 15 percent in the first three months in some of their funds.

Loeb said in a quarterly letter that most money managers were “caught offsides at some or multiple points” since August. That month, China’s surprise currency devaluation sent shock waves across markets.

Cohen said most people at his Stamford, Connecticut-based firm are not very good at timing when to invest or exit markets, though they are adept at picking stocks. He said external hires account for 20 percent of headcount at Point72, which prefers to groom analysts and money managers internally.

The Point72 founder said there are parts of the world where there are opportunities to generate more alpha, which are profits above a benchmark index, than in the U.S. His firm has offices in cities including Hong Kong and London.

Cohen, who under a settlement with regulators could manage outside capital again as soon as 2018, said he didn’t see the crowding problem in the hedge fund business easing any time soon.

“This industry has been around in a real way for 25, 30 years and excess profits get competed away in one way or another,” he said. “More people are going to enter the business and drive it down. It’s starting to happen now and will probably continue to happen. That’s a normal industry cycle.”