Bill Gross just got a huge vote of "no confidence" from billionaire investor George Soros.

Soros Fund Management LLC, which Soros chairs, pulled its roughly $500 million from an account run by Gross at Denver-based Janus Capital Group Inc. <JNS.N>, a source familiar with the matter said on Monday. The trade publication Pension & Investments earlier reported on the outflow.

The $500 million had been managed through a private investment vehicle named Quantum Partners LP and invested in a separate account. The portfolio followed the strategy that Gross was employing in his Janus Global Unconstrained Bond fund <JUCAX.O>, which has been struggling with poor performance and outflows.

So far this year, the fund is posting negative returns of 1.47 percent and lagging 74 percent of its peer category, according to Morningstar.

The withdrawal by Soros is particularly significant for Gross as his Janus Global Unconstrained Bond Fund, which has $1.4 billion in assets under management, holds more than $700 million of Gross' personal money.

"Janus Unconstrained has lost money this year as well as underperformed this year versus peers and the Barclays Aggregate index," said Todd Rosenbluth, director of ETF & Mutual Fund Research at S&P Capital IQ. "The fund has struggled to gather assets. For investors who had faith in Gross, including Soros, they have been disappointed."

A Janus spokeswoman declined to comment.

Soros Fund Management's Chief Investment Officer Scott Bessent last year approached Gross after he resigned from Pimco in late September, the source said, adding that the fund had considered Gross a legend. Gross co-founded Pimco in 1971 and built it into one of the largest investment firms in the world, managing $2 trillion of pension, endowment and retirement money.

Bessent, who has been overseeing Soros' $30 billion fortune for the last four years and whom some consider Soros' protege, will leave at the end of 2015 to start his own hedge fund firm. Bessent will continue to advise Soros Fund Management on an informal basis, however.

Soros Fund Management, founded in 1969, was reported to be one of the most profitable firms in the hedge fund industry in 2010, averaging a 20 percent annual rate of return over four decades.