What do you tell your financial advisor when you are leaving for an algorithm?

Joe O'Connor, a 52-year-old Connecticut salesman, had to have this conversation recently. It was delicate business explaining why he was ditching the planner he had been with for over a decade, to put his money in the hands of what is known as a robo-advisor - a web-based service that automates the allocation of your investment portfolio.

There were the usual responses: "But why?" "Was it something I did?" "What can I do to make it right?"

And of course, there is the timeless relationship classic line: "It's not you, it's me."

"It wasn't fun," O'Connor said.

That kind of awkward conversation is taking place more frequently these days, thanks to the rise of robo-advisors, which have about $20.1 billion in worldwide assets under management for new entrants, according to Switzerland-based research firm MyPrivateBanking.

Of course with total U.S. investable assets at $33.5 trillion, that is barely loose change under the couch cushions, industry analyst Michael Kitces points out. But projections are for heady growth with new robo-advisors expected to grow to $42.6 billion in 2016 and $86.7 billion in 2017.

Why Leave?

Looking strictly at fees, robo-advisors offer certain advantages. Prominent site Betterment (http://betterment.com), for instance, charges .25 percent on accounts between $10,000-$100,000, and .15 percent above that. Competitor Wealthfront (http://wealthfront.com) has a similar cost structure, charging .25 percent for accounts worth $10,000 or more.

Personal Capital (http://personalcapital.com), which Joe O'Connor uses, offers more of a blended service, combining its automated recommendations with humans (albeit primarily via video chat or email), charging .89 percent on portfolios up to $1 million.