According to a new survey from the CFA Institute, Wall Street is getting a bit worried about fintech replacing its jobs.

The majority of respondents, which included responses from about 775 chartered financial analysts around the world, view asset management as the industry most at risk from disruption by financial technology. Fifty-four percent of respondents said the sector would feel the biggest changes, followed by banking, securities, and insurance.

Robo-advisers, a low cost alternative to traditional financial advice, has garnered headlines recently as fees have come under increased scrutiny. According to remarks this week by Wall Street executives attending the Milken Institute Global Conference, the entire world of finance should fear job replacement.

However, not all consumer wealth brackets will be equally susceptible to change. The survey showed a wide range as to which groups will see the biggest benefits from automated advice. The mass affluent market overwhelmingly sees the most positive impact, derived from lower cost and increased access to investing guidance. However, survey respondents foresaw little in the way of benefits flowing to ultra-high earners.

This led the respondents to believe that automated financial tools are extremely unlikely to replace humans when it comes to the ultra-high net worth category, but the mass affluent is another story. The survey showed that 34 percent believe automated advice could entirely replace human advisers for that segment.

Still, a number of risks associated with robo-advisers had respondents issuing words of caution: Forty-six percent said flaws in the algorithms for which robos are known could prove to be an issue; 30 percent worried about getting bad financial advice; and 12 percent cited privacy as the biggest issue.

The respondents didn't view the fintech threat as a flash in the pan. They viewed robos as having a continued, large impact on the industry in five years, while they were more skeptical about the ability of crowdfunding and marketplace lending to last over the long run.

Survey respondents said it is unlikely that automated financial tools will replace engagement with human advisors for institutional investors and ultra-high-net-worth individuals. The implication is that the tailored nature of financial advice to these market segments is not amenable to standardized automation tools typically provided by robo-advisors, the CFA Institute said, adding that these groups of investors, with large portfolios and potentially diverse and complex investment needs, are likely to continue to favor personalized, human advice.

At the same time, the survey found that 89 percent of respondents think robo-advice tools will positively affect consumers’ investment costs. However, 47 percent think the quality of service could be negatively affected.

Staff Writer Karen DeMasters contributed to this article.