As more financial advisors turn to third-party exchange-traded fund strategists to invest their clients’ assets, and as more ETF strategists hang out their shingle in hopes of attracting these assets, it begs the question: Do advisors really know what they’re getting when they entrust client money with one of these strategists?

ETF strategists are firms that deploy a minimum 50% allocation to ETFs to create asset allocation strategies. These ETF-managed portfolios are typically delivered in separately managed accounts and come in a smorgasbord of different strategies. The space ranges from well-known players—including BlackRock, Vanguard, State Street, Charles Schwab and Invesco—to smaller, lesser-known firms. And these strategists run the gamut in terms of investment philosophy, fee structure, experience and expertise.

Of course, advisors should always do their due diligence when hiring a third-party manager. But given the relative newness of this space and its variegated offerings, some headline hiccups within the strategist world (remember F-Squared?) and the flow of new entrants looking for a slice of the pie, advisors might need to eat their Wheaties to up their vetting game.

“One of the difficult parts of this space is there are no industry standards,” says Brooks Friederich, Envestnet | PMC’s director of fund strategist portfolios, whose company offers an array of ETF strategist portfolios on its investment platform for financial advisors. “We’re well-accustomed to standards based on what we know of the style-box world, but we don’t have those standards in the strategist space.”

Perhaps the first place to go for guidance is Morningstar’s quarterly ETF Managed Portfolios Landscape Report, which classifies this industry by universe, asset breadth, portfolio implementation and primary ETF exposure types. And that’s further broken down into 15 subcategories.

As of September 2016, Morningstar tracked 838 strategies from 157 firms with total assets of $87.6 billion. Total assets were up 4.3% in the third quarter, marking the third consecutive quarter of growth.

But this isn’t a monolithic growth story. According to Morningstar, many managers in the global all-asset and global balanced strategies—which make up more than half of all assets in ETF-managed portfolios—turned in subpar performance that lagged benchmarks and led to outflows. But this was more than offset by inflows at other existing strategies, as well as the addition of new strategies to Morningstar’s database.

Morningstar’s report isn’t meant to be the bible on the ETF strategist space. “It’s a starting point; it’s a frame of reference to compare providers within similar strategies,” says Ben Johnson, Morningstar’s director of global ETF research and author of the quarterly report. “It’s strictly the beginning for a more robust and deeper due diligence process.

“Regarding our report, we provide a landscape overview in an area we wanted to shine a light on because going back three or four years ago it had the characteristics of the Wild West with a lot of gunslinging operators who weren’t household names—one of which proved to be based on a fiction and another one flamed out in spectacular fashion,” Johnson continues.

The “fiction” refers to F-Squared Investments, which in 2014 agreed to pay a $35 million fine to the Securities and Exchange Commission and admit wrongdoing for falsifying performance numbers in advertising and marketing materials for its flagship AlphaSector Premium Index strategy. Before running afoul of regulators, F-Squared was a leading ETF strategist that was popular with financial advisors. The company ultimately filed for Chapter 11 bankruptcy and sold its intellectual property and investment strategies to Broadmeadow Capital.

The “flameout” refers to Good Harbor Financial, whose Tactical Core U.S. portfolio produced solid and, in some years, spectacular net-of-fees returns from 2009 to 2013 before crashing more than 21% in 2014, versus the 13.7% gain that year in its benchmark S&P 500 Total Return Index. The strategy lost 8% the following year versus a 1.38% gain in the S&P 500, but had recovered in 2016 with a 6.59% return as of the third quarter, versus the 7.84% uptick in the S&P 500.

Qualitative Assessment
Larger institutions can provide a template for how advisors should think about evaluating ETF strategists. Before Envestnet puts an ETF strategist on its platform, for example, it does a general qualitative assessment of those firms.

“We first try to understand the story—the history of that firm; why they created its product; the ownership structure; and whether they eat their own cooking,” says Envestnet’s Friederich. “But that’s just a starting point because it doesn’t tell you anything about their investment process and philosophy, risk management or performance. All of these strategists have a unique investment philosophy, and we try to look at why they believe in their respective strategies, and do they have research behind it and is it a core capability at the firm, or is it just a sales and marketing gimmick?”

Envestnet next looks into the investment process of a portfolio strategist, along with its investment team and its tenure. “Here, you’re not dealing with folks picking individual stocks and bonds, but are picking ETFs and doing asset allocations and doing trading and rebalancing while taking more tactical or strategic approaches,” Friederich says. “There are more elements there, and it’s really more of a portfolio solution that they’re managing than a traditional management of just a sleeve of a portfolio.”

If a portfolio strategist passes muster to that point, then Envestnet finally looks at performance. “Does the output we can see align with our understanding of that strategy?” Friederich asks. “If they’re a tactical manager and they’re focusing on protecting on the downside, did they actually achieve that in early 2016 or in 2011?”

Brian McCormick, senior investment research analyst at Commonwealth Financial Network, says his company also takes a qualitative approach when evaluating ETF strategists. 

“The personnel is important—what kind of turnover do they have?” says McCormick, whose firm offers roughly a half-dozen ETF strategists on its investment platform. “How many analysts are on the team? What are their risk controls? How do they generate ideas … are they quantitative? We also look at operations—trading, legal and compliance. We dig pretty deep with these firms to make sure they’re something we want to offer to our advisors.”

McCormick notes that those ETF strategists on Commonwealth’s platform have mostly delivered on their stated mission. “It’s sometimes difficult to gauge them, especially the more tactical ones because what benchmark do you use for these to gauge relative performance?” he says. “So you need to monitor them on an ongoing basis to make sure they’re doing what they say they do.

“One of the things we want to do is set expectations for advisors, so when they ask about these strategists we want to make sure they can anticipate what to expect regarding performance,” McCormick adds.

At What Cost?
Todd Westby, CEO and founder of T Hayes Consulting LLC in Villanova, Pa., which focuses on ETFs, says that advisors need to recognize that not all ETF strategists’ value will be reflected in past performance.

“It may be their portfolio positioning reflects a strategy that has not played out yet,” Westby says. “Superior support and communications by the ETF strategist may offset some after-fee underperformance figures.”

And this being ETFs, where cost efficiencies have helped propel the exponential growth of this class of investment vehicles, advisors need to consider the fee structure of ETF strategists. “Costs matter, and is probably the greatest single predictor of future performance relative to peers in a specific category,” says Morningstar’s Ben Johnson.