Think A Smart Beta Index Sounds Too Good To Be True? You Could Be Right

Beware of the backtests, especially in smart beta indexes.

In recently reported analysis, Newport Beach, Calif.-based Research Affiliates found that smart beta index backtesting tends to find superior performance relative to a market capitalization-weighted benchmkart index, but after an index is officially launched, its performance relative to the S&P 500 hovers around the baseline, producing none of the outperformance in its backtest.

Smart beta index outperformance seems especially strong over the three-year period ahead of an index’s official launch, peaking on average six months ahead of the launch date, according to the report.

In five-year backtests, smart beta indexes in the report averaged 2.8 percent annualized excess return, with only 12 of 125 indexes producing negative alpha. After the indexes go live, however, the average annualized outperformance dropped to 0.7 percent over a five-year period and 0.5 percent over a 10-year period, and the number of indexes producing negative alpha nearly triples.

The gap between live performance and backtesting can be explained by data-snooping, which is mining investment research for signals that appeared to previously forecast performance, and the practice of ignoring transaction costs in backtests. Research Affiliates notes that backtests producing negative results or underperformance are rarely published, thus managers feel pressured to present backtests showing the performance of their index or product in the best possible light.

Research Affiliates found that around two-thirds of the track records for smart beta indexes consist of backtests, and that most live track records extend for a decade or less. On average, smart beta indexes have seven years of live history and offer 21 years of total history for investors to review.

Funds that did have significant live track records often struggle to gain assets for months or years, which means their reported performance may not be representative of what investors should expect in the future.

The S&P High Yield Dividend Aristocrats index performed the best among smart beta indexes in its five-year backtest posting annualized returns of 14.5 percent. However, after it went live in November 2005, it posted five-year annualized returns of just 1.7 percent, and 10-year annualized returns of 0.4 percent.

On the other hand, the Morningstar U.S. Mid Growth index was the worst of the smart beta indexes in backtest performance, posting five-year annualized returns of negative 5.1 percent. Yet after the index launched, it posted five-year annualized returns of 5.7 percent and 10-year annualized returns of 3.2 percent.

The findings appear to support a previous look at backtesting by Research Affiliates which concluded that ETF providers were trend-chasing index performance in creating products. The new research shows that providers also tend to trend-chase backtested results in creating smart beta indexes.