One of the big unanswered questions in the finance world is: Do returns reflect risk or mispricing? Defenders of the efficient markets hypothesis say that you can’t get higher returns without taking more risk, while behavioral finance says that there are often unexploited anomalies that will let wise, patient, or deep-pocketed investors beat the market without taking on more risk.

This debate is very relevant to the use of factor models. These models, which are used to design investment portfolios with specific characteristics, have been one of the most successful methods to come out of academic finance in the past 40 years. Most financial institutions, and all of the sophisticated ones, now use factor models to measure their risk. Many also use them to optimize their returns. For example, so-called smart beta investing strategies, one of the most popular investing fads of the past decade, are mostly just the application of factor models. You can also now buy exchange-traded funds that take advantage of a wide array of factors.

Factor models basically just say that any return that a diversified portfolio earns in excess of the risk-free rate (the rate on Treasury bills) should be based on its correlation with a risk factor. For example, small stocks outperform the market on average, but occasionally they do much worse. If you invest in a bunch of small stocks, according to the model, you will beat the market on average, but you will also be taking more risk, since you’ll lose money during those times that small stocks underperform. Advocates of factor models claim that there are deep economic forces at work that make small stocks, as a group, inherently more risky, and thus justify their higher average returns. Someday, they say, we will understand what those deep forces are.

But what if there is no deep underlying force making small stocks -- or value stocks, or momentum stocks -- more risky? What if these stocks earned better-than-average returns in the past not because they are riskier, but because investors just overlooked them, or were biased against them?

If this is the case, we should expect that publicizing these factors will lead them to shrink in importance. As more people become aware of a mispricing, they trade on it, and the mispricing goes away. So one way to solve the puzzle of factor models is to look at how factors perform after you make them public.

A new research paper by R. David McLean and Jeffrey Pontiff, forthcoming in the Journal of Finance, focuses on exactly this. McLean and Pontiff look at 97 different factors that finance researchers have written papers about. They find that after the papers are published, the excess returns associated with these factors go down by about 58 percent!

One reason factors might lose their luster after being trumpeted to the public actually involves data mining, rather than any market inefficiency. Finance researchers sift through data to find factors, and they often turn up spurious ones. A fake factor will tend to disappear shortly after it is “discovered,” because it was only identified by accident in the first place.

McLean and Pontiff take this possibility into account. They compare how much the factors decline after the discovery to how much they decline after publication in an academic study. What they find is that although factors lose about 26 percent of their strength after discovery -- an effect that includes the results of data mining -- they take another 32 percent dive after they appear in academic papers. This means that a large part of factor disappearance isn't due to data mining alone, but to the publicity that academic studies provide.

This is a dramatic and very important finding. It means, in a nutshell, that markets aren't nearly as efficient as many would like to believe. If factor models reflect a risk-return tradeoff, they shouldn’t be affected by the activities of academics. Deep-rooted economic risks shouldn't vanish just because economists discover them. Mispricings, however, do vanish when discovered.

In other words, this paper is a dramatic confirmation of the predictions of behavioral finance.