At the core of modern finance is the claim that risk and return are tightly linked. Higher performance is joined at the hip with higher risk. History suggests that's a pretty good description of general market behavior in the long run, but the relationship can become unstable over a span of weeks or months. In fact, you can count on it.

That's the main glitch with conventional finance theory, and one that's likely to cause trouble for the average investor. It's no picnic for financial advisors and other professionals, either. But it's not all darkness and doom. Sometimes the glitch works in our favor. In 2007, for instance, the equity market's trailing volatility (its standard deviation of return) plunged while performance soared. No one was complaining, though the arrival of one extreme is a reminder that its evil twin may be lurking. For anyone who thought otherwise, the events of late 2008 forced an attitude adjustment.

A disciplined investor who's truly focused on the long run may be able to ignore the short-term noise and wait for the risk-return relationship to right itself. If you're tactically inclined, maybe you can exploit the bouts of instability with clever trading. Alas, both approaches have their limits for most folks. In the long run, we're all dead, as Keynes famously quipped, and talented market timers are a rare breed otherwise.

Fortunately, there's a third choice: diversifying across asset classes. As the standard approach to risk management, asset allocation works just fine-most of the time. But every so often, perhaps more often than generally assumed, the conventional defense is defenseless. The market "is a learning machine that continually errs, corrects itself and makes new errors," writes Kent Osband in Pandora's Risk: Uncertainty at the Core of Finance, an intriguing new book that re-examines assumptions about market behavior and financial risk.

The limits of standard risk management inspire supplementing asset allocation with hedging strategies that offer some protection against periods of "learning." If you're not going to be compensated in a timely manner for suffering higher volatility when the market crashes, there's a case for insuring against this unrewarded risk. Unfortunately, implementing this concept is complicated.

The Risk Of Managing Volatility
One danger is going overboard. Because risk and return are linked, you can only minimize volatility so far before expected return tumbles as well. Another hurdle is keeping a lid on expenses and the opportunity cost that often accompanies hedges. There's no shortage of products for shaving risk, but the price tag isn't always reasonable.

It's easy to curtail the downside risk for an equity market allocation by adding, say, put options or short-strategy ETFs. Another possibility is buying one of several exchange-traded products that track the CBOE Volatility Index, or VIX. It's often called the fear index because it measures the implied volatility of the S&P 500. When stocks crumble, the VIX tends to jump, as does a simple measure of volatility (standard deviation). But the problem with a straight hedge by way of a volatility proxy, or a short-strategy ETF, is the hefty opportunity cost when investors use it as a buy-and-hold strategy.

Consider the history of the VIX versus the S&P 500 (Figure 1). A $100 investment in the U.S stock market for the 20 years through this past May would have grown to roughly $370. The same buy-and-hold investment in the VIX would be more or less unchanged over the last two decades. That's a horrible return for such a long period, but it's not surprising.

Finance theory tells us not to expect a risk premium for holding volatility. Sure, prices of stocks and other asset classes bounce around a lot, but the volatility levels usually end up as a wash in the long run. The price tag is even higher after factoring in the various costs of securitizing volatility. For instance, several exchange-traded notes (ETNs) that track several incarnations of equity market volatility have pricey expense ratios of 0.85% to 1.65%, according to Morningstar Principia. By comparison, you can buy a broad equity market ETF for under 0.1%.