(Bloomberg News) It has been another disappointing year for investors in actively managed funds.

In 2011, about 79 percent of large-cap mutual fund managers trailed the Standard & Poor's 500 Index, according to Morningstar Inc. The average equity mutual fund lost almost 3 percent last year, compared with a 2 percent gain for the S&P 500, says Lipper U.S. Fund Flows. Hedge funds fared even worse, with an average loss of 5 percent, according to Hedge Fund Research Inc.

These weren't aberrations. Numerous studies show that most active funds have underperformed passive benchmarks over time, primarily due to costs and fees. Not surprisingly, most academics and many market professionals are recommending passive investing. Even superstar active managers such as Peter Lynch and Warren Buffett have advised most investors to hold index funds.

Investors have been listening. Exchange-traded funds are gaining popularity. Also, the share of assets invested in equity index funds relative to all equity mutual funds grew to 14.5 percent in 2010, from 5.2 percent in 1996, according to the Investment Company Institute.

Yet these figures indicate that most money remains invested actively. Why is the active management industry so large when its performance has been so poor?

Stupid Investors

Most explanations for this continued dominance suggest that investors aren't very smart. For example, they may be fooled by slick salespeople or are unable to correctly interpret performance data. Or they may believe they can all pick above- average fund managers. There is some truth to all these interpretations.

But investors don't have to be stupid. In a recent paper I wrote with Robert Stambaugh of the Wharton School of the University of Pennsylvania, we argue that the large size of the active management industry is understandable even if investors are smart. Our conclusion is based on the realization that this industry faces decreasing returns to scale: Any fund manager's ability to outperform a passive benchmark decreases as the industry grows.

The idea is simple. As the active industry grows, more money chases opportunities to outperform passive benchmarks, and such opportunities become harder for managers to find. Conversely, if the industry shrinks, less competition among the remaining active managers makes it easier for them to find mispriced securities and outperform.

At the extreme, if investors moved all of their money to index funds, markets would probably be rife with mispricing. With no active managers searching for mispriced assets, the first dollar invested actively would earn a high return by picking low-hanging fruit, tempting investors back into active funds.

For an analogy, think of active managers as police officers and of mispricing as a crime. If there were no officers patrolling the streets, there would probably be some crime. But as the number of officers increases, the amount of crime is likely to decline.

Decreasing returns to scale imply that the industry's alpha, which is the expected benchmark-adjusted return from investing in active funds, isn't constant. Instead, alpha decreases as the industry grows, and it increases as the industry shrinks, where industry size is measured in relative terms as the share of actively managed assets in total assets under management.

Alpha Revision

This inverse relationship between industry size and alpha offers an explanation for the continued popularity of active management. Investors require a small, but positive, alpha to invest in active funds to compensate for the associated risks. They don't know the exact value of alpha, but they learn about it by observing the funds' returns. If these returns turn out to be disappointing, investors revise their beliefs about alpha downward and reduce their investment in active funds.

Importantly, the reduction in active investment in response to underperformance is cushioned by the inverse relationship between industry size and alpha. In such instances, investors know that alpha is too low at the current industry size, but they also know that alpha will go up after they reduce their investment in active funds. As a result, the industry shrinks only modestly after a period of underperformance.

When returns are decreasing with scale, past underperformance doesn't imply future underperformance; it implies only that investors should shift some money from active funds to index funds. Following a period of underperformance, the active industry should shrink in relative terms, but it can remain large.

Stambaugh and I developed a model of active management under decreasing returns to scale and applied it to equity mutual funds. We showed that despite the industry's poor track record since 1962, its current size can be rationalized with decreasing returns to scale.

By contrast, under the more traditional assumption of constant returns to scale, according to which alpha is unrelated to industry size, the industry would have disappeared years ago.

We also simulated the industry's future performance under decreasing returns to scale. We found that the industry is likely to remain large for decades even if it continues to underperform passive benchmarks.

Importantly, the future performance of active managers is likely to be better than their past performance. Because of the rapid growth of indexing, the active industry today accounts for a smaller share of assets under management than ever before. A smaller industry should perform better in the future.

In addition, the growth of indexing itself creates opportunities for active investors. Index funds tracking popular indexes such as the S&P 500 and Russell 2000 tend to rebalance at index reconstitution dates so as to minimize tracking error. This rebalancing creates price pressure, which leads indexers to buy high and sell low. Active managers can profit at the expense of indexers by trading ahead of reconstitutions.

Sooner or later, the effects of smaller scale and index reconstitutions should improve the active managers' performance to attractive levels. But whether that will happen this year or 20 years from now is anybody's guess. Meanwhile, if you want to be on the safe side, stick to index funds.

(Lubos Pastor is professor of finance at the University of Chicago Booth School of Business and a contributor to Business Class. The opinions expressed are his own.)