There's no denying that it is tough for traditional money managers to beat their respective benchmarks.  In fact, most active managers cannot outperform their benchmark after taking expenses into account.  So if you are going to invest with active managers, you'd better get it right, by investing in the right people who can take the time to adhere to a solid due diligence process.

The most commonly used approaches for streamlining fund manager selection include:

1) Relying on past performance

2) Relying on the advice of third party ranking services

3) Using a checklist to weed out managers with potential problems

This article discusses the problems inherent in solely following these approaches.  We will illustrate how, despite the availability of more information than ever, and the advent of third party ranking services and other short cuts, choosing a truly exceptional investment manager comes down to experience, judgment and knowing what to do with all that information.

Popular Approach #1:  Rely on exceptional past performance

This is the easiest, most straightforward approach to selecting a money manager: select the manager with exceptional past performance over a certain time period.  Indeed, the beauty of this method is its simplicity; it's easy to sell the investment to clients and requires limited investment of time and resources.

So what's wrong with this method?  Looking simply at past performance, one must make a number of assumptions that oftentimes break down after further analysis. First and foremost, it assumes that outperformance will continue into the future.  There's a reason for the fine print listed on any mutual fund prospectus:  "Past performance is no guarantee of future results."  Persistence is in no way guaranteed, especially if we don't understand how the manager outperformed.

Today's winners are often tomorrow's losers

Some even argue that outperformance is oftentimes mean-reverting, and that many managers who have outperformed over one period tend to trail over the next.  From what we have seen, this does often happen.  For example, when we looked at the universe of U.S. large cap value managers for the five years ended June 30, 2009, we saw that a significant number of the "outperformers" followed certain strategies.  They either held cash at the right time (a market-timing move that is incredibly difficult to do consistently), invested in smaller cap stocks (small cap stocks indeed outperformed large cap stocks over that five year period), or invested in high dividend-paying stocks (which held up better than the broad market, particularly during the 2008 financial crisis).

When we ran that screen again for the five year period ended just over a year later, we saw a completely different list of winners, many of whom did not hold up well during the crisis of 2008 but who posted exceptional returns during the post crisis period.  Many of the winners from the screen a year earlier were nowhere near the top any longer.  By picking the best performing managers after the first screen, we could have left a lot of money on the table.

Of course, past performance considerations always play a role in manager selection, because at the end of the day we are looking for managers who have been successful.  If they weren't successful in the past, why would we expect them to be successful in the future?  The point here is that a manager's past performance should be viewed in context with their strategy and process, their level of risk, and the market conditions during the timeframe in which performance is viewed.

Popular Approach #2: Rely on outside opinion

A great business developed years ago: third party evaluation and ranking of mutual funds.   If you understand the limitations of picking managers yourself, it's comforting to let the experts guide you.  Similar to looking at past performance numbers, relying on third party rankings is an easy, straightforward and relatively resource-light approach.  Some third party evaluators do take other factors into account besides performance, making them more valuable but oftentimes more expensive as well.

So what's wrong with solely relying on this method?  Picking managers based on third party rankings might still rely heavily on past performance.  Morningstar is perhaps one of the most widely used third party ranking services for mutual funds.  They categorize managers into sub-groups based on the types of stocks and bonds the manager buys.  They then compile and rank managers by their risk-adjusted returns over multiple time periods.  (Risk in this case is defined by volatility of returns - standard deviation.)

While risk-adjusted performance is a step up from just looking at raw performance numbers, it still just relies on those numbers.  It is faulty to assume that performance will persist into the future.

Strong ranking can actually backfire for high-flying managers

There can also be a dangerous outcome when relying solely on third party rankings.  The managers with the highest rankings typically attract assets from other followers of the ranking services.  Oftentimes a manager sees significant new business after receiving third-party accolades.  This new business, most of which probably would not have existed without the performance-based notoriety, can quickly become lost business if the manager's great performance isn't replicated and the rankings go down.

Followers of the ranking methodology dump the once high flying manager for the next high flyer. The potential rapid decline in assets caused by fickle investors can negatively impact the fund's performance due to the many expenses associated with selling stocks.  In addition, if the manager built up their resources to accommodate all the new business, losing that business may require them to reverse course and cut resources, including staff - the lifeblood of an investment management organization.  We have seen this happen and the impact can be destructive to investment performance.

Moving in and out of funds often means: sell low, buy high

Even Morningstar has come out cautioning investors on the limitations of relying purely on their ranking metrics.  While they have not changed their ranking system in any way, we appreciate this acknowledgement.  Given the confidence many investors have placed on third party metrics, it's no wonder why the average mutual fund investor has drastically underperformed the broad market.  It's not because the mutual funds are drastically underperforming (although some do).  It's that investors move in and out of funds, selling funds after they have hit bumps in the road in favor of funds that just completed a great performance period.  The unfortunate result:  they sell low and buy high.

Indeed some third party ranking services and data providers rely on more than past performance when ranking a manager.  The point, however, is that savvy investors need to understand the limitations of the ranking systems.

Popular approach #3:  Manager due diligence via a checklist

A step beyond relying on performance data alone is the development of a set of "check-the-box" items that help investors answer the question, "How do I know if this manager with great past performance can still perform well in the future?"  By building a checklist that answers "yes" or "no" to basic questions like, "Have there been any changes to the team managing the strategy within the past five years?" you can consider some important facts that go beyond the numbers. The answers can come either from third party databases, or from speaking with the investment management team.

So what's wrong with relying solely on this approach?  Having a number of checklist items that go beyond simple performance measures is certainly a good practice; the more we know about a manager, their strategy, their process and their organization, the better.  Reliance on straightforward yes-or-no answers can, however, lead to a false sense of security.

For example, we once asked a prospective manager if there had been any recent team changes, and he responded "no."  "No" is generally a good response because there is no way to know that new team can work together and replicate the prior team's past success.  In this instance, however, while there were no changes to the people on the team, the portfolio manager had recently launched three new products, thus drastically reducing his time and focus on the strategy we were considering.  So the "no" answer wasn't good after all.  A simplistic "check the box" system would have led us to a manager who was spending a lot less time on the strategy than he had in the past, minimizing the chance that past performance would persist.

Some managers just tell you what you want to hear

Another problem with this method is that sometimes managers are not as truthful as we would like; in that case, a checklist is simply a waste of time.  We recall asking another manager the same question, and he claimed that no changes had occurred on the money management team.  But when we looked at their organizational chart, we noticed that one of the key members of the team was missing.  When questioned, he admitted that the key member had been terminated, but then claimed the individual was never an integral part of the team - so we shouldn't worry about it.

Let's look at our "team member change" question a different way.  Let's assume that the firm had recently brought in a new team from another firm to replace the old team with the strong track record.  This "yes" response would, to many check-the-boxers, be an automatic deal-breaker, based on the assumption that any change is bad.

We had one experience where a team change was for the better, because the new team had a far superior performance track record and investment process.  By simply just checking the box and moving on, we would have overlooked a potentially great investment opportunity.

While the above examples might seem intuitive and straightforward, the point is that many questions require additional follow-up; what appears on the surface can look very different and therefore lead to a different conclusion once all the stones are turned over.

Go beyond a checklist and become a detective!

So what is a savvy investor to do if the more simplistic ways of picking managers are riddled with shortcomings?  Picking an investment manager is no different than making any major decision in life.  You need to fully understand the people managing the money, the firm those people work for, and why those people were able to put up such great returns.  This can only be done by becoming a detective-gathering as much evidence as possible, determining which evidence is important and which is not, questioning the suspects on the pertinent evidence, and coming to a verdict only after a careful evaluation of the facts. This requires a lot of time.

The popular short-cut approaches to choosing investment managers came about in large part because investors do not have the time to do the detective work.  Also, they may not have adequate access to the people who manage the money.  Without a significant amount of money to potentially deliver to these managers, or if the manager is not convinced that you intend to be a long term partner, they may not not grant you time the time you need for a full evaluation.

So what does a thorough investigation look like?

To illustrate what we think it means to be a detective, let's briefly walk through a manager search Federal Street recently conducted.  We'll show how we determined what information was pertinent, and comment on how more simplistic methods of manager selection would have led us to a potentially different solution.

In early 2009, still in the throes of the financial crisis, we were looking for a small cap equity manager.  The performance track records we were reviewing included the rocky second half of 2008.  We initially compiled a focus list of managers, based on past performance over a three and five year period.  Given the poor market performance in 2008, however, we realized that we had to be careful when looking at this measure.

We were concerned that we might be missing a great manager who may have struggled in 2008 and whose shorter term numbers were not telling the whole story.  We therefore looked out at the longer time periods, to see if there were managers who had great long term track records but whose three and five year returns were impacted tremendously by 2008.

We found a manager who had experienced a difficult 2008 but had an exceptional longer term track record.  Before meeting with the manager, we asked for a list of his current and past stock holdings, and did some detective work on our own.  What we found was that many of the stocks he owned, especially those issued by the smallest of small cap companies, sold off dramatically as investors dumped their "risky" assets.  In fact, at the time we were looking at the manager, we noticed that many of the companies he owned were trading at prices below the companies' net cash on the balance sheet.  It was like buying a piggy bank for a dollar when there was clearly $1.40 in change inside!  So not only were we able to isolate the underperformance to the short term period, we also saw that we had an opportunity to buy into these great companies at an excellent time through this fund.  If we had simply relied on their three or five average annual year performance numbers (which were so impacted by the 2008 performance), we would have walked away from this exceptional manager.

Another "check the box" fact could have made us walk away from the manager.  The database we used for information about the manager indicated that there had been a manager transition in the past few years, and that the person who started the firm and developed the original process had been replaced.  For the check-the-boxers, this would have been another deal-breaker.  Upon questioning the management team, however, we found that the proprietor had actually stepped back almost a decade before.  He had stayed on the strategy as the "named" portfolio manager, purely in an advisory role.  This meant that the current portfolio manager had already been in charge of day-to-day portfolio management and stock selection responsibilities for ten years.  Through in-depth discussion, we became comfortable that the track record we were looking at was in fact that of the current portfolio manager.   All told, we found a truly exceptional manager that at first glance might not have looked promising.  Not surprisingly, this manager went on to deliver a 53% return to our clients in 2009, when the average small cap manager was up 32%.

Once you got 'em, when do you let 'em go?

The same popular shortcuts for picking a manager can be used to decide whether or not to keep a manager you are already using.  Wouldn't you rather become a detective and uncover issues before they translate into a performance problem?

Wouldn't you like to have a crystal ball that shows you if a manager's underperformance will reverse or persist?  While we don't proclaim to predict the future, we believe from experience that a robust investigation process can differentiate between a manager's short term performance problem and one that could persist for longer.

Even exceptional managers underperform sometimes

Underperformance is never fun, and is certainly painful - especially when prolonged for more than a year.  It is natural to assume something is very wrong when a manager underperforms, and the easiest response is termination.  When we terminate a manager, we look responsive, and it reassures clients that we are doing "something."  More importantly, we could be getting out of the way of a complete train wreck.

Termination is surely the safest thing to do, but is it what's best for the client?  What if that manager is ready to have explosive outperformance, better than any other option we could put in its place?  By putting on our detective hat and by understanding the drivers of not just the portfolio's performance but overall market conditions, we can better understand what's going on.  Occasional underperformance is a very common occurrence and most exceptional managers have indeed stumbled from time to time relative to their respective benchmarks, only to come back and regain their exceptional performance.

During tough periods, we must ask ourselves, "Is the manager's performance within our range of expectations?"

Do we think this manager can outperform going forward?  Again, let's look at a real-life example.  During the 2005 through 2007 period, one of the international equity managers we recommended trailed the benchmark significantly during a rising market.  The manager invests in high quality, financially strong companies and because she is so discriminating in the types of companies she buys, she did not hold many of the stocks that were increasing significantly in price.

In our evaluation of her performance, we looked at a number of factors. The manager's long track record revealed that her recent performance, while painful, was in fact within expectations.  Next, we looked at how she picked the stocks that were going into her portfolio; had there been any changes that might have caused her underperformance?  The short answer was "no." She had always invested in high quality companies, and most of them were left behind during the 2005-2007 market rally.

Additionally, the manager is very valuation conscious, and she had never paid a high price for the prospect of growth in a stock.  Consequently, the portfolio was not invested in the lower quality, higher priced areas of the market that were strongest during that particular period.   This made sense to us.

We liked this manager because she focused on high quality companies and we believed that those types of companies can not only do well over long periods of time, but should hold up well in a market decline.  Sure enough, when the market declined in the latter half of 2008, her portfolio lost half as much as the broad market, giving our clients the exceptional downside protection we had expected.  If we had used past performance as the sole metric in deciding to keep her, we would have replaced her with a manager who had performed well during the 2005-2007 period.  Our clients would have then lost almost 50% with the market in 2008.

By gathering the facts, understanding the reasons behind her performance, and reaffirming our understanding of the strategy and investment thesis, we were able to continue recommending the strategy to clients.  They were ultimately rewarded with superb downside protection when they needed it most.

There is no reliable short-cut

At Federal Street Advisors, we believe that there is no easy formula for successfully choosing investment managers.  Sure, there are plenty of short cuts, but not only can they lead to sub-par results; they can actually result in extreme underperformance. It is no wonder why so many advisors and individuals have gone the route of Index funds.  Investment manager due diligence takes a lot of time, focus and a rigorous level of judgment and understanding that only come from experience and dedication.  If you are unable to invest the time and resources necessary to become a true due diligence detective, you might as well Index.

Andrew Shepard is a research analyst with Federal Street Advisors.