Few mainstream software models employed by financial advisors provided any clues in early 2008 about the devastation that awaited the financial markets. So it's not surprising that over the last several months, a great deal of criticism has been leveled at Monte Carlo simulations and their use as a financial planning tool. An article in the May 2, 2009 issue of The Wall Street Journal entitled "Odds on Imperfection: Monte Carlo Simulation" is fairly representative of the criticisms that have been leveled against MCS recently. It calls into question Monte Carlo's ability to quantify various risks, such as the risk of a portfolio being depleted within a certain period of time.
If the critics are right, and if Monte Carlo simulations are so imperfect as to be of little use, the implications for both consumers and professional financial planning software are extremely serious, since many of the most widely used financial planning tools include an MCS component. The Journal was quick to highlight this fact in the article when it stated:
"If one had asked a financial adviser 18 months ago for retirement-planning guidance, there is a good chance he would have run a 'Monte Carlo' simulation. This calculation method, as it is commonly used in financial planning, estimates the odds of reaching retirement financial goals."
So are the critics right? Is the Monte Carlo simulation faulty, or is it still a valid tool that advisors can use with confidence? The answer requires some explanation, but there's no need to get very technical. The intricacies of better models can be challenging, but the basics of MCS can be understood without an advanced degree in rocket science. Once we've addressed the validity of MCS as a planning tool, we'll look briefly at the question of what else can be done to improve on the advice we give to clients and then discuss the implications for the planning profession.
What Is A Monte Carlo Simulation?
According to Wikipedia: "Monte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used when simulating physical and mathematical systems. Because of their reliance on repeated computation and random or pseudo-random numbers, Monte Carlo methods are most suited to calculation by a computer. Monte Carlo methods tend to be used when it is unfeasible or impossible to compute an exact result with a deterministic algorithm."
The conceptual framework surrounding Monte Carlo simulations has been around for a long time. The term "Monte Carlo method" was coined in the 1940s by physicists working on nuclear weapon projects in the Los Alamos National Laboratory. Wikipedia states: "The name 'Monte Carlo' was popularized by physics researchers Stanislaw Ulam, Enrico Fermi, John von Neumann and Nicholas Metropolis, among others; the name is a reference to the Monte Carlo Casino in Monaco where Ulam's uncle would borrow money to gamble. The use of randomness and the repetitive nature of the process are analogous to the activities conducted at a casino."
In order to evaluate the value of Monte Carlo simulations as they are being used today for personal financial planning, there are at least three factors one must consider: the quality of the model being employed, the quality of the inputs and the way the results are conveyed to clients.
The Quality Of The Models
"To say that all Monte Carlo models are good or bad is not just inaccurate; it's crazy," says Thomas Idzorek, CFA, the chief investment officer and director of research and product development at Ibbotson Associates (a Morningstar company). "Not all Monte Carlo is equal."
Some Monte Carlo engines built for personal financial planning applications are quite robust and sophisticated, others are much less so. For example, some engines are tax aware, while others take into account no tax consequences whatsoever. Since most planners may not have the in-house expertise to evaluate the underlying mathematical model, they may have to rely on a third party to evaluate the validity of the model (Morningstar/Ibbotson is one of a number of firms that have extensive expertise in this area). At the very least, users should understand the factors that the model is attempting to account for when running simulations and also understand the general theoretical framework underlying a specific model. Any reputable company offering Monte Carlo engines should be able to supply the advisor with such documentation.
The Quality Of The Inputs And Assumptions
Much of the recent criticism leveled at Monte Carlo simulations lately seems to be targeting the inputs and assumptions built into many of the Monte Carlo tools being used for personal financial planning. There's been much written about normal versus log normal distributions, fat tails and other terms that can be intimidating to the uninitiated. What it all really boils down to is the question of whether a Monte Carlo model accurately depicts the likelihood and the frequency of one really bad year showing up in a series of Monte Carlo iterations (or two bad years in a row). Let's look at a few problems that can occur with inputs and assumptions. This is by no means a comprehensive list, but rather a sample to illustrate the point.