We live in the age of the quant. Mathematics drives an increasingly large number of investment portfolios. In many ways, this is an improvement over the less evidence-based approaches that often relied on instinct and gut feeling. But as we learned during the financial crisis, models are imperfect and they can and do break down.
There are many reasons for this: noisy data series are subject to revision. Sometimes the assumptions underlying models are wrong. Occasionally, time simply moves forward, and events occur that are unanticipated by a model’s designer.
I was reminded of the dangers of using models to predict future outcomes this week by an article in the New York Times Sunday Business section. A column by Jeff Sommer discussed the electoral-prediction model of Ray Fair, an economics professor at Yale. It is a stark illustration of how and why models can fall apart, though in this case the model deals with politics rather than financial markets.
For investors who may rely on models to deploy their capital, there are lessons to be found.
Fair’s presidential forecasting model, which he developed in 1978, has an OK track long-term track record -- it was only wrong three times in the past century, based on recent forecasts and back tests. But two of those errors were fairly recent. It forecast a Republican win for the White House in 1992, when Democrat Bill Clinton defeated Republican George H.W. Bush. The Fair model also failed its most recent test: it predicted that Republican Mitt Romney would defeat Democrat Barack Obama in 2012.
Based on the nine presidential elections since the model's invention, that's a miss rate of 22.2 percent. For an all-or-nothing prediction, this isn't really a very useful record.
Here’s the money quote from the Sommer article:
Consider the limits of the model itself. It has worked reasonably well in most elections since 1980, and it performs well in back-tested analyses of elections since 1916, but it is far from infallible, even in achieving its goal: a prediction of the popular vote for the two main parties. If there is a third candidate (or a fourth) in the general election, the model doesn’t acknowledge the candidate’s existence, in effect assuming that the two traditional parties are affected equally. That assumption may not be correct.
It probably was incorrect in the 1992 election, a terrible one for the model, which predicted that President George H. W. Bush would be re-elected. Ross Perot received 19 percent of the vote, probably hurting President Bush in ways not captured by the model, Professor Fair says. That was also the election in which the campaign of the victor, Bill Clinton, relied on the mantra, “It’s the economy, stupid,” which could also be the slogan for Professor Fair’s model. He tweaked the model after that election, emphasizing economic factors even more. That was the last time he altered the model.
As I noted before, it also got the latest presidential election wrong, even though Fair says he adjusted the model to give greater weight to economic concerns -- of which there were plenty in 2012.