If a recent report authored by Arthur Laffer, Stephen Moore and Jonathan Williams is accurate, New York and California will lose a combined 800,000 residents over the next three years. The trio also predict that other high-tax states like Connecticut, New Jersey and Minnesota will see 500,000 of their residents move elsewhere.

Many will be high-income citizens, driven to relocate by the recent tax law capping the limit on their state and local tax deductions to $10,000 per family, they argue. In the past, high earners in states where the income tax was in the 8 to 10 percent area were able to reduce the effective tax by one-third—or export the tax burden to other states, as Laffer and Moore put it—when they deducted those taxes from their federal returns.

Others see it very differently. From the vantage point of many, including the late New York Senator Daniel Patrick Moynihan, blue states like New York and California send $1.40 or $1.50 to Washington for every dollar they receive back from the federal government. But taken a step further, it makes sense for all taxpayers to locate many government operations like military bases in states where costs are lower.

Wherever you live, however, Laffer and Moore, who penned an article in today's Wall Street Journal, have a point. As a New Jersey resident, I personally know quite a few people talking about leaving the state, though many are not high-income individuals. Factors like job opportunities are more important for most Americans than state income and local property tax rates.

What may be a better indicator of how affluent residents of New York and New Jersey are thinking is the direction of real estate prices. Most arrows are pointed down across a vast swath of upper middle-class properties in the greater NYC-Tri-state area.

In an interview earlier this month, market strategist and economist Ed Yardeni said he would not be surprised if New York, Connecticut and New Jersey had their own rolling recessions, largely due to the reasons cited by Laffer and Moore. If 800,000 high-income really moved out of a handful of expensive, high-tax states, they probably would see economic downturns.

The folks Laffer and Moore are obsessed with are earning more than $1 million annually. Small in number, they provide a disproportionate share of their states’ tax revenues—it has been estimated that if the 5,000 highest-earning Californians packed their bags, the Golden state millionaire's tax woul turn revenue-negative.

However, many of these people have jobs and businesses that their incomes may be chained to. Giant businesses like Pimco and Google (where the median income is $200,000) may be transferring certain operations to enclaves like Austin, Texas and Boulder, Colo., but they still maintain headquarters in California.

What comes around goes around and there may be other factors at play in national migration patterns. New York, California and other urbanized areas have enjoyed strong rebounds since 2009, during what has otherwise been a slow expansion, particularly in the Midwest and numerous rural areas.

Surging living expenses, especially affordable housing costs, are now driving young families in relatively low tax brackets out of places like New York and San Francisco overpopulated by wealthy foreigners and venturists seeking to hit the big time. That is a real problem in its own right, but a very different one than the purported exodus of million-dollar earners.

Some observers, like CNBC’s wealth guru Robert Frank, point to research indicating that Laffer and Moore are dead wrong. He cites a study on millionaire migration by Cristobal Young and Charles Varner of Stanford University who call the Laffer-Moore prediction “pure nonsense.”

Frank notes that while 3.5 million people have moved from high-tax to low-tax states during the last decade, only a small fraction of these people were high earners. Low earners are far more likely to move, either for a job or because “the rent is too damn high.”

He adds that Phoenix Marketing research has found that since 2010, the ranks of millionaires in New York have climbed by 305,000 to 7.6 million, and by 730,000 to 13.4 million in California. Even New Jersey, which was among the states hardest hit and slowest to recover from the housing recession, has added 119,000 millionaires since 2010, bringing its total to 3.3 million. Rising 401(k) balances no doubt have driven much of this wealth creation.

Young and Varner’s study was based partially on the adoption of millionaire taxes by a growing number of states. It concluded that millionaire tax flight was occurring, but “only at the margins of statistical and socioeconomic significance.”

In fact, an analysis of 13 years of data found that Americans earning $1 million or more annually moved at a 2.4 percent rate versus 2.9 percent for the general population.

Their paper cited Nike founder Phil Knight’s prediction that Oregon’s millionaire tax would trigger “a death spiral” in which “thousands of our most successful residents will leave the state.” For reasons only they know, Laffer and Moore didn’t mention Oregon, which has a 9.9 percent state income tax rate, higher than many states they did mention. More importantly, it is located next to Washington, which has no income tax.

Young and Varner examined migration along the Washington-Oregon border, noting that moving between several counties on the state line is more akin to changing neighborhoods “within a city.” Because the change in lifestyle is so minimal, this area provides an excellent laboratory to examine tax flight. So do the Vermont-New Hampshire and North Carolina-Tennessee borders.

The Stanford academics find that “one-time millionaires” show no sensitivity to tax rates, but persistent millionaires, those who regularly earn $1 million or more, are the most responsive. Contradictory though it may sound, persistent millionaires have the lowest overall migration rates. “This supports the hypothesis that elite incomes have a strong place-specific component that ties millionaires to their home states,” Young and Varner conclude. “People who expect continuous flows of million-dollar income over time do not tend to move.”

They also suggest that many high earners are the “late-stage working rich,” with deep personal ties to their communities. Politicians in many high-tax states are acutely aware that their regions generate lots of wealth and they know inertia and proximity prompt wealthy citizens to seriously weigh the hassles of relocation.

It should be noted that Young and Varner completed their study in 2016, when millionaires could still deduct their state and local income taxes. We’re in a new world, so a new chapter still has to be written. From what I've seen anecdotically in the Tri-state region, more people are at least talking about moving than ever before.

But there is also a flaw in the Laffer-Moore analysis. It is the overt political bias in the states they single out. Other states of all political colors from Idaho to Iowa to Montana to Wisconsin to South Carolina to Maine sport top marginal state income tax rates of 7 percent or more.

Idaho, with a 7.4 percent tax rate on joint incomes above $22.086 (that's right), is the nation's fastest growing state.Physically, it's beautiful. Politically, it fails to fit the Laffer-Moore script.

Idaho's high rate on a relatively small income may be an outlier. But in many of these states, high earners hit the top tax rate at much lower income thresholds than their counterparts in the blue states Laffer and Moore cite. Just talk to a financial advisor with physician clients in South Carolina or Maine and they'll tell you these doctors are hardly thrilled with the new tax law that some see as penalizing the upper middle-class.

It often isn't practical for a 57-year-old professional or business owner with an established clientele to pick up and move, especiallly if they are eyeing retirement in 10 years or less. And if you are a farmer in Iowa paying the state 8.98 percent on the last dollar of earnings above the top threshold of $70,785, relocating isn't even an option.

At least Iowa is considering a reduction in state income taxes. It's a different story in New Jersey where newly elected Gov. Phil Murphy, a former Goldman Sachs partner with a nine-figure net worth, is struggling to bring back a millionaire's tax. Many of Murphy's fellow Democrats oppose the concept, arguing it will only accelerate tax flight.

If nothing else, the state tax laboratories are expanding.