Millionaires are getting nervous and holding off on investing, according to the research firm Spectrem.

In fact, the Spectrem Millionaire Investor Confidence Index saw its biggest one-month drop in confidence in the index’s 13-year history.

The confidence of those who are not millionaires but merely affluent is also shaky, though a little bit better, the index says.

The confidence indexes are compiled monthly by Spectrem based on the investing practices of millionaires, those with at least $1 million in investable assets, and affluent investors, those with $500,000 to $1 million in investable assets. The index ranges from a high of 51, representing strong positive investor confidence, to -51, for a strong lack of confidence.

The index number for millionaires fell from 20 in April to 3 in May, the largest one-month drop ever recorded by Spectrem. The affluent investor index fell from 10 to 6 for the same months, leaving the affluent investors a little more confident than the millionaires, which is also unusual, according to Spectrem.

“Even though the stock market remains at near-record high levels, millionaire investors are becoming increasingly cautious,” said Spectrem President George H. Walper Jr. in a statement.

“This is likely due to growing concerns about the weakening political position of President Trump given recent controversies, the declining likelihood of substantive tax reform in the near term, as well as concerns about the recently submitted proposed federal budget,” he added.

“Although non-millionaires also recorded a drop in confidence, the fact they are slightly more confident now than millionaires is a strong indication that we may be entering a tumultuous period for investors,” Walper said.

Almost 40 percent of millionaire investors said they intend to avoid investing any additional funds in June, which is the highest percentage of investors saying they are staying on the sidelines since December 2013, Spectrem says.

Fifty-four percent of millionaire investors say the political environment is the main cause of their trepidation, far outweighing other concerns, says Spectrem.