Policymakers are preparing to hold rates steady later this month.
Consumers are becoming more dependent on their paychecks to maintain their standard of living.
Younger workers and Blacks and Hispanics consider themselves most exposed.
Investors are pricing in the near certainty of a rate hike at the Fed's meeting next week.
"It would be a healthier economy if we were back towards 2%" inflation, monetary expert John Taylor said.
Workers have still not made up the ground they lost when prices surged coming out of the worst of the pandemic.
Traders in the money markets are betting that the Fed will raise rates one more time in July.
It's not clear how much unemployment should rise to achieve better balance in the labor market, Ben Bernanke said.
Their strong consensus is showing signs of splintering.
The financial markets remain fragile and stressed, the IMF said.