Kansas Governor Sam Brownback, a Republican, and Maryland Governor Martin O'Malley, a Democrat, are among state officials who have called for eliminating or reducing the deduction at the state level. Brownback's proposal to eliminate the tax break stalled in the state legislature. O'Malley's plan to reduce the deduction for wealthy taxpayers led to protests from real estate agents, who said such a proposal would make it tougher for people to own a home.

At the federal level, the deduction is allowed for interest on mortgages up to $1 million for both primary and secondary residences. Only taxpayers who itemize can take the deduction. Some economists maintain that the deduction disproportionately benefits the wealthy and spurs people to take on more debt.

"Most economists don't like it," said Kim Rueben, senior fellow at the Tax Policy Center in Washington, a nonpartisan research group, noting that there is little evidence that the tax break spurs home ownership.

"We are effectively encouraging people to buy bigger homes and take out debt because we are allowing them to deduct interest," Rueben said.

David Albouy, assistant professor of economics at the University of Michigan, says the deduction along with subprime lending are responsible for pushing people into homes they otherwise couldn't afford. The decline in homeownership and, subsequently, the drop in the use of the deduction may be the home market righting itself, he said.

'Fuel the Bubble'

"The mortgage interest deduction helped fuel the bubble," he said. "As people were buying these giant houses, they were claiming a giant portion of their income in mortgage interest. Now they are losing that. They are now not billing the government indirectly for their risky investment in real estate."

The use of the mortgage interest deduction peaked at the federal level in 2007 with the tax break showing up on 40.8 million returns. That fell to 36.5 million for 2009, according to the IRS. The amount deducted declined from $491 billion to $421 billion for those tax years.

Preliminary data shows the number of returns on which taxpayers claimed the deduction increased slightly to 36.9 million in 2010. The amount of the write-offs fell to $387 billion.

"One of the most striking things about the itemized deductions in the last few years is the decline in the number of people claiming this deduction," said Matthew Gardner, executive director for the Institute on Taxation and Economic Policy, a Washington-based nonprofit research group. "You're seeing five million fewer returns between 2007 and 2009 claiming the deduction -- it is a real drop."