It isn’t just an IRS official pleading the Fifth Amendment that’s slowing efforts to uncover the facts behind the Internal Revenue Service scandal. It’s Section 6103.

The tax code’s privacy protections -- designed to prevent disclosure of taxpayers’ information -- were mentioned 26 times in the first round of congressional hearings on the IRS, repeatedly stopping lawmakers’ inquiries. The constitutional protection against self-incrimination was invoked once.

Lawmakers investigating the tax agency’s scrutiny of small-government groups are finding their inquiries slowed as they navigate the few paths allowed by the privacy rules. The restrictions will make it difficult for Congress and the public to get a complete picture of what happened inside the IRS.

“Even normal routine oversight is really stymied,” said Dean Zerbe, a former tax aide to Republicans on the Senate Finance Committee. “There’s not nearly as much oversight as you would see in similar agencies, both by the press and by Congress, because of this opaqueness.”

Because of Section 6103, the IRS can respond only with generalities to accusations hurled at the agency by taxpayers or lawmakers. In cases where the IRS erred, its executives can hide behind the privacy shield.

Stifling Questions

In testimony at three hearings this month about IRS scrutiny of small-government groups, agency officials and the Treasury inspector general for tax administration invoked section 6103 to stifle questions. The Ways and Means Committee’s next hearing, set for June 4, will give the groups a chance to air their grievances about delays and intrusive questions from the IRS.

The law gives the chairmen of the Ways and Means and Senate Finance panels authority to request otherwise-private information about tax returns, such as what triggers an audit or how taxpayers are committing fraud.

Specific taxpayer information is available to the panels only in a closed session. The most common way in which taxpayer information becomes public is through court cases.

“Investigating the IRS was pretty difficult because of the great restrictions placed on our ability to use the information in a public way,” said Donna Flynn, staff director of the Ways and Means oversight subcommittee from 1995 to 1998.

The limits created by section 6103 became clear during the hearings this month. On May 17, Representative Aaron Schock of Illinois asked Steven Miller, then the acting IRS commissioner, to comment on questions received by anti-abortion groups. One was about whether the group educates on both sides of an issue and another was on the content of its prayers.

‘Pains Me’

“It pains me to say I can’t speak to that one either,” Miller said, though he said that as a general rule, he would be surprised by that type of question to a group.

Specific examples shape public perception, giving taxpayers an edge when engaging the IRS in a congressional hearing, said Joshua Blank, a tax law professor at New York University who supports current privacy rules.

“The IRS is not able to counter those arguments with similarly persuasive specific examples,” he said. The focus on IRS misdeeds is an unusual departure from the agency’s typical ability to shape its image by emphasizing prosecutions of tax cheats, he said.

Congress created Section 6103 in 1976, following revelations that President Richard Nixon had sought IRS audits of his political opponents.

While the tax system had some privacy protections, Section 6103 created a clearer set of rules in place of executive branch discretion. They included an assumption that taxpayers’ information is private without a specific exception.

Prison, Fines

Unauthorized disclosure of such information is a felony with penalties of up to five years in prison and a $250,000 fine. IRS employees also can be disciplined for violations.

Taxpayer privacy is ingrained in agency employees, and includes restrictions on using internal access to look up such things as the tax records of celebrities or their own daughters’ boyfriends, said Mark Matthews, a former IRS deputy commissioner and criminal investigation chief.

Senior IRS executives almost never look at individual tax returns, he said, explaining what he would say to friends inquiring about their tax bills or charitable contributions.

“My standard answer was, I’m about the last person on the planet who could ever look at your tax return,” said Matthews, who left the IRS in 2006.

Hand Tied

In congressional hearings, he said, the tax agency is in effect fighting with one hand tied behind its back.

IRS officials’ invocations of Section 6103 mirror the use of the Fifth Amendment right against self-incrimination, said Joseph Thorndike, a tax historian and contributing editor at Tax Analysts in Falls Church, Virginia.

“I suspect that mostly it’s used as a convenience for the IRS and as a genuine protective device for taxpayers,” he said. “Everybody assumes that it’s an admission of guilt, even when it’s not or even when it’s necessary.”

The rules provide a series of exceptions that allow state governments, law enforcement agencies and others to obtain taxpayer-specific information.

The Ways and Means and Finance chairmen rarely request taxpayer information, said Floyd Williams, the IRS chief liaison to Congress for 16 years before he retired in 2012. Other panels need resolutions from the House or Senate for access.

‘6103 Barrier’

The limits can frustrate congressional inquiries, said Zerbe, now national managing director at Alliantgroup LP.

“When you’re trying to find out what the IRS is doing and how they’re doing it and techniques, it’s very difficult to get much headway without running into a 6103 barrier,” he said.

The rules also frustrate the IRS, because the tax agency can’t tell its side of what seems like a taxpayer horror story.

“If the taxpayer does not give any kind of waiver, then the IRS is totally constrained, even if the stuff is in the newspaper,” said Williams, now a senior counsel at Public Strategies Washington Inc.

That was a hurdle during congressional hearings in the 1990s, Williams said, when taxpayers testified about agency abuses that he said were “trumped up.”

Ready Response

The privacy rules give IRS officials a ready response when confronted with taxpayers’ complaints. Afterward, upon request, they can explain more to the panel’s chairmen and staff members.

“It may give you some momentary relief in a situation like that, but it’s not like the internal game plan is let’s use 6103 and hide our sins,” said Matthews, now an attorney at Caplin & Drysdale in Washington. “Because somebody’s going to get it eventually.”

Every year, the IRS releases a report on the number of disclosures under section 6103. The 2012 total, including bulk data for research purposes, was 8.3 billion, with about 4.5 billion going to states, 2.4 billion going to congressional committees and 1.3 billion to the U.S. Census Bureau. The IRS made three disclosures to aid in investigations of terrorism.

The rules also apply to tax-exempt groups, which are at the center of the current controversy. Their applications for tax- exempt status aren’t public unless they are approved by the IRS. Donor lists filed as part of their tax returns are private.

Interpretations of the rules can vary.

Nonprofit Groups

At the Ways and Means hearing, Chairman Dave Camp asked about two separate incidents. In one, documents belonging to the National Organization for Marriage became public. In another case, the news organization ProPublica asked for and received confidential documents about nine politically active nonprofit groups, including one for Crossroads GPS, backed by Republican Karl Rove.

Miller said he thought the inspector general had determined the disclosures to be “inadvertent” and that an employee was disciplined for failing to follow correct procedures.

By contrast, J. Russell George, the inspector general, wouldn’t answer directly.

“I have to be very careful as to exactly how I respond, and whether or not I can even acknowledge publicly some of these revelations that you’re inquiring about,” he said.