Former billionaire entrepreneur Samuel Wyly’s “difference of opinion” with the Internal Revenue Service could turn an offshore fortune into an onshore fraud if the agency gets its way.

Wyly, 81, and the estate of his late brother Charles joined forces for a trial Wednesday against the IRS in Dallas, seeking to wipe out the agency’s $3 billion claim for back taxes in their Chapter 11 bankruptcy cases.

The Wyly brothers, who developed companies including arts and crafts retailer Michaels Stores Inc., hid stock offshore and made illegal trades for 13 years, taking in $550 million in illegal profit, a federal jury in Manhattan found in 2014. That case, a victory for the U.S. Securities and Exchange Commission, triggered demands for years worth of back taxes and penalties by the IRS and forced Wyly and his brother’s widow, Caroline “Dee” Wyly, into bankruptcy.

The dispute in the latest trial hinges on the validity of trusts set up on the Isle of Man by the Wylys starting in 1992. Lawyers for Sam Wyly and his sister-in-law claim the trusts were legitimate, while the IRS argues they were used to avoid taxes and finance lavish lifestyles for family members from Texas to Colorado.

“It’s not fraud,” the Wylys’ lawyer, Donald Lan, said in his opening statement at the trial. “We think there’s an honest difference of opinion over the law.”

IRS lawyer Cynthia Messersmith said the case was about lies and “deception” surrounding one of the biggest tax frauds in U.S. history.

Evade Taxes?

The offshore structure was specifically designed to evade taxes, Messersmith said in her opening statement. The profits were repatriated to the U.S. through a variety of schemes, she said, including the trusts’ purchasing of jewelry, art, furnishings and other luxury items for use by the Wylys. The trusts also financed loans to family members and gifts to the brothers’ children worth millions of dollars, Messersmith said.

Sam Wyly testified for about 20 minutes before being excused for the day, due to what his lawyers said was a combination of his age and health. During that time, Wyly described his childhood growing up in small-town Louisiana, where he had a paper route, played high school football and became an Eagle Scout. At his lawyer’s prompting, Wyly recited the Boy Scout oath and also the Boy Scout law.

“A scout is trustworthy, loyal, helpful, friendly, courteous, kind, obedient, cheerful, thrifty, brave, clean and reverent,” Wyly said. His testimony will resume Thursday.