(Bloomberg News) The subprime mortgage crisis isn't the only calamity Wall Street created that's upending the finances of U.S. states and cities.

For more than a decade, banks and insurance companies convinced governments and nonprofits that financial engineering would lower interest rates on bonds sold for public projects such as roads, bridges and schools. That failed promise has cost more than $4 billion, according to data compiled by Bloomberg, as hundreds of borrowers from the Bay Area Toll Authority in Oakland, Calif., to Cornell University in Ithaca, N.Y., quietly paid Wall Street to end agreements since 2008.

California's water resources department this year spent $305 million unwinding interest-rate bets that backfired, handing over the money to banks led by New York-based Morgan Stanley. North Carolina paid $59.8 million in August, enough to cover the annual salaries of about 1,400 full-time state employees. Reading, Pa., which sought protection in the state's fiscally distressed communities program, got caught on the wrong end of the deals, costing it $21 million, equal to more than a year's worth of real-estate taxes.

"It was brilliant, and it all blew up on me," said Brian Mayhew, chief financial officer of the Bay Area Toll Authority, the state agency that gave Ambac Financial Group Inc., the New York-based bond insurer that filed for bankruptcy this week, $105 million to end $1.1 billion of interest-rate agreements. The payments equal more than two months of revenue on seven bridges the authority oversees around San Francisco.

Budget Deficits

The termination payments to Wall Street firms come at the worst possible time. The longest recession since the Great Depression left states facing budget gaps of $72 billion next fiscal year, according to the National Conference of State Legislatures. U.S. cities saw their general fund revenue fall the most since at least 1986 in the budget year that ended June 30, according to the National League of Cities.

Wall Street banks and insurers peddled financial derivatives known as interest-rate swaps to governments and nonprofits that bet they could lower the cost of borrowing. There were as much as $500 billion of the deals done in the $2.8 trillion municipal bond market before the credit crisis, according to a report by Randall Dodd, a senior researcher on the Financial Crisis Inquiry Commission, published by the International Monetary Fund in June.

$4 Billion

Borrowers from New York to California are now paying to get out of agreements. Altogether, they have made more than $4 billion of termination payments to firms including New York- based Citigroup Inc., New York-based JPMorgan Chase & Co. and Charlotte, N.C.-based Bank of America Corp. since the beginning of 2008, according to a review of hundreds of bond documents and credit-rating reports by Bloomberg News.

In contrast to the subprime crisis, few taxpayers know anything about the cost of untangling municipal swaps. The only disclosure of payments to Wall Street often is buried in documents borrowers have to give investors when they sell bonds.

In many cases, firms getting payments aren't explicitly identified and government officials often don't call attention to payments made to cancel contracts. Many of the telephone calls and e-mails from Bloomberg News to dozens of government and nonprofit officials over the last eight months seeking comment on derivative transactions went unanswered.

'No Reason'

"Money that should be invested in students, classrooms and fixing infrastructure in Pennsylvania is instead lining the pockets of Wall Street," Jack Wagner, the state's auditor general, said in a statement in April after calling on lawmakers to ban swaps. "State and local governments must stop gambling with public money," he said.

In an interest-rate swap, two parties exchange payments on an agreed-upon amount of principal. Most of the swaps Wall Street sold in the municipal market required borrowers to issue long-term securities with interest rates that changed every week or month. The borrowers would then exchange payments, leaving them paying a fixed-rate to a bank or insurance company and receiving a variable rate in return. Sometimes borrowers got lump sums for entering agreements.

The swaps were popular because governments and nonprofits could pay a rate that was lower than what they would otherwise face had they sold conventional fixed-rate securities. The agreements backfired after the credit crisis broke out. While borrowers had to continue selling adjustable-rate securities under the deals, the payments made by Wall Street plunged and no longer were enough to cover the municipalities' own debt costs.

1990s Design

Banks and insurance companies such as New York-based American International Group Inc. started designing municipal swaps in the 1990s as derivatives trading on Wall Street soared. Derivatives are agreements whose value is derived from stocks, bonds, loans, currencies and commodities, or linked to specific events such as changes in interest rates or the weather. They were blamed in part for causing the global financial panic.

The financial manipulation was a boon for Wall Street. While banks got paid to underwrite municipal bonds for public projects, they were able to generate additional fees if the borrower used a swap with the transactions. Because the contracts were unregulated and privately negotiated, the profits that Wall Street booked were never disclosed.

"The basic idea from the bank's perspective is just to do a swap because that's where the money is," said Andrew Kalotay, head of the debt-management advisory firm Andrew Kalotay Associates Inc. in New York. "Look at all the fees they get."

Jefferson County

In Alabama, $5.8 billion of swaps Jefferson County used in a sewer-system financing in 2002 and 2003 produced $120.2 million in fees for banks, as much as $100 million more than it should have based on prevailing rates, according to James White, an adviser hired by the U.S. Securities and Exchange Commission. The derivatives, which pushed the home of the city of Birmingham to the brink of bankruptcy, led to a $722 million settlement with JPMorgan in November 2009 after an SEC probe and the conviction of a county commissioner who steered business to bankers in exchange for bribes.

The New Jersey agency that makes college-student loans and grants paid tens of millions of dollars when it canceled derivative agreements with banks led by UBS AG and Citigroup in January.

The deals by the Higher Education Student Assistance Authority date back to April 2001, when the agency was getting ready to sell $190 million of fixed-rate bonds. Paul Wozniak, a UBS investment banker, told a meeting of the authority's board in Trenton it could borrow more cheaply by using swaps rather than selling conventional tax-exempt bonds, according to minutes and a copy of his presentation obtained by Bloomberg News after a request to state officials.

Cost Covered

All the authority had to do to get the deal from UBS was to sell auction-rate securities, he said. The Zurich-based bank would help cover the cost of that adjustable-rate debt in exchange for annual fixed-rate payments from the authority, he said. The fixed rate was 4.65 percent, about a half percentage point less than the 5.18 percent the state would pay if it sold conventional bonds, he said.

The disadvantages were few, Wozniak told board members. The swap was a contract, so it would have to be footnoted in the authority's financial statements, he said. The state would have to count on getting periodic payments from UBS over the deal's life, he said.

"They found the swap agreements extremely complicated," New Jersey's former Inspector General Mary Jane Cooper said in a report in May after auditing the authority and interviewing board members who listened to Wozniak's pitch.

No Help

"The explanations were not particularly helpful," she said. In the end, they relied on recommendations made to them by management, according to the report.

The 18-member board, which consisted of college administrators and New Jersey government officials and students, approved about $1 billion of the deals over the next five years. The authority started exiting the contracts in January, making $49 million in termination payments, including $23 million to UBS and $17 million to Citigroup.

"Government operates with a very short-term mentality," said Matt Fabian, a senior analyst at Municipal Market Advisors in Westport, Connecticut. "There isn't much upside to look long term. They are looking for near-term savings on things."

AnnMarie Bouse, a spokeswoman for the authority, referred to Cooper's report, which included written responses from management.

Board-Member Action

"A board member's decision to rely on the recommendation of management where the underlying transaction remains unclear is a reflection on that particular board member, not necessarily an authority deficiency," Michael Angulo, the authority's executive director, wrote in the report.

Wozniak, who is chairman of Las Vegas-based education lender College Loan Corp. and left UBS in 2008, said he doesn't recall the meeting. "You wouldn't have done it if you wouldn't have thought it would save you money," he said in a telephone interview.

Douglas Morris, a UBS spokesman in New York, wouldn't comment, nor would Alexander Samuelson, a Citigroup spokesman.

New York Governor George Pataki was seeking ways to close an $11 billion budget deficit in 2003 when he embraced Wall Street's alchemy. The former governor included a provision in his spending plan that authorized all state agencies to use swaps, resulting in a total of $5.9 billion of the deals with firms such as Goldman Sachs Group Inc., based in New York.

Evaporated Savings

The state sold floating-rate securities to refinance existing fixed-rate bonds and then locked in lower fixed rates on the new debt using swaps. Before the credit crisis, officials said they had generated $203 million of savings. Since the crisis, unwinding the swap contracts has cost $247 million, according to the state budget office.

Pataki didn't return telephone calls and e-mails to his office at the New York-based law firm Chadbourne & Parke LLP seeking comment. Erik Kriss, a spokesman for the state's budget department, said the "swap portfolio will continue to show modest savings," in part because state officials are refinancing existing debt with lower fixed rates.

New York was among about 40 states that passed laws, often at Wall Street's urging, permitting municipal derivatives before the credit crisis, according to Dodd's research for the IMF. Tennessee passed rules in 2001 that required borrowers to attend a swap school.

Morgan Keegan Classes

Memphis-based Morgan Keegan Inc., a division of Birmingham, Alabama-based Regions Financial Corp., was selected to teach the classes. The firm sold many of the $12.7 billion of the deals subsequently done by more than 40 counties, municipalities, districts and authorities, according to Justin Wilson, the state comptroller.

"There's just no reason these entities should be playing with this stuff," said Christopher Whalen, managing director at the Torrance, California-based research firm Institutional Risk Analytics. "They don't have the capacity to evaluate these instruments. They are totally lost."

Just as banks loosened mortgage underwriting standards as part of the effort to create more subprime-linked securities, Wall Street targeted some of the riskiest credits in the municipal market with its swaps pitch. Nonprofit and government-run health-care providers, which pay higher tax-exempt interest rates because they have among the lowest bond ratings, accounted for 40 percent of the derivative deals, Standard & Poor's found in a study in 2007.